Hybrid Natural Language Processing Framework for Quality Assessment and Normalization of Clinical Data with Human-in-the-Loop Validation
DOI:
https://doi.org/10.18687/LACCEI2026.1.1.1045Palabras clave:
Data Quality, Natural Language Processing, Normalization, Fuzzy Matching, Human-in-the-LoopResumen
Data quality is a problem in healthcare information systems, especially when the capture of diagnostic information is not standardized. In this article, a hybrid computational framework for clinical data normalization is designed and evaluated, combining deterministic text processing methods with fuzzy similarity algorithms and a human-in-the-loop validation mechanism. The proposed system was implemented and tested on 2,777 outpatient care records from the Villa Nueva Health Center in Honduras between January and December 2024. The proposed multi-level normalization pipeline reduced the cardinality of unique diagnoses by 55.56% (from 90 to 40 categories) and geographic locations by 30.07% (from 153 to 107 categories), with 99.46% of mappings achieved by exact match (Level 1). Diagnostic entropy decreased from 3.06 to 2.55 bits (16.65%), while the Herfindahl–Hirschman index increased by 20.91%, indicating greater uniformity in the frequency distribution. The ranking stability analysis yielded Jaccard indices of 0.667 and Kendall’s τ coefficients of 0.929 for the top 10 diagnoses. Thirteen ambiguous cases (0.47%) were identified that required manual review, demonstrating the feasibility of the semi-automated method. The temporal drift analysis showed an average vocabulary stability of 43% (Jaccard) between successive months. The proposed architecture is replicable, scalable, and exportable as a programd pipeline, providing a practical solution for data governance in epidemiological surveillance systems.Descargas
Publicado
2026-07-27
Número
Sección
Articles
Derechos de autor
Derechos de autor 2026 LACCEI
Licencia
Esta obra está bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional.
LACCEI conserva el copyright de todos los artículos publicados bajo los términos de su acuerdo de transferencia de copyright. Como titular del copyright, LACCEI distribuye los artículos al público bajo la Licencia Internacional Creative Commons Atribución-NoComercial-CompartirIgual 4.0 (CC BY-NC-SA 4.0).
Cómo citar
Madrid, M., Agudelo-Santos, C., Giacaman, L., Simons, P., Madrid, M., & Argueta, E. (2026). Hybrid Natural Language Processing Framework for Quality Assessment and Normalization of Clinical Data with Human-in-the-Loop Validation. LACCEI, 1(14). https://doi.org/10.18687/LACCEI2026.1.1.1045