Abstract Purpose To address the feasibility, reliability and internal validity of natural language processing (NLP) for automated functional assessment of hospitalised COVID-19 patients in key International Classification of Functioning, Disability and Health (ICF) categories and levels from unstructured text in electronic health records (EHR) from a large teaching hospital. Materials and methods Eight human annotators assigned four ICF categories to relevant sentences: Emotional functions, Exercise tolerance, Walking and Moving, Work and Employment and their ICF levels (Functional Ambulation Categories for Walking and Moving, metabolic equivalents for Exercise tolerance). A linguistic neural network-based model was trained on 80% of the annotated sentences; inter-annotator agreement (IAA, Cohen’s kappa), a weighted score of precision and recall (F1) and RMSE for level detection were assessed for the remaining 20%. Results In total 4112 sentences of non-COVID-19 and 1061 of COVID-19 patients were annotated. Average IAA was 0.81; F1 scores were 0.7 for Walking and Moving and Emotional functions; RMSE for Walking and Moving (5- level scale) was 1.17 for COVID-19 patients. Conclusion Using a limited amount of annotated EHR sentences, a proof-of-concept was obtained for automated functional assessment of COVID-19 patients in ICF categories and levels. This allows for instantaneous assessment of the functional consequences of new diseases like COVID-19 for large numbers of patients. Key messages Hospitalised Covid-19 survivors may persistently suffer from low physical and mental functioning and a reduction in overall quality of life requiring appropriate and personalised rehabilitation strategies. For this, assessment of functioning within multiple domains and categories of the International Classification of Function is required, which is cumbersome using structured data. We show a proof-of-concept using Natural Language Processing techniques to automatically derive the aforementioned information from free-text notes within the Electronic Health Record of a large academic teaching hospital.
【저자키워드】 COVID-19, rehabilitation, Electronic health record, ICF, functioning, natural language processing, 【초록키워드】 Tolerance, exercise, reliability, feasibility, hospital, classification, Electronic health record, natural language processing, Validity, Quality of life, automated, disability, recall, work, language, information, disease, function, COVID-19 patients, Electronic health records, COVID-19 patient, employment, EHR, Functional assessment, life, precision and recall, natural language, walking, health record, Exercise Tolerance, Non-COVID-19, natural, Precision, internal validity, functional consequences, hospitalised COVID-19 patients, domain, categories, processing, proof, functions, material, Record, Cohen’s kappa, survivor, Moving, Mental, physical, Result, required, functional, assigned, category, reduction in, was obtained, Ambulation, automatically, functional consequence, hospitalised COVID-19 patient, instantaneous, numbers of patients, personalised, were assessed, 【제목키워드】 electronic patient record,