AI-Enhanced Validation of Survey Instruments: Integrating Semi-Automated Methods in Cognitive Interviews for Children’s Self- and Proxy-Assessments of Health
 

Objective

The objective of the project is to enhance the efficiency of cognitive interviews through the integration of human expertise with innovative AI-supported coding methods. This integration will facilitate the expansion of the scope of the method. Additionally, the project contributes to the investigation of how children and parents assess child health in large-scale studies. To this end, the project will investigate potential differences in health assessment strategies between respondent groups and identify systematic variations by age and gender.

 

Background

To ensure high quality in survey research, survey instruments are validated with both qualitative and quantitative methods. However, both methods have their limitations. While quantitative evaluations efficiently establish generalizable associations between different concepts and systematic variances between respondents due to large sample sizes, their ability to comprehend the intricate cognitive processes within individuals is restricted. In contrast, qualitative methods such as semi-structured cognitive interviews offer deeper insights into participants' thought patterns. However, their high costs often limit sample sizes and scalability, affecting their overall validity, generalizability, and the ability to account for subgroup differences.

Integrating the strengths of both methods can significantly improve survey research by providing a holistic understanding of phenomena while taking advantage of quantitative validation. However, merging both approaches is a challenge.

The project AI-SIC contributes by developing an AI-supported approach for semi-automatic coding using an active learning framework. Machine coding algorithms are combined with human coding capabilities. Furthermore, new methods for efficient validation of survey instruments will be used to close research gaps in relation to the already established measurement of self-rated health. This will help to bridge the gap between qualitative and quantitative methodology and answer open questions about how children and their parents assess their children's health.

 

Approach

The project is divided into four work packages.

The first work package “Development of the semi-automatic coding framework InTraCo” (Dr. Andreas Niekler & Stephan Poppe; University of Leipzig) aims to integrate language-based machine learning and reliable semi-automatic coding procedures methodically and technically into the toolbox of computational social sciences. The aim is to combine machine coding algorithms and human inductive coding skills in order to increase the efficiency of coding extensive qualitative interview data.

The second work package “Deployment and Assessment of InTraCo” (Dr. Andreas Niekler & Stephan Poppe; University of Leipzig) is dedicated to the application of the newly developed approach as well as its validation and adaptation.

The third work package “Illuminating Children's Self- and Parents’ Proxy-Rating Strategies (Dr. Jacqueline Kroh; Leibniz Institute for Educational Trajectories) uses the highly complex data obtained and examines whether the new method can provide added value for content-related evaluations of individuals’ subjective health ratings using machine learning procedures. This will provide a more comprehensive understanding of how both children and parents assess children’s health.

The fourth work package “Comparability of Assessment Strategies and Outcomes between Children’s Self- and Parents’ Proxy-Ratings” (Prof. Dr. Julia Offenhammer-Tuppat; University of Leipzig) deepens the insights gained in work package three and examines similarities and differences between different respondent groups in the children's self-assessments and the parents' proxy assessments of children's health.

 

Data collection

AI-SIC uses qualitative data and conducts cognitive interviews based on web-based and real face-to-face interview.

 

Project Profile

  • Project management and application: Dr. Jacqueline Kroh (LIfBi), Dr. Andreas Niekler (Leipzig University), Dr. Stephan Poppe (Leipzig University), Prof. Dr. Julia Offenhammer-Tuppat (Leipzig University)
  • Project management at LIfBi: Dr. Jacqueline Kroh
  • Project duration: 07/2025 - 06/2028
  • Funding: German Research Foundation (DFG)
  • Link to this page: www.lifbi.de/AISIC
 
Project partners
Leipzig University