Process-Based Diagnostics of Text Comprehension With Multiple Documents
 

Aim

The aims of the project were: 

  1. Developing a theoretically informed computer-based test of Multiple Documents Literacy
  2. Empirically testing hypotheses of theories on MDL (Multiple Documents Literacy)
  3. Analyzing processes of MDL and developing process indicators for further diagnostics
  4. Validating interpretations of test values based on two starting cohorts at two locations and in two disciplines

MultiTex was a joint project of the Leibniz Institute for Educational Trajectories, the University of Bamberg and the German Institute for International Educational Research (DIPF).

 

Background

In order to cope with the demands of their studys, university students of all disciplines need to be able to deal with information in their field in a self-regulated way in order to expand their competencies. In learning, for example, they usually have to work with multiple sources and (text) documents that may contain redundant, complementary, or even contradictory information. In doing so, they must not only understand individual texts, but also appropriately relate and integrate information from these multiple documents and sources. However, previous studies indicate that students often fail precisely at this complex integration task.

 

Approach

For the MDL test, three to five units with two to three texts per unit were designed. Each unit contained 5 - 15 items. The MDL test aimed to represent the theoretically described range of requirements of text comprehension with multiple documents and also to generate process or protocol data. The CBA ItemBuilder (Rölke, 2012) was used for the computer-based implementation of the test.

The MDL test, which was also constructed using data from a development study, was subsequently used in a validation study at the Universities of Bamberg and Frankfurt. In this study, (cross-sectionally) both first-year students (bachelor in the 1st semester) and final-year students (master in the last semester) were examined in order to obtain evidence for the change sensitivity of the test. In addition, the relationship of classical reading literacy (assessed with a computer-based administered reading literacy test from the National Educational Panel) and MDL was examined. Other variables such as motivation, study success, and epistemological beliefs were also recorded to provide further evidence of convergent and discriminant validity and, if necessary, to identify subsamples with regard to suboptimal expression of MDL or the strategies implemented.

 

Results

The project developed and validated a computer-based instrument for students to assess their competence in understanding multiple documents (multiple documents literacy, MDL). A particular focus was on additional process-based measures that were used as 1) additional diagnostic information and 2) to identify processing strategies when dealing with multiple documents and to empirically test theories of MDL.

For further use in practice at universities, the transfer project MultiTex-Transfer was applied for and approved following MultiTex. Here, the test is to be made available to students of the humanities and social sciences as an aid in the introductory phase of their studies.

Access to the MultiTex test instrument for researchers

The test instrument for assessing students' comprehension of multiple documents can be used free of charge by researchers for scientific research questions. The download is provided by RDC Education and is possible after registration and application and after the authors' consent. The purpose of use should be described in the application. Further conditions are regulated by the test utilisation concept.

Download
 

Project profile

 
Project partners
University of Bamberg
DIPF Leibniz-Institut für Bildungsforschung und Bildungsinformation
 

Publications

2024

Schoor, C., Zink, T., Mahlow, N., Hahnel, C., Deribo, T., Kroehne, U., Goldhammer, F., Naumann, J., & Artelt, C. (2024). Diagnostik des Leseverständnisses multipler Texte: MultiTex. In T. Richter, & W. Lenhard (Hrsg.), Diagnose und Förderung des Lesens im digitalen Kontext (S. 101-116). Hogrefe. https://doi.org/10.1026/03256-000

2023

Schoor, C., Hahnel, C., Artelt, C., Goldhammer, F., & Kröhne, U. (2023). Testinstrument: Multiple document comprehension-test (MDC-Test) [Testinstrument: Version 1.0]: Erstanwendung 2017. Forschungsdatenzentrum Bildung am DIPF. https://doi.org/10.7477/506:316:57

2022

Mahlow, N., Hahnel, C., Kroehne, U., Artelt, C., Goldhammer, F., & Schoor, C. (2022). The role of domain-related epistemic beliefs for mastering cognitive requirements in multiple document comprehension. Learning and Individual Differences, 94(3), Article 102116. https://doi.org/10.1016/j.lindif.2022.102116
Schoor, C., Zink, T., Mahlow, N., Hahnel, C., Deribo, T., Kroehne, U., Goldhammer, F., & Artelt, C. (2022). Das Textverstehen von Studierenden beim Lesen multipler Dokumente und dessen Förderung. In S. Alker-Windbichler, A. Kuhn, B. Lodes, & G. Stocker (Hrsg.), Bibliothek im Kontext: Akademisches Lesen – Medien, Praktiken, Bibliotheken (5, S. 57-86). Vienna University Press / V&R unipress. https://www.beck-shop.de/alker-windbichler-kuhn-lodes-stocker-akademisches-lesen/product/33347680

2021

Hahnel, C., Goldhammer, F., Kroehne, U., Mahlow, N., Artelt, C., & Schoor, C. (2021). Automated and controlled processes in comprehending multiple documents. Studies in Higher Education, 46(10), 2074-2086. https://doi.org/10.1080/03075079.2021.1953333

2020

Mahlow, N., Hahnel, C., Kroehne, U., Artelt, C., Goldhammer, F., & Schoor, C. (2020). More than (single) text comprehension? On university students’ understanding of multiple documents. Frontiers in Psychology, 11, Article 562450. https://doi.org/10.3389/fpsyg.2020.562450
Schoor, C., Hahnel, C., Artelt, C., Reimann, D., Kröhne, U., & Goldhammer, F. (2020). Entwicklung und Skalierung eines Tests zur Erfassung des Verstehens multipler Dokumente von Studierenden. Diagnostica, 66(2), 123–135. https://doi.org/10.1026/0012-1924/a000231
Schoor, C., Hahnel, C., Mahlow, N., Klagges, J., Kröhne, U., Goldhammer, F., & Artelt, C. (2020). Multiple document comprehension of university students: Test development and relations to person and process characteristics. In O. Zlatkin-Troitschanskaia, H. A. Pant, M. Toepper, & C. Lautenbach (Eds.), Student learning in German higher education: Innovative measurement approaches and research results (pp. 221–240). Springer VS. https://doi.org/10.1007/978-3-658-27886-1_11

2019

Hahnel, C., Mahlow, N., Schoor, C., & Cerdán, R. (2019). Promoting reading goals and strategy knowledge to improve university students' comprehension of multiple documents (KoKoHs Working Papers No. 12). http://www.kompetenzen-im-hochschulsektor.de/index_ENG.php