YEAR 2020

Back to news archive

LIfBi Lecture on the Forced Choice Approach: "An Effective Strategy to Reduce Falsification"

12/7/2020

Rating scales have a long tradition in questionnaire construction - but they also have weaknesses, especially when it comes to falsification and extreme response styles. With her LIfBi Lecture at the end of November Prof. Dr. Eunike Wetzel from the University of Koblenz-Landau explained to what extent the "Multidimensional Forced Choice Approach" represents a helpful alternative.

 

Rating scales, which are often used in the educational, social and behavioural sciences, record a study participant's agreement with given statements by means of a scale, e.g., from "completely agree" to "completely disagree". The weaknesses of such rating scales (RS) come into play when the test subjects tend to answer in styles such as extreme or middle crosses or to intentional falsifications, for example in the form of socially desirable answers. In her LIfBi Lecture, Wetzel presented an alternative approach to surveying personality traits: The Multidimensional Forced Choice (MFC) approach is said to be less susceptible to formal response styles as well as to intentional faking.


In a questionnaire designed with the help of the MFC, the test person is presented with a set of different statements, for example about personality traits, which have to be put in order. With each new set, the test person must therefore repeatedly make a choice and weigh the statements against each other in terms of content. According to Wetzel, this is the advantage of the MFC approach: "In our ranking task, it is not possible to falsify all characteristics simultaneously and to the same extent." The MFC format thus represents a suitable way to reduce falsification in surveys, she said.


If the data collected by means of MFC are evaluated with classical methods, so-called "ipsative measured values" are produced, which allows no comparability between persons, but only an intra-individual analysis of the queried characteristics. In order to be able to conduct inter-individual comparisons, the evaluation of MFC data is therefore preferably carried out on the basis of a new statistical procedure, the so-called Thurstonian Item Response Model (T-IRM). It allows the derivation of normative trait estimates and thus makes interindividual comparisons possible.

To this end, Eunike Wetzel presented a statistical simulation and two studies that examined the hypothesized advantages of MFC over RS.

The statistical simulation (Frick, Brown, & Wetzel, under review) examined the extent to which evaluation of the MFC format with the T-IRM leads to normative estimates of person ability compared to classical evaluation. Person estimators from the T-IRM showed no bias (e.g., in trait correlations) in contrast to classical test scores, except in the condition with only positively polarized items. The MFC format evaluated with the T-IRM showed similar results to RS in most conditions, with RS allowing more accurate estimates overall.

The first study (Wetzel & Frick, 2020) used empirical data from nearly 1,000 individuals to compare the MFC and RS formats on a personality instrument measuring the so-called "Big Five" (extraversion, conscientiousness, agreeableness, openness, neuroticism). The agreement of the measurements (convergent validity) for the traits extraversion, openness and neuroticism were satisfactory, but worse for the traits conscientiousness and agreeableness.

The second study (Wetzel, Frick, & Brown, 2020) examined whether the MFC format reduces socially desirable responses. To do so, subjects had to complete the Big Five instrument (in either MFC or RS format) twice each. The first time, they were asked to give honest answers; the second time, they were asked to present themselves as positively as possible. It was found that the difference between the scores (2nd - 1st measurement time point) was smaller for the MFC format than for the RS format. This provided an indication that the MFC format makes intentional faking more difficult: "The MFC format is not faking-proof, but it is an effective strategy to reduce faking," said the co-leader of the Diagnostics Research Group at the University of Koblenz-Landau.

In her summary, Eunike Wetzel came to the preliminary conclusions that the cost-benefit aspect must be weighed with regard to the MFC format. On the one hand, she said, there is a more difficult questionnaire construction that requires more items, as well as the IRT evaluation procedure that allows normative trait estimates. "On the other hand, the MFC format offers important benefits that may outweigh the costs, such as eliminating extreme response styles and reducing intentional bias," Wetzel said.