Separating Systematic Measurement Error Components Using MTMM in Longitudinal Studies

Short Description: In this project we develop a new research design that enables us to measure and disentangle multiple types of error: method, social desirability, acquiescence (tendency to select first category) and random error. Furthermore, we investigate how these errors change in time and how they compare cross-culturally.

Methodological Details: The design can be described as an extended split-ballot multitrait-multimethod (MTMM) design. Participants were randomized to 56 different groups who received different self-report items on the topic of immigration. The following experimental factors were manipulated: number of scale points (2 point or 11 point scale), socially desirable direction (positively or negatively formulated item on immigration), and acquiescence direction (agree-disagree or disagree-agree scale). This yielded 2×2×2 = 8 possible item wordings (treatments) for each of the items. Although there are 8 treatments, structural equation models depend only on the pairwise covariances of the variables, so that only combinations of pairs of wordings are required. For each pair of questions, the first format was presented early in the questionnaire, and the second format at the end of the questionnaire, with at least 20 minutes of other questions in-between. There were 28 possible pairs of question formats. The order of presentation within pairs was randomized, resulting in 56 groups. However, the two orderings of each pair can be combined, so that each covariance between observed variables can be calculated from n/28 (instead of n/56) observations.

Find the proposal for the module here

Contact	Syear	Respondents	Dataset	Variables	Availability	Field	Method	Replication
Alexandru Cernat (University of Manchester), Daniel Oberski (Utrecht University)	2015	~1000	inno	w54a, w24a, w65b, w64a, w72a, w13a, w26b, w71a, w42b, w81a, w83a, w85a, w25a, w86a, w22a, w56b, w31a, w15b, w46a, w11a, w36b, w55b, w16a, w23a, w82a, w46b, w13b, w22b, w43a, w76a, w53a, w32a, w21a, w25b, wsplit, w45b, w16b, w15a, w82b, w34a, w86b, w14b, w11b, w21b, w12a, w33a, w56a, w74a, w35a, w31b, w44a, w51a, w75b, w66b, w53b, w73a, w41a, w33b, w63b, w62a, w61b, w61a, w63a, w42a, w55a, w14a, w52a, w54b, w52b, w72b, w23b, w84b, w75a, w73b, w85b, w44b, w36a, w34b, w74b, w43b, im_mtmm, w84a, w35b, w26a, w66a, w41b, w12b, w51b, w76b, w45a, w32b, w62b, w83b, w64b, w24b, w81b, w71b, w65a	04/2018	Statistics	Experiment	2014,2016