Psychological methods Research Program
The research program of the Psychological Methods group is concerned with the conceptual analysis, technical development, practical implementation, and dissemination of methodological and statistical techniques in psychology.
To improve the quality of empirical research is to enhance the validity of conclusions that are based on it. In methodology, validity refers to the question whether conclusions derived from empirical research are justified. In general, such justification requires a) the adequate conceptual specification of substantive theory, b) the translation of this theory in a formal model, c) the optimal implementation of this model in empirical research, and d) the proper interpretation of empirical results in terms of the theoretical concepts that figure in the substantive theory. The central objective of the research program is to improve methodological practice in psychology; not only by developing new methods for psychological research, but also by adequately communicating methodological advances to the general psychological audience, and by assisting working researchers in the proper application of methodological techniques.
The research program is concerned with the analysis of problems that occur in each of these steps. Because these problems are highly diverse, their resolution requires insights from different fields, most notably philosophy of science, statistics, and psychometrics. Thus, as was clearly recognized by the group's founder, Adriaan de Groot, methodology is an inherently multidisciplinary endeavor.
The research program in Psychological Methods effectively uses this multidisciplinary orientation for a cross-fertilization of ideas from these different fields. Thus, each research topic is approached from different angles, which are triangulated to arrive at novel perspectives on psychological research problems. At the same time, we strive for a high degree of integration of ideas from different fields, and emphasize collaboration between specialists in different areas. The current research lines center around several topics.
To improve the quality of empirical research is to enhance the validity of conclusions that are based on it. In methodology, validity refers to the question whether conclusions derived from empirical research are justified. In general, such justification requires a) the adequate conceptual specification of substantive theory, b) the translation of this theory in a formal model, c) the optimal implementation of this model in empirical research, and d) the proper interpretation of empirical results in terms of the theoretical concepts that figure in the substantive theory. The central objective of the research program is to improve methodological practice in psychology; not only by developing new methods for psychological research, but also by adequately communicating methodological advances to the general psychological audience, and by assisting working researchers in the proper application of methodological techniques.
The research program is concerned with the analysis of problems that occur in each of these steps. Because these problems are highly diverse, their resolution requires insights from different fields, most notably philosophy of science, statistics, and psychometrics. Thus, as was clearly recognized by the group's founder, Adriaan de Groot, methodology is an inherently multidisciplinary endeavor.
The research program in Psychological Methods effectively uses this multidisciplinary orientation for a cross-fertilization of ideas from these different fields. Thus, each research topic is approached from different angles, which are triangulated to arrive at novel perspectives on psychological research problems. At the same time, we strive for a high degree of integration of ideas from different fields, and emphasize collaboration between specialists in different areas. The current research lines center around several topics.
Mathsgarden/Rekentuin
MathsGarden (Or Rekentuin in Dutch) is a new web based training-tracking system to playfully train and measure mathematical skills. It builds on a long history of psychometric research at the department of psychogical methods.
The Psychosystems project
For more information, see: psychosystems
Measurement invariance
The concept of measurement invariance is concerned with the question under which circumstances observed scores can be used to make inferences regarding psychometric differences between groups. Mellenbergh pioneered the formalization of the theory of measurement invariance in several key publications in the '80s and early '90s. Since, the group has studied the concept of measurement invariance from the point of view of various IRT models as well as factor models. Hessen (2003) researched bias detection methods in various nonparametric and parametric models, and Borsboom, Mellenbergh, and van Heerden (2002) introduced a new distinction in types of measurement invariance with important implications for the relation between test validity and bias. Significant progress was further made by Conor Dolan and his co-workers, who introduced the study of measurement invariance through multi-group confirmatory factor analysis in the literature on Black-White differences in psychometric IQ-scores (Dolan, Roorda, and Wicherts, 2004), the study of the Flynn-effect (the worldwide IQ-gains that have been recorded across the past forty years; Wicherts, Dolan, Hessen, Oosterveld, van Baal, Boomsma, and Span, 2004), and pioneered applications in behavior genetics (Lubke, Dolan, and Neale, 2004). Dolan’s current research is funded by an NWO innovational scheme on the use of multi-group confirmatory factor analysis in the context of Black-White differences in IQ.
Psychometric theory
The research line on theoretical psychometrics has a conceptual and a mathematical part, both of which are embedded in a long standing tradition at the Psychological Methods group. At a conceptual level, the program strives to develop a better understanding of central test theoretical concepts such as measurement precision, bias, and test validity. This research stems directly from Adriaan de Groot’s theoretical work and leans strongly towards the philosophy of science.Borsboom’s research has largely taken place on this front, and has resulted in important results concerning the status of latent variables and the concept of validity (Borsboom, Mellenbergh, and van Heerden, 2003; 2004). The more mathematically oriented line of research originated with van Naerssen and was successfully continued in the work of Don Mellenbergh.In the past seven years, researchers in the Psychological Methods group have contributed significantly to progress in the field through the development of new Item Response Theory (IRT) models (Hessen, 2004), by working out the implications of existing models like the common factor model (Krijnen, 2004), and through the development of techniques for mixture modeling (Dolan, van der Maas, and Molenaar, 2002) and time series analysis with latent variables (Hamaker, Dolan, and Molenaar, 2003).
Intraindividual processes versus interindividual differences
In psychology, the statistical analysis of a majority of the empirical studies is based on population statistics, like population means and covariances. However, the majority of substantive theories are not about the structure of variables in the population, but about the structure of psychological processes that take place at the level of the individual. The question that arises is whether theories concerning intraindividual processes can be tested on data that are aggregated over individuals, like mean differences and covariance structures.In an important work, Molenaar, Huizenga, and Nesselroade (2003) suggested that the conditions under which this can be done are highly limited: In the context of structural equation modeling, they demonstrated a surprising lack of correspondence between the structure of intra- and interindividual differences. The lack of correspondence between these domains has formed the impetus for research conducted in the Psychological Methods group from various angles. Borsboom, Mellenbergh, and van Heerden (2003) studied the problem from a philosophical point of view, while Hamaker (2004) developed modeling techniques that can be used to conduct latent variable modeling at the level of the individual. The research on this topic culminated in Molenaar's (2004) manifesto, in which he called for an intraindividually oriented, yet highly formalized, type of idiographic psychology. In 2004, Borsboom received a NWO Veni-grant to study the relation between interindividual differences and intraindividual processes.
