Pdf differential item functioning dif has been increasingly applied in fairness studies in psychometric circles. To take account of dif in order to retain precision of measurement, split of difitems into separate sample specific items has become a frequently used technique. Item characteristic curves in the reference and focal groups and differential item functioning dif effectsize used in the simulation model for the four kinds of dif. Differential item functioning analysis system difas. This includes the mantelhaenszel common logodds ratio, the breslowday test of trend in odds ratio. A general framework and an r package for the detection of. Differential item functioning analysis system show all. The irt dif graphing tool is an excel spreadsheet designed to help you visualize and evaluate the presence of differential item functioning dif based on item response theory irt. Measurement invariance and differential item functioning. Differential item functioning dif is when a test item favors or hinders a characteristic exhibited by group members of a testtaking population. Differential item functioning analysis system difas computes odds ratio estimates of differential item functioning, differential test functioning, and. As demonstration, a dataset from a 40item math test in a statewide testing program is analyzed by two widely. Tests for dif compare two or more groups of interest.
The differential item functioning analysis software penfield, 2005 and the easydif software gonz alez et al. Pmc free article broekman bfp, nyunt sz, niti m, jin az, ko sm, kumar r, ng tp. Multilevel generalized mantelhaenszel for differential. Differential item functioning dif has been increasingly applied in fairness studies in psychometric circles. The coronary artery risk development in young adults cardia study timothy j. Assessment developers design and construct questionnaires or tests including sets of items that measure, for example, cognition, personality traits, or political views.
Software for analyzing differential item functioning using the mantelhaenszel and. However, for rare events data, the maximum likelihood estimation method may be biased and the asymptotic distributions may not be reliable. Item response theory detects differential item functioning between healthy and ill children in qol measures. Read the options below and select the version of jmetrik that is appropriate for your computer. Simply select your manager software from the list below and click on download. Several methods have been proposed in recent decades for identifying items that function differently between two or more groups of examinees. This is the webpage for the handbook on differential item functioning.
Differential item functioning for minority examinees on the. The use of the term bias refers to various contexts, both statistical and social. Differential item functioning analysis with ordinal logistic regression techniques difdetect and difwithpar paul k. Erm software school of education uncg soe unc greensboro.
Differential item functioning shareware, freeware, demos. Stata module to detect and adjust for differential. Dif detection generally focus on the identification of uniform and nonuniform dif, where. A more robust method for evaluating item level bias is to test for differential item functioning dif, which takes into account differences at every level of candidate ability 16, 17. Judicious application of this methodology by the researchers, however, requires an.
This 2 12day minicamp will introduce participants to modern psychometric methods for developing and validating measurement instruments for evaluation, assessment, and research. Dif analysis investigates a differential characteristic of a test item between subpopulations of examinees and is useful in detecting possibly biased items toward a particular subpopulation. Rasch analysis with a focus on differential item functioning dif is increasingly used for examination of psychometric properties of health outcome measures. Identifies differential item functioning, creates dummyvirtual items to be used to adjust ability trait estimates in parscale, writes the code and data file needed to process the updated item list in parscale, runs parscale, and merges the ability estimates and standard errors into the original data set. Lewis, a note on the value of including the studied item in the test score when analyzing test items for dif.
Differential test functioning differential test functioning dtf is present when individuals who have the same standing on the latent construct or attribute, but belong to different groups, obtain different scores on the test the presence of dif may lead to dtf, but not always some dif items favour the focal group, whereas others. The purpose of the paper is to present and summarise recent advances of. Software for analyzing differential item functioning. If dif is found for many items on the test, the final test scores do not represent the same. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the. Differential item functioning on mathematics items using multilevel sibtest 475 population on a number of important demographic variables e. A handbook on the theory and methods of differential item functioning dif. Software for analyzing differential item functioning using the mantelhaenszel and standardization procedures show all authors. This module should be installed from within stata by typing ssc install difdetect.
Software for analyzing differential item functioning using the mantel haenszel and standardization procedures. Differential item functioning analysis system difas computes odds ratio estimates of differential item functioning, differential test functioning, and differential step functioning effects, along with associated tests of significance. Differential item functioning dif, as an assessment tool, has been widely used in quantitative psychology, educational measurement, business management, and insurance and healthcare industries. In this study, the performance of the regular maximum likelihood ml estimation is compared with two bias. Modifying measures based on differential item functioning. It is meant as a posthoc tool for analysis that is done with other software, such as dfit. X fits an item response model when x are item scores e. Users may download and print one copy of any publication from the public portal for. Differential item functioning sibtest assessing differential itembundle functioning difdbf sibtest implements a nonparametric estimation and hypothesis testing statistical method of assessing dif in one or more items andor dbf in one or more bundles of items. Recommendations for conducting differential item functioning. Analyze dif with specialized software like dfit or parscale. Detection of and adjustment for differential item functioning dif.
Dif makes it possible to test if the items measure the same trait dimension when administered to two or more distinct groups controlling for true group mean differences. Using differential item functioning to evaluate potential. For these items, a moderately positive relationship existed between item difficulty and the differential item functioning statistic, showing that black examinees performed. Cunningham, centers for disease control and prevention, 4770 buford highway ne, mailstop k22, atlanta, ga 30341 email. Pdf an introduction to differential item functioning. The logistic regression lr model for assessing differential item functioning dif is highly dependent on the asymptotic sampling distributions. Some of these procedures, such as the mantelhaenszel chi. Correlations between item difficulty and differential item functioning were generally low, with the exception of quantitative items in the blackwhite analyses. Differential item functioning dif is a statistical characteristic of an item that shows the extent to which the item might be measuring different abilities for members of separate subgroups.
Differential item functioning columbia university mailman. Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the same way for all subgroups. Bureau of the census projections for 2007 and the u. Identifies differential item functioning, creates dummyvirtual items to be used to adjust ability trait estimates, and calculates the ability estimates and standard errors. This illustration uses data from 5,429 children using the. We present an ordinal logistic regression model for identi. Pdf an introduction to differential item functioning researchgate. Assessment of differential item functioning in the experiences of discrimination index. A more robust method for evaluating itemlevel bias is to test for differential item functioning dif, which takes into account differences at every level of candidate ability 16, 17. Differential item functioning for minority examinees on. The module is made available under terms of the gpl v3.
Gibbons, phd, lance jolley, ms, and gerald van belle, phd introduction. Within the framework of the irt, differential item functioning dif is central to the investigation of the measurement equivalence of a scale at the item level. Differential item functioning analysis with ordinal. With the rising concerns over the fairness of language tests, differential item functioning dif has been increasingly applied in bias analysis. Differential item and test functioning difdtf, which is examined in the world. Dif analyses are statistical procedures used to determine to what extent the content of an item affects the item endorsement of subgroups of testtakers. Differential item functioning dif differential person functioning dpf, differential test functioning dtf detailed tables and plots of uniform and nonuniform dif size and significance. Differential item functioning dif has been widely used in healthcare, business management, and educational measurement. Judicious application of this methodology by the researchers, however, requires an understanding of the technical complexities involved. Differential item functioning analysis with ordinal logistic. Differential item functioning magnitude and impact measures. Sep 19, 2017 rasch analysis with a focus on differential item functioning dif is increasingly used for examination of psychometric properties of health outcome measures. Stata module to detect and adjust for differential item.
Differential item functioning magnitude and impact. Sep 01, 2017 this is the webpage for the handbook on differential item functioning. Burton, the effect of item screening on test scores and test characteristics. Differential item functioning dif analysis is a key component in the evaluation of the fairness and validity of educational tests. Free differential item functioning to download at shareware. Differential item functioning dif is a direct threat to the mi of test items and occurs when item parameters differ across equal ability groups, resulting in the differential likelihood of a particular e. Assessment of differential item functioning in the. Differential item functioning of the geriatric depression scale in an asian population. If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Apr 12, 20 differential item functioning dif is when a test item favors or hinders a characteristic exhibited by group members of a testtaking population. This course will provide a detailed overview of the highly related concepts. Current problems and future directions hossein karami, university of tehran, iran mohammad ali salmani nodoushan, iecf, iran. Dorans, evaluating hypotheses about differential item functioning. Windows users should not attempt to download these files with a web browser.
Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods. The purpose of the present analysis is to use differential item functioning dif to identify differences in the performance of native and immigrant students in pisa 2009 that can be directly related to their responses to particular items. Differential item functioning dif refers to group differences in performance on a test item that cannot be explained by group differences in the construct targeted. Measuring differential item and test functioning across. Software for analyzing differential item functioning using the mantel haenszel and.
We analyzed 95 cognitive reading items, administered to students in 29 european countries. Differential item functioning analysis system difas oit. Paper 29002015 multiple ways to detect differential item. Tilburg university differential item functioning and educational risk. This article provides an applied example using sibtest statistical software to detect dif in u. Differential item functioning dif is a statistical characteristic of an item that shows the extent to. We examined differential item functioning dif indicators for four variables that repeatedly have. Differential item functioning software free downloads. Item response theory analysis and differential item. Software for analyzing differential item functioning using the mantelhaenszel and standardization procedures. Psychometric tools difpack source and executable code by. Neither the list of the software nor the studies cited are meant to be. Differential item functioning sage research methods.
Differential item functioning dif is an important issue of interest in psychometrics and educational measurement. To date, the statistical software designed for assessing differential item functioning dif with mantelhaenszel procedures has employed the following statistics. An introduction to differential item functioning analysis eric. Differential item functioning between ethnic groups in the epidemiological assessment of depression.
Appendix statistical software for differential item functioning. Recent advances in analysis of differential item functioning. Department of educations national center for education statistics hussar. Logistic regression modeling as a unitary framework for binary and likerttype ordinal item scores. Item response theory detects differential item functioning. Jun 18, 2019 differential item functioning dif is a direct threat to the mi of test items and occurs when item parameters differ across equal ability groups, resulting in the differential likelihood of a particular e. Flexible application to many types of selectedresponse items.
A comparative study of the bias correction methods for. Stata module to detect and adjust for differential item functioning dif, statistical software components s4580, boston college department of economics. This article provides a brief primer overview of differential item functioning dif analysis. Table 30 supports the investigation of item bias, differential item functioning dif, i. Differential item functioning dif in composite health. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods, an r. Multilevel generalized mantelhaenszel for differential item. Windowsbased program that computes odds ratio estimates of differential item functioning, differential test functioning, and differential step functioning effects, along with associated tests of significance. A new approach for differential item functioning detection. Statistical software for differential item functioning analysis.
1663 944 939 147 1145 1265 490 1408 692 557 1593 1559 846 211 1422 106 141 398 150 1604 740 560 1291 556 1131 1191 1217 481 19 1156 975 1441 1112 1446 1072 774 571