Deriving Cut-off Scores for Significance Testing of NCDIF Indices Within the DFIT Framework Through Item Parameter Replication Method

Master Thesis


Differential item functioning (DIF) deals with how test items perform in different demographic groups. Although many operationalizations of DIF are proposed, no single approach proves to be exclusively superior. The current study investigated the properties of DIF indices in the differential functioning of items and tests (DFIT) framework and further tested the power of item parameter replication method (IPR) in identifying biased items within the item response theory (IRT) paradigm. The results indicated the IPR method is a useful tool in DIF detection. The source of item parameters covariance structure was found to affect the results only slightly. The results indicated that the proportion of biased items on the test might not be as influential factor as the actual number of biased items. This finding, along with low false negative rates when 99.9th percentile cut-off scores is used, supports the view that sequential elimination of biased items is preferable over the simultaneous detection.

download pdf

Defended on August 8, 2008, in partial fulfillment of the requirements for the degree of Master of Arts, in the Quantitative Psychology program of Middle Tennessee State University.

Table of Content

I. Introduction

  1. Differential Item Functioning (DIF): Concepts and Definitions
  2. Theoretical Context: DIF Classifications and IRT Basics
  3. DFIT framework and its indices
  4. Significance testing of NCDIF
  5. Oshima, et al. (2006) study
  6. The Purpose of the Current Study

II. Method

  1. Generating Data for Original Conditions
  2. Item Parameter Replication Method
  3. Generation of response data
  4. IRT Analysis
  5. NCDIF computation

III. Results

IV. Discussion


Andriy Koval
Assistant Professor of Health Management and Informatics

I am a data scientist with background in quantitative methods and interest in data-driven models of health and aging