您当前的位置: > 详细浏览

基于称名反应模型的CD-CAT选题策略比较研究

A Comparative Study of Item Selection Methods in CD-CAT based on Nominal Response Model

摘要:当前大多数CD-CAT有关的研究都是基于0-1计分的数据资料展开的,而在实际的教育与心理测验中,还包含大量称名反应数据。本文基于称名反应认知诊断模型(NR-cRUM)开发了适用于称名反应数据的CD-CAT(以下简称NCD-CAT),并将7种0-1计分CD-CAT的选题方法引入NCD-CAT中。比较不同条件下,不同选题方法对被试判准率和测验效率的影响。结果表明NR_PWCDI、NR_MPWKL等PWKL系新方法和NR_SHE/MI方法能较好地适用于NCD-CAT,且在大多数条件下优于基线方法NR_PWKL。研究拓展了称名多级计分CD-CAT的选题方法。

英文摘要:Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) combines the advantages of cognitive diagnosis and CAT, which could improve the efficiency and accuracy of CD-CAT. CD-CAT can be divided into two types: dichotomous and polytomous. Presently, the majority of researches on CD-CAT are based on dichotomous CD-CAT. However, among the practical tests in psychology and education, there are many polytomous items, which can be further divided into nominal polytomous and ordinal polytomous items according to whether there is an order or grade between every response category. Nominal polytomous items are items whose response categories are independent and without orders or grades between every response category. Although researchers have developed (ordinal) polytomous CDMs and corresponding CD-CAT, few nominal CDMs and CD-CAT are based on nominal responses. This study introduces seven commonly used item selection methods in dichotomous CD-CAT into NCD-CAT (CD-CAT based on nominal response models). PMR (pattern match ratio) and test efficiency index are evaluated under different conditions between these item selection methods. Here are details of two simulation studies below. Study 1 compared the performance of NR_PWKL, NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, NR_MI, and NR_GDI methods under different test lengths (5, 10, 15, 20) and item pool qualities (high and low) in NCD-CAT. Results showed that: (1) the PMRs of NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI are higher than or equal to that of NR_PWKL, especially in short tests. (2) as test length gets longer, that PMR advantage is missing, which is the same as the results of Zheng and Chang (2016). (3) compared to test length, item quality has a greater impact on PMR. For instance, with item quality descending, the PMR declined about 30% among all conditions. Study 2 is an experiment on variable-length NCD-CAT that was conducted to compare the performance of each item selection method under the conditions of three maximum posterior probabilities (0.8, 0.85, 0.9) and two item qualities (high and low). The results showed that: (1)under all experimental conditions the average test lengths of NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI are shorter than that of NR_PWKL; the difference is more than 0.738. (2)affected by item quality, the average length of NR_GDI is smaller than that of NR_PWKL under high-quality conditions and larger than it under low-quality conditions. To sum up, this study compared the performance of 7 commonly used item selection methods of dichotomous CD-CAT in NCD-CAT with different conditions (fixed and variable length). The simulation study showed that under most conditions, the NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI methods performed well when compared to baseline algorithm NR_PWKL. This study has expanded the alternatives of item selection methods in NCD-CAT.

版本历史

[V2] 2022-05-12 16:44:20 chinaXiv:202205.00030V2 下载全文
[V1] 2022-05-08 11:34:10 chinaXiv:202205.00030v1 查看此版本 下载全文
点击下载全文
同行评议状态
待评议
许可声明
metrics指标
  • 点击量6142
  • 下载量373
评论
分享
邀请专家评阅