TAP CHi KHOA HOC DHSP TPHCM Nguyin Phir&c Hdi vd tgk x/PHAN TICH VA LITA CHON CAU HOI T R A C NGHIEM KHACH QUAN DU''''A T R | ; N BANG SP , PHAN TICH QUAN HE XAM VA DUOfNG CONG ROC NGUYEN PHUdC HAI", Dy[.]
Trang 1x/PHAN TICH VA LITA CHON CAU HOI T R A C NGHIEM KHACH
QUAN DU'A T R | ; N BANG S P , PHAN TICH QUAN HE XAM
VA DUOfNG CONG ROC
NGUYEN PHUdC HAI", Dy T H 6 N G N H A T "
TOM TAT
Muc dich ciia bdi viet ndy Id di xudt phucmg phdp phdn tich vd lua chgn cdu hoi trdc nghiem khdch quan (TNKQ) dua trin bdng S-P, phdn tich quan he xdm vd du&ng cong ROC Kit qud nghien cicu cho thdy phuang phdp ndy khong chi giup gido vien cdi thien chdt luong vd hiiu qud ciia viec thiet ki cdu hdi TNKQ md cdn sie dung di xdy dung ngdn hdng cdu hoi TNKQ
Tit khda: cau hoi yac nghiem khach quan, bang S-P, phan tich quan he xam, duong
cong ROC, ngan hang cau hoi
ABSTRACT
The analysis and selection of objective test items based on S-P chart,
Grey Relational Analysis, and ROC curve
The purpose of this paper is to propose the analysis and selection method of objective test items based on S-P chart Grey Relational Analysis and ROC curve Results showed that this method not only helps teachers to improve the quality and efficiency of designing objective test items buf also can be used to build an objective test item bank Keywords: objective test items, S-P chart, grey relational analysis, ROC curve, item
bank
1 Gioi thieu
Viec sir dung ngan hang cau hoi va thi TNKQ hien dang dupc cac truang khuyen khich, tuy nhien da s6 cac cau hoi TNKQ do giao vien ty bien soan co the chua theo dung quy trinh, dac biet cac cau hoi sau khi sir dung thuang khong duoc phan tich, danh gia nen cac d8 thi hien nay la chua tot va chat luong chua cao Ket qua nghien cuu ciia bai viet nay se la tai lieu can thiet gop phan vao qua trinh cai thien chat lupng
va hieu qua ciia viec bien soan va thilt ke de thi TNKQ, dong thoi gop phan nang cao
kT nSng cua giao vien trong viec thi6t ki cau hoi TNKQ diing de kiem tra danh gia ket
qua hpc tap ciia hpc sinh (HS) Sau day la phan gioi thieu so lupc ve bang S-P, phan ti'ch quan he xam va duong cong ROC
Nam 1969, Sato da de xuat bang S-P (Student-Problem Chart), no dugc sir dung khong chi de chan doan va danh gia trong hpc tap, ma con gop phan nang cao hieu qua
' GV, Tru'6'ng Cao d^ng S u phgm Kign Giang, Email- phuochai1979@gmail.com
" NCS, Tru'dng Ogi hpc S u phgm Qu6c lap O^i Trung, Dai Loan
Trang 2trong giang day Bang S-P cung cap thflng tin vfl he so chti y cua HS va he sfl chu y ciia cau hoi [8] Nam 1982, Deng da dfl xult li thuyfa he thflng xam (Grey System Theory) trong dfl phan tich quan he xam (Grey Relational Analysis - GRA) la mpt trong nhiing cong cu toan hpc dugc sir dung rit hiflu qua [2] Chirc nang cua no la dfl tinh toan cac sfl lieu roi rac va dinh lugng cac nhan tfl thflng qua slp xflp trinh tu dfl giai quyfa cac mfli lien he phiic tap gitia cac nhan tfl [9], [10] Trong nhQng nam gin day, GRA da dugc sir dung ttong rit nhiflu Hnh vuc, dac biet la frong Hnh vuc giao dye [4], [5], [6J, [7], [8] Ducmg cong ROC (Receiver Operating Characteristic) co ngufln gflc tir linh vuc quan su, no dugc ung dung frong viec phat hifln tau ciia djch fren man hinh radar trong Thfl chifln thit 2 [1] Duong cong ROC da dugc inig dung chan doan va tien lugng trong y hpe rit thanh cong [3] Trong nhiing nam gan day, ductag cong ROC ciing dupc su dung trong ITnh vuc giao due dfl phan tich, chan doan va danh gia trong qua frinh day hpc [5], [6] Hifln nay cac li thuyet ve bang S-P, phan tich quan he xam
va duong cong ROC chua dugc sir dung phfl bien o Viet Nam, dac biflt la dimg de phan
tich, chan doan va danh gia frong giao due
Trong bai viflt nay, nguai nghifln ciru sir dung kflt hpp bang S-P, phan tich quan
he xam va duong cong ROC dfl phan tich va lua chpn cau hfli trac nghiflm Phuong phap nay da dupc sir dung dfl phan tich sir nhan thuc sai (misconception) cua HS frong qua trinh hgc tap [6], [7] Hon ntta, ngucri nghifln ciru cfln sir dung phan mem MATLAB dfl thifa kfl mot hop eong cu MATLAB cho phan tich va lua chpn cau hoi trie nghiem Hop cflng cu MATLAB giup cho qua frinh tinh toan dl dang, nhanh chong, chinh xac, hien thi kflt qua va hinh anh tren giao difln dfl hga nguoi diing mpt each true quan sinh dpng
2 Cff SOf li thuyet va phirong phap nghien cuu
2.1 Bdng S-P (Student-Problem Chart)
Bang S-P dugc dfl xuit boi Takahiro Sato vao nam 1969 No thuong dimg dfl sjp xep, phan tich va phan lo^i kflt qua hpc tap cua hge sinh va eau hfli trie nghiem dua trfln he sfl chu y ciia hgc sinh (CS) va he sfl chu y eua cau hoi (CP) [8]
Bangl Bdng S-P
^ ^ " ^ ^ g a u hoi (CH)
H9C sinh ( H S ) ^ ^ ^ ^ ^ ^
Sfl hgc sinh
S„i=\,2, ,m
Tflng sfl HS tra lfli dung
CP
So cSu hoi
P,,j = \,2, ,n
^ = K U
Nhiflu <-• it
CP,
Tong diem
Cao
t
Thap
-CS
CS,
Trang 3
-Trong bang S-P, X la ma tran cfl m hang va n cgt, trong do x,, = 1 nflu HS tra loi
dung CH va x,= 0 nflu HS tra Ioti sai CH Sfl hpc sinh la S.,i=lZ-;m- sfl cau hoi la
Pj,j = \,2,-,n
He sfl chu y cua HS dupc tinh bing cong thiie sau:
cs,=i-^=ijv — (1)
T(.x )-(xi,)(x)
7=1 ^
7=1 '' '" 7=1
He so chu y cua CH dugc tinh nhu sau:
ttong dfl ;c = - i :t vi N = x,, - E x„
n j=\ J I
-I ( A : , - , ) ( X , ) - ( ; C )(x')
^^y^'-^nsr — (2)
J.(Xi.)-<.x,j)(x')
, _, 1 H , "
ttong do x'= — Z Xj va M = JT,, = X x,,
mi=l " '} ,-=1 'J
Trong bai viflt nay, nguflfi nghien cuu ehi sir dung he sfl chii y cua CH dfl kfa hpp
vflfi gia fri Gamma va gia tri A UC phan tich va iua ehgn cau hfli TNKQ
2.2 Phan tich quan he xdm (Grey Relational Analysis)
Nguflfi nghifln cim chi sir dung phan tich quan he xam dua theo gia tri lcm nhit
(Lager-the-Better) dfl lam vector tham khao x^ frong nghien cim nay [9] Phan tich
quan he xam dupc tinh toan nhu sau:
Dua fren dQ liflu tho tir bang S-P de thiet lap vector Xg, vector x^ la gia tri lcm
nhat o moi cflt va x, la so lieu timg hang dua fren dir lieu thfl de so sanh vflfi X(,
x„=(x,(i),x,(2),-.x„(.k),-,x„(m)) (3)
x,=(x,(l),Ar,(2), ,x,(i),-,x,(m))
x,=(x,(.\),x,(2).-,x,(.k),-,x,{m))
X, =(x,(l),x,(2), ,x,(*:),-,i,(m)) (4)
X,=(.X,(.l),X,(.2). ;X,ik),-;X,{m))
i = l2, ,n
Trang 4Sau khi da thifa lap dugc sfl lieu phan tich thi tifln hanh tinh toan miic do quan he
xam Cflng thire tinh miic dp quan he xam da dupc dua fren li luan co ban vfl khodng
each Minkowski Mirc dp quan he xam dupc ki hiflu la Gamma va gia tri Gamma nim
trong khoang tit 0 dfln 1 Gia tri Gamma dupc tinh nhu sau [10]:
n , =rix„(,k),x,(.k)) = ^"~^°' ,i = l,2.-,n (5) trong do, A„, la tflng khoang each sai sfl myet dfli giila x, vflfi x„
A„,^ va A,„„ tuang irng la gia fri lon nhit va gia tri nhfl nhat ciia A„,, trong bai
vifa nay nguoi nghien cilu da sir dung p = 2 dfl tinh gia tri Gamma eho timg CH
2.3 Budng cong ROC (Receiver Operating Characteristic)
Duong cong ROC dilng dfl danh gia cac kfa qua cua mpt du doan va img dung
diu tifln cua no la cho viee nghien cim cac he thong nhan difln trong viec phat hifln cac
tin hiflu radio khi cfl su hifln dien ciia nhiflu vao thap niSn 1940 [1] Trong nghien cim
nay, dfl xay dung duong cong ROC cac nha nghifln ciiu can phai tinh toan dp nhay va
dp dac hieu cua timg cau hoi dua fren gia tri thue tfl va gia tri du bao dfl xac dinh cac
frang thai ducmg tlnh va am tinh
Bdng 2, Bdng 2x2 cua duong cong ROC
Gia trj dv bao
Gia tri thirc te Duong tinh that
ia)
Am tinh gia
Am tinh that
(b)
Duong tinh gia
id)
Cach xac dinh trang thai duong tinh va am tinh ciia CH nhu sau: Dya tren ket qua
ciia bang S-P va can cir vao tong so HS tra loi diing cau hoi dS xac dinh trang thai
duong tinh (ki hieu la 1) va am tinh (ki hieu la 0) ciia gia tri du bao Sau do can cir vao
gia tri thuc te de tinh cac trang thai a, b,cva.d cua CH
Dp nhay (Se)
-a + b
Dp dac hieu (Sp) =
c + d
(7)
(8)
Trang 5Difln tich bfln duoi duong cong ^iiC = ^^^^^^^+^^^^^^ (9)
Ducmg cong ROC cfl true tung la ti lfl duong tinh that (dp nhay) va true hoanh la
ti lfl duong tinh gia (1 trir cho dp dac hiflu) Ca hai ti lfl nay sit dung xac suit dfl tinh va
CP ehung gia tri dao dflng tir 0 den 1 Theo nhiflu nghien eim dien tich bfln duflfi duong
cong ROC (A UC) dugc xem la phan biflt tflt gitra hai frang thai duang tinh va am tinh
On AUC> 0,7; va khflng phan biflt tflt giila hai trang thai khi ^!7C< 0,7 [1], [3]
2.4 Thiit kd hop cdng CV MATLAB
Trong nhtmg nam gan day, dfl thuan tien cho viflc tinh toan nhanh chong va chinh
xac eae phep tinh phire tap nhiflu nha nghien citu da sir dung phin mflm MATLAB dfl
thifa kfl mpt hop eflng cu MATLAB [4], [7], [8] Trong bai vifa nay, nguoi nghifln cim
thiet ke mgt hop cong cu MATLAB cho phan tich va lua chpn cau hfli TNKQ, chuang
trinh xir li dQ lieu ciia hop cflng eu MATLAB trong nghifln ciiu nay dugc tflm tit gflm
cfl 7 buofc nhu sau (hinh 1):
Budc L Nhap du lieu Du lieu la ma tran X da dugc nhap vao duoi dang tap tin
*.csv hoac *.xlsx
Budc 2 Kiflm tra dfl tin cay ciia du lieu
Budc 3 Tinh tflng sfl HS tra lai dung cau hfli; tinh he sfl chu y cua CH, tiep theo
la sap xflp theo gia tri CP tir nhfl dfln Iflfn; sau do sap xep theo tflng sfl HS tra loi dung
CH tir Iflfn dfln nho
Budc 4 Thifa lap vector x^, tiep theo tinh tflng khoang each sai sfl tuyet dfli ciia
timg CH; tinh gia tri Gamma ciia timg CH; sau do thifa kfl kfa qua va hinh anh
Buac 5 Xac dinh cae trang thai duang tinh va am tinh; tinh cac gia tri a, b, c va d;
tinh dien tieh bfln duoi duflfng eong ROC ciia timg CH; sau dfl thifa ke ket qua va hinh
anh duang eong ROC
Bu&c 6 Thifa kfl hifln thi cac kfa qua va hinh anh dfl hifln thi fren giao difln do hpa
nguoi diing Nguoi su dung efl thfl luu lai ket qua duai dang tap tin *.csv hoac *.xlsx
va hinh anh duflfi dang tap tin *.JPG
Budc 7 Tiep tuc hoac thoat khoi chuong trinh Nflu nguflfi sii dung nhap du lieu
mai chucmg trinh sfl tiflp tuc va fro ve buac 1, hoac thoat khoi chuang trinh
Trang 6Ty lieu tham khao
( Ketthljc j ( Trovg V Trovl jf Trove j
Hinh 1 Ltm do phdn tich vd lira chon cdu hoi trdc nghiem
3 Kflt qua nghien cihi va thao luan
3.1 Dir lieu thuc nghiem
DQ lieu frong nghifln cim nay duge lay tu mflt truong THPT huyen Hfln Dit, tinh Kifln Giang Du lieu thuc nghiem la kfa qua fra lai 25 cau hoi mfln Sinh hpc 11 cua 33
HS (DO lieu dupc trinh bay a bang 3) Trong bai vifa nay, ngucri nghien ciiu sit dung kfa hgp bang S-P, phan tich quan he xam va duong cong ROC dfl lira chpn cau hfli trie nghiem Sinh hgc 11 Truoc khi tifln hanh phan tich lua ehgn cau hoi trie nghiSm, ngudi nghifln ciiu da kiflm tra dp tin cay cua dQ lieu thflng qua viflc kiflm dinh he sfl Cronbach's Alpha He sfl Cronbach's Alpha cua du lieu frong nghien cihi nay la 0,858;
he sfl nay eho thay du lieu cfl do tin cay cao
Trang 7Bdng 3 Ket qud trd ldi 25 cdu hoi trdc nghiem cua 33 hoc sinh
(trd ldi dung dugc ki hiiu la 1: trd ldi sai dugc ki hieu Id 0)
H S \
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
ii
2
1
0
1
1
1
0
1
0
1
1
1
1
1
1
1
1
1
1
1
1
1
0
1
1
1
1
0
0
0
0
1
1
0
3
0
0
0
0
1
0
1
0
1
]
1
0
0
0
0
0
0
1
1
0
0
0
0
0
0
1
1
0
0
0
0
0
0
4
0
0
0
0
0
0
0
0
0
1
1
0
0
I
0
0
0
1
1
1
1
0
0
I
0
0
1
0
I
0
0
0
0
5
0
0
0
0
0
0
1
0
0
0
0
i
1
0
0
0
0
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
0
6
0
0
0
I
7
0
0
0
0
0
1
1
0
8
0
0
1
0
1
0
0
0
1
1
1
0
0
0
0
1
1
1
0
1
0
1
0
1
0
0
0
0
0
1
1
0
0
9
1
0
1
0
1
0
1
0
0
1
I
1
0
1
0
i
1
1
1
1
1
0
0
0
0
1
1
0
0
0
1
0
1
10
0
0
0
0
0
11
0
12
1
1
1
1
0
0
1
0
0
1
1
1
1
1
1
0
0
I
0
1
1
0
1
1
0
1
0
0
0
1
1
1
0
13
0
0
14
1
I
1
1
!
0
1
0
0
1
1
1
0
1
1
0
0
1
0
1
1
0
1
0
0
1
1
0
0
0
1
1
0
15
0
1
0
0
0
0
0
0
0
0
I
0
0
0
0
1
1
0
0
1
0
0
0
0
0
1
0
0
0
0
1
0
1
16
0
0
0
1
1
0
1
0
1
1
1
0
1
0
0
0
\
0
1
]
1
0
0
0
1
1
1
0
1
0
1
0
0
17
fl
0
0
0
1
0
0
0
0
1
1
1
0
0
1
0
1
1
0
1
0
0
1
1
1
1
0
0
0
0
1
I
0
18
fl
1
1
1
1
n
1
n
1
1
1
0
0
1
0
0
1
I
1
1
1
I
1
0
1
1
0
1
0
1
!
0
0
19
0
0
0
0
0
20
1
n
0
0
1
0
!
0
0
1
I
0
0
0
1
0
1
0
1
I
1
0
0
1
0
1
0
0
0
1
1
1
0
21
1
0
1 t)
n
n
I
0
0
1
1
0
0
1
0
0
0
1
0
1
0
0
1
0
0
0
1
0
0
0
1
0
1
22
0
0
0
0
1
0
0
0
0
1
1
1
0
1
1
0
1
1
0
]
0
1
0
0
0
1
1
0
0
0
0
1
0
23
0
0
0
0
0
0
0
0
1
1
1
0
0
0
0
0
0
0
0
0
I
0
0
0
0
1
1
0
0
0
0
0
0
24
0
1
0
0
0
0
0
0
0
0
1
0
1
25
0
1
1
0
0
1
]
0
0
0
1
0
0
0
0
1
0
0
1
0
1
0
I
0
0
0
0
1
1
0
0
0
0
Trang 8Ty lieu tham khao
3.2 Ket qud nghien ciht
Trfln giao dien dfl hga nguflfi dimg cua hop cflng cu MATLAB (hinh 2), co thl
thiy kfa qua he s6 chil y cua cac CH, cac gia fri Gamma, difln tich ben duoi duflng cong ROC va hinh anh vfl gia fri Gamma, cac duflfng cong ROC ciia CH frong qua ttinh
phan tich va lua chpn cau hfli TNKQ dua frfln bang S-P, phan tich quan he xam va
ducmg cong ROC Kfa qua 6 bang 5 cho thiy co 12 CH co gia tri A UC nho hon 0,7 va
CO 13 CH CO gia tri AUC [an hon Jioac bing 0,7 Dua theo phan loai cac nhom CH o
bang 4 cd thfl thiy frong sfl 13 CH dugc lira chpn frong nghien cim nay co 1 CH thupc nhflm A, 2 CH thupe nhom B, 3 CH thupc nhflm C, 4 CH thupc nhom D va 3 CH thupc nhomE
Bdng 4 Phdn Ioai cdc nhdm cdu hdi dua theo gid tri Gamma cda cdu hdi [5]
Nhom
A
B
C
D
E
Gia tri Gamma (0< y <l)
Y>0,8 0,6 < y < 0,8 0,4 < 7 < 0,6 0,2 < y < 0,4 }'<0,2
Mu-c do cau hoi Ritde
De Trung binh Khfl Rit khfl
PHAK nCH ^ LUA CHON CAU HOl IHAC NGHIEM DUAlTtEN BANG S-P, GRA VA ROC
s
e NGUYEN PHUOC hWl, GV rata Tu rrten - Tin hoc, Trwxfl Cao m n g Su pfBm Kien G u r q
Htnh 2 Giao dien dd hoa ngudi dimg cua hop cdng cu MATLAB
Trang 9Bdng 5 Ket qud tinh he sd chu y cua cdu hdi (CP), gid tri Gamma vd AUC
Tong so HS tra IM dung
•S-^S^^ 0,86 O.Il
) ^ ' f^'^JH, ,.'
' ^ j i 4 '
0,14 .0j3
3
0,43
3.3 Thdo luan
Dua vao bang kflt qua (bang 5) co thfl thiy ring he sfl CP cang cao thi gia fri A UC cang thip va ngugc lai Khi he so CP cang cao va gia tri A UC cang thap thi diflu nay
cho thiy CH tucmg img co dp phto biet thip Boi vi, khi do HS co nang luc thip lai tra lcri dllng cac cau hfli khfl Vi vay, phuong phap nghien eiiu nay giup xac dinh va loai bfl
cac CH cfl he sfl CP cao va gia tri y4 C/C nhfl hon 0,7
Trang 10Kfa qua nghifln cim nay se la tai lieu quan trpng ttong viflc xay dung ngan hang cau hoi trie nghiflm khach quan diing dfl kiflm fra danh gia kflt qua hgc tap cua HS Hop cong cu MATLAB frong nghien ciiu nay ciing cho thay la rat htm dung va tien ich, giup eho viflc tinh toan fra nfln nhanh chong, chinh xac va hifln thi eac kflt qua, hinh anh mpt each true quan sinh dpng Dua frfln phuong phap nay cOing cfl thfl tinh dupc he
sfl chii y CS, gia tri Gamma va gia tri A UC cua timg HS Tren co so do co the xac dinh
dupc nang lue hge tap cua HS Ngoai ra, no cung cung eip cho giao vifln nhimg thflng tin cin thifa nhim xac dinh dung han ve nhan thirc cua HS frong hpc tap tir dfl dfl xuit kip thai cac bien phap diflu chinh hoat dpng day hpc, thuc hifln muc dich day hgc
4 Ket luan
Bai viflt nay se la mgt tai lieu tham khao rat hiru ich cho cac nha quan li giao due, giao vifln va tat ca nhitng ai quan tam den viflc nang cao hiflu qua cua viec kiem tra, danh gia thflng qua cau hfli trac nghiflm khach quan
Nghifln ciru nay da thiflt kfl thanh cong mgt hop cong eu MATLAB eho phan tich
va lua chpn cau hfli trac nghiflm khaeh quan Hop cflng cu MATLAB nay co nhieu uu diflm nhu la dfl dang sii dung, tiet kiflm thai gian, tinh toan chinh xac, hifln thi ket qua
va hinh anh mflt each true quan sinh dpng
Tir nhimg ket qua nghifln eiiu cho thay day la phuang phap ma eac giao vien co thfl su dung nhim cai thien chit lupng bien soan va thifa kfl dfl thi, de kiflm fra TNKQ, dong thoi nghien ciiu nay cung cho thay co thfl ap dung phuong phap nay xay dung ngan hang cau hfli TNKQ dfl kiflm tra danh gia kfa qua hpc tap cua HS
TAI LIEU THAM KHAO
1 Arian, R., Erkel V., Peter M., and Pattynama T (1998), "Receiver operating characteristic (ROC) analysis: Basic principles and applications in radiology"
European Journal of Radiology, 27, pp 88-94
2 Deng J L (1989), "Introduehon to grey system theory" The Journal of grey system,
1(1), pp 1-24
3 Hanley, J A., and McNeil B J (1982), "The Meaning and Use of the Area under
Receive Operating Characteristic (ROC) Curve", European Journal of Radiology,
143, pp 29-36
4 Sheu T W., Nguyen P H., Nguyen P T., Pham D H (2013), "A Matlab Toolbox for AHP and LGRA-AHP to Analyze and Evaluate Factors in Making the Decision",
InternationalJournal ofKansei Information, 4(3), pp 149-158
5 Sheu T W., Nguyen P H., Nguyen P T., Pham D H., Tsai C P., and Nagai M (2013), "A New Proposal for Using Grey Relational Analysis and Receiver
Operating Characteristic to Analyze Items", International Conference on Grey System Theory andKansei Engineering Conference, pp 17-24