PHliONG P H A P D A N H GIA CHAT LUONG NfiANJlANG CAu HOITHI^ METHODS TO EVALUATE THE QUALITY OF THE TEST QUESTIONS BANK H OJ ThS Nguyen Tien HUng, ThS Pham DilTc CUdng Bgi hgc Suphgm Ky thudt Nam Din[.]
Trang 1CHAT LUONG NfiANJlANG CAu HOITHI^
METHODS TO EVALUATE THE QUALITY OF THE TEST QUESTIONS BANK
H-OJ ThS Nguyen Tien HUng, ThS Pham DilTc CUdng
Bgi hgc Suphgm Ky thudt Nam Dinh
Tom tat
Ap dung ly thuyet do lUdng de d i n h g i i chit Idpng ngan hang c i u hdi thi trie nghiem ddng mdt vai trd quan trong trong viec nang cao chat lupng dao tao cua nha trudng Hien tai ed hai phuang p h i p ca b i n de d i n h gia chat lUpng n g i n hang c i u hdi thi dd la phdang p h i p chuyen gia va phuang phap dinh luang Bai bao nay sddung phuang phap danh gia dinh tinh va djnh luang de p h i n tich va d i n h g i i chat lUpng n g i n hang c i u hdi th t r i e nghiemi, Thdng qua dd, cac Nha giao cd nhihig bien p h i p tich cgc de dieu chinh ngan bang cau hdi thi mdt each hop ly
TCrkhda: Ngdn hdng cdu hdi, danh gid, do lUdng phuang phdp chuyin gia, phuong phdp dinh lugng
Abstract
Applying measurement theory to assess the quality of Test question bank plays an important role in improving the academic quality of the schools Currently there are two baste methods to assess the quality of Test question bank that is expert methods and quantitative methods This paper uses qualitative methods and quantitative methods to analyze and evaluate the quality of the Test questions Through it,Teachers will have positive methods to adjust the exam questions bank for reasonable
Keywords: Questions bank, Evaluation, Measuremnet, expert methods, quantitative methods
I O a t v a n d e cho cic nhdm giao vien phu trach hoc phSn,
De nang cao dUpc chat lupng dao tao, viee mon hpc thgc hien Can bp giang day (CBGD) danh gia de thi ed vai trd het sdc quan trpng, "^y dgng bd de hau het deu theo kinh nghiem nham mye dich d i n h g i i chinh xic ket qua hpe chu quan cCia minh, chUa dupe trang bj day du
t i p ciia sinh vien Dieu nay chSng nhdng la khiu ve kien thdc va ky n i n g thiet ke de thi, cic de thi phan hdi chat Idpng cua ngan hang eau hdi do CBGD bien soan dupc dUa vao sQ dyng trge thi (NHCH) hay de thi trie nghiem (DTTN) chat t'ep, khdng qua qua trinh thd nghiem va danh
Idpng boat dpng ciia ea q u i trinh giing day ma gia dan den cac de thi kem chat lupng Chfnh
cdn la yeu td quan trong cd chdc nang tdang t i c vi vay, d b i i vi^t niy chung tdi dua ra 2 phuang ngupe trd lai de dieu ehinh, cii thien cac khau p h i p dinh g i i ; ehuyen gia va dmh luang de trude dd nham giup eho q u i trinh dao tao dam danh gia chat lUang de thi vdi muc dfch nang bao chat lupng v i hieu q u i tdt Thgc te hien nay, cao chat lUpng va hieu qua ciia edng t i c danh vifc xay dgng cic bd n g i n hang de thi dupc giao g i i khi tien hinh xay dgng NHCH,
Trang 2hdi thi
2.1 Phuang phap chuyen gia
Day la phuang phap danh gia djnh tinh, de
danh g i i bd n g i n hang cau hdi thi can danh
g i i theo eac tieu ehi sau d i y :
-SgphCi hap gida cau hdi va mdc do danh
gia;
- Su tuang duang ve khdi luang kien thdc
gida cac cau trong mdt gdi cau hdi;
- Su tuang duang ve dd khd eua eae eau
hdi trong cung mgt gdi cau hdi;
- Dap an eua cau hdi thi;
- Mdc dp rd ^ n g , dk hieu d mdi c i u hdi thi
I) SUphu hap giUa ndi dung cau hoi va mCfc
do ddnh gid
De bien soan ngan hang cau hdi thi, viec
dau tien can phai xay ddng b i n g thdng ke kien
thdc, ky nang can d i n h g i i va mdc dd danh
gia [1] Khi viet cau hdi thi, ngudi bien soan da
bam sit cae mdc dd danh gia da thdng nh^t
eho tdng eau hdu Tuy nhien, vi nhieu ly do khac
nhau, ddi khi cau hdi lai khdng phu hpp vdi
mdc dp d i n h g i i d i dg kien, cd khi lai qua khd
hoac qua de Chuyen gia ein so sinh ndi dung
cau hdi v i mdc do danh gia de cho y kien d hai
mdc: A la hdp ly v i B ia chua hpp ly Ket q u i
d i n h gia npi dung nay dimg phieu sau:
Phieu danh gia sd phii hcfp giuTa npi dung
cau hdi thi va mdc do danh gia
Ten hpc phan:
Hp va ten ngUdi danh gia:
Cau hoi M ^ d 9
phii hop
Cdu hoi muc Nhd
Cdu hoi muc Hieu
Cau 2
y kien/ Ly do
ehuyen gia can ghi rd ly do de can bd bien soan
ed the can cd v i o do de xem xet, chinh sQa
2) Do khd cua cdc cau trong cung mgt goi cau hdi
Mdt trong cae yeu cau bien soan n g i n
h i n g eau hdi thi la cac cau hdi trong cimg mdt nhdm {eung mQe diem, cung mdc dp d i n h gii) p h i i cd dp khd tQOng dQOng nhau [2] de khi to hpp ngau nhien thanh de thi, eac de thi ed dd khd tuang duang nhau Chuyen gia
d i n h g i i can xem xet ky c i u hdi, d i p i n de nhan xet ve dp khd eua cie cau hdi theo cic mdc: A la hpp ly v i B la ehua hpp ly Nhdng eau hdi danh g i i mdc B p h i i chinh sda lai de
cd dp khd, dd d i i tuang duang nhau Neu danh g i i tdt tieu ehi nay ciing chi ra dUOc eie cau q u i khd hoac qua de, gid lai nhdng c i u hdi vda sQc vdi ddi tQpng danh gia
Phieu danh gia
Oo kho cda cac cau hoi thi
Ten hpe phan:
Hpvaten ngddi d i n h gia:
Cau hoi Mircd6
danh gia
Goi cdu hoi so 1 (2.0 diem)
caul
Goi cdu hoi so 2 (4,0 diem)
Cau 2
Y kien/ Ly do
3) MUc dd tuang duang ve khoi lugng kien thUc giUa cdc cdu trong mdt gdi cdu hdi
Tdang t g nhu dd khd, khdi luang kien thdc gida cie eau trong mdt gdi cau hdi cung phai tdOng ddang nhau Khdi lupng kien thdc d i n h
g i i d hai mdc: A la hap ly va B la ehUa hap ly NhQng eau hdi d i n h g i i mdc B va p h i i chinh sda lai de eac eau hdi ed khdi iQpng kien thdc tuang duang nhau
[20] K H O A HOC D A Y NGHE Sd 30 thing 3/2016
Trang 3-Khoi lUtfng kien thufc cua cac cau h6i thi
Ten hpc phan:
Hpva tdn ngudi danh g i i :
Cau hoi Mircd$
danh gia
Goi cdu hoi so 1 (2,0 dtim)
Caul B
Gdi cdu hoi so 2 (4,0 diem)
Cau2 1
Y kien/ Ly do
4) Banh gid dap dn cua cau hdi thi
D i p an la mdt bd p h i n quan trpng cda
NHCHT, nd quyet djnh chat luong eiia bd n g i n
h i n g cau hdi thi D i p i n phai n g i n gon, ehinh
xic tao thuan lai trong q u i trinh cham diem;
thang diem p h i i phii hop va phai ehia nhd
den mdc hpp 1^ Chfnh vi vay, viee d i n h g i i sU
ehinh xic, hpp ly eiia d i p i n la rat can thiet,
Cic mdc dd d i n h g i i d i p i n : A la hap ly v i B
la ehUa hop ly
Phieu danh gia
Dap an cua cac cau hoi thi
Ten hpc phan:
Ho vaten ngUdi danh gia:
Cau hoi danh g\k Mircdg
Goi cdu hoi so 1 (2,0 diem)
Cau 1 A
G6i cdu hoi so 2 (4,0 diem)
Cau 2
V kien/ Ly do
Nhdng c i u hdi danh gia mdc B, trong cdt Ly
do can ghi rd nhdng sai sdt can ehinh sda
5) MUc dd rd rang, di hiiu (each dien dot) cua
cau hoi
Trong quy dmh bien soan ngan hang eau
hdi thi d i cd yeu eau ve viet eau hdi thi: " D i m
b i o tfnh khoa hpe, chfnh xae, c h i t ehe, ldi v i n
thuat ngd theo quy dinh hien hanh"[2] Vdi
c i u hdi trac nghiem edn quy dinh rd:
- Sddung ngdn ngdphuhppvdiSV, khdng hdi y kien rieng eua SV; ehi hdi sg kien, kien thQe
- Cau hdi trac nghiem nhieu Iga chpn: cau hdi thddng cd hai phan, phan dau dupe gpi l i phan dan, neu ra van de, cung cap thdng tin can thiet hoac neu mdt c i u hdi; phan sau la cic phuang an de chon t r i ldi, thddng dimg 4 phdang an dupc danh dau bang eie chd cii A,
B, C, D Khi bien soan cic phuong i n chon ehi
ed duy nhat mdt phuang i n dung, cae phuang
an sai p h i i ed ve hpp ly, phai sap xep phuong
an dung mdt eich n g i u nhien; d i m bao eho cau dan ndi lien vdi mpi phuang i n chpn theo dimg ngdphap;tranh dimg eau phii djnh, dac biet la phii dinh 2 lan; tranh viee tao phUang
i n dung khac biet so vdi eac phuang an khie (dil ban hoac ngan hon, md t i ty my han ) Ddi vdi eau hdi t d luan, van d i p eung quy djnh: "Cau hdi can rd r i n g va xac dinh, lam cho SV hieu rd can phai lam c i i gi, Neu can cu the ban, cd the phac hpa eau true ehung ciia
b i i tra ldi" Nhu vay, cich dien dat eau hdi rat quan trpng, de sinh vien hieu dung va day dii
ndi dung eau hdi 6 tieu chf nay danh g i i d hai
mdc do: A la hpp ly va B la ehda hpp ly
Phieu danh gia Cach dien dat cua cac cau hoi thi
Ten hpc phan:
Hpvaten ngudi danh g i i :
Cau hoi M ^ d o danh gia
Goi cdu hoi so I (2,0 diem)
Cau 1-01-Cl A
Goi cdu hoi so 2 (4,0 diem)
1
Yki^n/Lydo
Trang 4do can ghi rd nhdng sai sdt can chinh sda
Sau khi tung ehuyen gia danh gia theo tdng
tieu chf can lap b i n g thong ke tong hpp mdc
dp d i n h g i i , nhdng cau hdi ed tieu ehi dinh g i i
mdc A dudi ngUdng can p h i i ehinh sda
Bang tong hdp
danh gia ngan hang cau hdi thi
Ten NHCHT:
Hp ten eac can bd danh gia:
TT
1
Ma
cau hoi
Cau 1
1 ^ IS ddnh gii
6'mircAtbeotimgtieuclii(%)
TCI
66,6
TC2
66,6
TC3
100
TC4
0 TC5
Cau hdi cd nhdng tieu chi cd ty le t d 60%
trd len sd y kien danh gia d mQc A la dat chat
luong, cic tidu chf cd ty t>' le d i n h gia mdc A
dudi 60% can xem xet, chinh sda [1],
2.2 Phitang phdp dinh fitang
Cie eau hdi thi, sau khi dem ra sQ dung
can phai tien hanh p h i n tich de xem cic eau
hdi thl d i bien soan cd phu hpp vdi myc dich
kiem tra, d i n h g i i khdng.Tren easddd loai trd
nhdng eau hdi khdng pbij hpp hoac chinh sda
de n i n g cao chat lUang eau hdi thi Viee p h i n
tfch tdng cau hdi thi va toan bd de thi phu
thudc v i o mue dfch ciia ky thi, do dd cac d i e
trung thdng ke phai p h i n i n h duac muc dfch
nay Khi p h i n tfch eau hdi thi, hai ehi sd co ban
ein phan tich l i do khd v i dp p h i n biet Odi
vdi c i u hdi trac nght&m nhieu lua ehpn c^n
ed chf sd nda can p h i n tfch, dd la mdc ddi Idi
eudn v i o cae phdang i n tra Idi [2] Phan nay
duac k h i o s i t phan tfch t d 295 phieu dieu tra,
I) Do khd cdo cdu trac nghiem
Gpi Pi l i dp khd eua cau trie nghiem t h d I,
tacd:Pi=Si/N[3].Trongdd:
Si: sd sinh vien lam dung eau hdi thd I
N: tdng sd sinh vien tham gia lam c i u hdi
cho ta biet mQc khd, de cda c i u trie nghiem
ma khdng can biet ndi dung eiia nd thudc ITnh vdc n i o Gii tri ehi sd Pi thay ddi trong pham vi
t d O - 1, g i i trj Pi eang nhd thi c i u trac nghiem eang khd va ngdOc lai, gia tri Pi cang Idn thi cau trie nghiem cang de Vi du cd he sd Pi ciia hai
c i u hdi trie nghiem nhu sau:
P/Alra liri S6 luanfi Gia tn P
A B
12 IS
0,04 1 0,06
C«
248 0,84
D
17 0,05
B6
0 T6ng
295
Ghi chO: * ky hieu phuang dn tra ksi dOng
Qua vf du tren eho thay: cd den 248/295 SV
t r i Idi diing, he sd P = 0,84, nhd vay cau hdi nay thudc loai de
P/AtralM
So luong Gia t r i p
A*
61 0,21
B C
68 90
0,23 1 0,84
D
66 0,30
Bd-sot
10 0,22
Tang
295
Ghi chii- *kyhiiu phuang an tra idi dUng
Ciu hdi nay ed P = 0,21, chi cd 61/295 SVtri Idi dung, nhu vay c i u nay thudc cau khd
Khi Iga chpn cac c i u trac nghiem theo dp
khd, thudng phai loai cac cau q u i khd hoac
q u i d§ Mdt bai trac nghiem tdt la bai cd nhQng cau cd dp khd trung binh (Pie) Dd khd trung binh eiia eau trac nghidm dUpc djnh nghTa l i trung binh cdng giQa xac suat lam dung h o i n t o i n c i u hdi ay vdi xic suat may rill t g nhien Mdi loai cau trac nghiem cd mgt
x i c suit may rui t g nhien khie nhau vi v i y do khd trung binh cung khic nhau Cdng thdc tfnh dp khd trung binh eOa cau hdi i nhu sau: Pie^{1+n)/2.[3]Trongdd:
Pie la do khd trung binh eiia eau thd i
n la xae suat may rui t g nhien -Vdi eau hdi dung sai:
n= 0,5 nen Pie= {1 -i-0,5)/2 = 0,75 -Vdi eau hdi cd 4 Ida chpn:
n= 0,25 nen Pie- 0,625
Trang 5r)=0,2nen Pie=0,6
De ed ca sd xac dinh mdt cau t r i e nghiem
de, hay khd, hay vda sdc eua sinh vien, dau
tien ta xic dinh do khd Pi eua cau hdi ay rdi
dem so sinh vdi do khd trung binh Pie ciia
nd Dd khd thgc te tiem ein vdi dd khd trung
binh thi duac chap n h i n Tuy nhien Pi nam
trong khoing nao la hpp ly?Theo GS.TS, L i m
Quang Thiep [3] vdi cau hdi cd 4 Iga ehpn Pi
eiia cau hdi ed the chap nhan dUpc nam trong
khoing t d 0,25 den 0,75; c i u cd Pi Idn ban 0,75
la qua de, ed Pi nhd han 0,25 la c i u q u i khd
Theo Osterlind [7], g i i tri Pi nen n i m trong
khoing t d 0,40 den 0,80 Dudi 0,4 nghia l i
eau hdi qua khd va tren 0,80 la q u i de ddi vdi
nhdm thi sinh.Theo chung tdi, viec lay gia trj P
trong khoang gia tri nao phu thudc v i o hudng
sQ dung ket qua danh gia Neu d i n h g i i theo
chuan (so sanh thi sinh n i y vdi thi sinh khie) thi
de thi can cd nhOmg c i u hdi khd de phan loai
thi sinh thi c i u hdi thi can cd Pi cd gia tri nhd
TrUdng hop n i y cd the ehpn Pi trong khoing
t d 0,25 den 0,75 Khi d i n h gia theo tieu ehi,
danh gia ddpc sd dung de xic dinh mdc dp
thgc hien eiia mdt c i nhan nao dd so vdi eac
tieu ehi xac dfnh cho trUdc khdng c i n cau hdi
qua khd, trong trUdng hpp n i y dd khd Pi can
lay trong khoang 0,40 den 0,80 la phu hap {Pi
chenh lech so vdi Pie xung quanh g i i tri ± 0,2)
Nhu vay vdi n g i n hang eau hdi thi ket thue
hpc p h i n Pi lay trong khoang 0,40 den 0,80
Mdt van de d i t ra la sd m i u phan tfch tdi
thieu bao nhieu la d i m b i o dd tin cay Theo
eie ehuyen gia do ludng danh gia, khi sd ngudi
tham gia tra Idi c i u hdi trac nghiem cang Idn
thi ket q u i e i n g chfnh xic, tuy nhien sd ngudi
tra Idi phai tdi thieu l i 50 mdi d i m bao dp tin
cay [4] [5]
2) Bd phdn biet cOa cau trdc nghiem
Dp phan biet {dp phan cich) la kha nang
cau trac nghiem thgc hien dupc su phan biet
binh, kem
Cie budc xic dmh dp phan biet ciia cau trac nghiem: sau khi cham toan bd bai trac nghiem cua sd mau tham gia, tien h i n h tfnh
dp p h i n biet eiia c i u trac nghiem, v i l i m theo cic bude sau:
Budc T Sip xep eac bai kiem tra theo thd
t g t d d i e m eao den diem thap hoac ngupc lai,
Budc 2: Liy 27% tdng sd bai lam tinh t d
diem cao nhat xep xudng va xep vao nhom
cao, ddng thdi lay 27% sd bai lam tfnh t d diem
thap nhat xep len va xep vao nhom thdp LUu y
rang theo Kelly (1939) [6] eho rang eon sd 27%
cd the cho mdt chi sd dn dinh ve su khac nhau gida hai nhdm ed nang Idc cao va thap
Budc 3: Tinh tdng sd bai la dung eau hdi i
trong nhdm cao v i goi l i Ci, ddng thdi tinh tdng sd b i i l i dung eau hdi i trong nhdm thap
va gpi l i Ti
Budc 4: Xic djnh dd p h i n eich D eiia cau
hdi i theo bieu thdc sau: Di = (Ci - Ti)/n [6] Trong dd:
Ci l i sd sinh vien trong nhdm eao lam dung cau i
Ti l i sd sinh vien trong nhdm thap la dung eau I
N la tdng sd sinh vien trong nhdm cao {hoac nhdm thap)
Theo cdng thdc xic djnh dp phan biet nhQ tren thi Di thudc doan [-1; 1] Tuy nhien, khi
dd p h i n biet cd g i i trj i m , nghTa l i sd ngddi
d nhdm thap lam dung cau i nhieu hdn sd ngddi d nhdm cao, khi dd dp phan biet khdng
cd y nghTa Cac nha nghien cdu [3] [6],[7] ve do ludng va kiem djnh giao due da dda ra tieu chf
d i n h g i i dd p h i n biet nhd sau:
Di < 0,2: eau t r i e nghiem i ed dd phan biet kem
0,2 < Dl < 0,29: eau trac nghiem i ed dp
p h i n biet trung binh
- Sd 30 thing 3/2016 KHOA HOC D A Y NGHE [23]
Trang 6biet kha tdt
0,4 < Di: e i u t r i e nghiem i ed dd phan bi§t
tdt
Yeu cau ve chi sd dp p h i n biet bao nhieu
l i hpp ly?Thdng thddng, bai trac nghiem theo
chuan cd dp phan biet t d 0,3 trd len la tdt Cic
b i i trie nghiem danh gia hpc phan (theo tieu
ehi) thl nhQng e i u hdi ed tinh chat phan loai
SV mdi can Di > 0,3, cic cau hdi cdn lai chi can
ed dd phan biet Di > 0,2 NhQng eau hdi cd dd
p h i n biet la i m thi can p h i i xem xet chinh sda
hoac loai bd
3) MUc do ldi cuon vdo cdc phUdng dn tra ldi
Ddi vdi c i u t r i e nghiem khieh quan loai
eau nhieu Ida chpn, ngoai hai chi sd ve dp
khd va dp phan biet, cdn cd mdt chi sd nda
can quan t i m phan tfch, dd la mQe dp ldi eudn
vao cac phdang an tra ldi Neu mdt phuang an
nhieu [2] eiia eiu nhieu Iga chon lai khdng ed
SV t r i ldi hoac cd rat it SV Iga chpn thi chdng td
eae phuang an dd la sai hien nhien, khdng cd
sdehapdangi.Vfdu:
P/Attiloi A
So luong
Gia trip
0
0,00
B
75
0,25
C
13
0,04
D»
204
0,69
Bo U i sot r°"8
3 295
Ghl chO: * ky hieu phuang an tra ldi dung
Vi dy tren cho thay, phuang i n A khdng
ed SV nao chpn, nhU v i y phdang an A sai q u i
Id lieu Tuong t u nhu vay phuang i n C ed P
-0,04 chi danh Ida ddpc mdt sd it SV, can p h i i
ehinh sda lai phdpng an A va D
Trong trddng hpp mot phuong an nhieu
cd q u i nhieu sinh vien Iga ehpn, tham chf han
nhieu so vdi phuong an dung dieu nay ehdng
td cd sU hieu lam nao do gida phdang i n dung
v i phdang i n n h i l u Vfdu:
P/Aticaloi
Soluong
Gia trip
A*
123
0,41
B
128
0,43
C
23
0,07
D
21 0,07
Bo sot:
0 T6ng
295
Ghi chU: * ky hiiu phuang an tra ldi dUng
tren cd dp khd Pi d ngudng chap n h i n ddpc l i 0,41 nhung phuang i n nhieu B lai cd nhieu SV Iga chpn han phuang i n dung 0.4, do v i y can
p h i i chinh sda lai c i u hdi nay Nhdng phdang
i n nhieu cd ehi sd P trong khoang 0,07 den 0,2 la hpp ly
Qua p h i n tfch tren eho thay ddi vdi c i u hdi trac nghiem nhieu lUa ehpn can p h i i p h i n tich tf ml tdng phuflng i n t r i Idi dga tren he sd
P de ed the chinh sda cho c i u hdi n g i y eing hoan thien
3 Ket luan
TQ nhdng phan tich vdi sd lieu cu the tren
cd the ket luan viec i p dyng ly eie phdang
p h i p chuyen gia v i dinh iQpng de phan tich danh g i i chat lupng NHCH va DTTN l i tUang
ddi chinh x i c va hieu q u i Cac phdang p h i p
tren khdng nhdng l i cdng eu giiip cho giang vien khi x i y dgng eac NHDT va DTTN cd chat Idpng m i edn giup g i i n g vien danh du doan duac diem manh, diem yeu ciia sinh vien, t d
dd cd nhdng bien phap kjp thdi de giup d d sinh vien trong q u i trinh hoe t i p D
Tai lieu tham khao
1, Pham Xuan Thanh, (2013), Bdi giang mdn
Ly thuyit do ludng Vien D i m b i o chat lupng
giao due, Dai hoc Qudc gia Ha Ndi
2, Pham Xuin Thanh, (2012), Bai giang mdn
Md hinh Rasch va Phan tich dU lieu bdng phdn mem QL/£S7, Vien D i m b i o chat IdOng giio due,
Dai hoe Qude gia Ha Ndi
3, Lam Quang Thiep, {2012), Do lUdng va
ddnh gia hogt ddng hoc tap trong nha trUdng
Nxb Dai hoe Su pham
4, Hoing Trpng, Chu Nguyen Mdng Ngpe
{2008), Phdn tich dO lieu vdi SPSS - Vol.H, Nxb Hdng DUc
5, Bruce Lav/rence, LUNE How/ard (2013),
Qualitative research methods for the social sciences Paperback
6,Kelley,TrumanL.,/ntemaf/ona/fncyc/oped/a
of the Social Sciences ^ 968 Encyclopedia.com
7 Osterlind, S J., {1989) Constructing test items Boston: Kluwer Academic Publisher