ASSESSMENT OF SERVICE QUALITY AND CUSTOMER SATISFACTION AT MILITARY BANK

TABLE OF CONTENTS ACKNOWLEDGEMENTS i LIST OF FIGURES iv LIST OF TABLES v ABSTRACT vi INTRODUCTION 1 CHAPTER 1: LITERATURE REVIEW 5 1.1. Service 5 1.1.1. Concepts of service and bank service 5 1.1.2. Characteristics of service 5 1.2. Service quality 6 1.2.1. Concepts of service quality 6 1.2.2. Major aspects of service quality 7 1.2.3. Measuring service quality 8 1.3. Service quality and customer satisfaction 12 1.3.1. Concepts of customer satisfaction 12 1.3.2. Relationship between service quality and customer satisfaction 12 1.4. Research model and scales 13 1.4.1. Research model 13 1.4.2. Hypotheses 14 CHAPTER 2: RESEARCH CONTEXT AND METHODOLOGY 16 2.1. Research context 16 2.1.1. Military Bank 16 2.1.2. Transaction Center No.3 Lieu Giai, Hanoi 19 2.2. Methodology 19 2.2.1. Designing questionnaire 19 2.2.2. Sampling and collecting data 21 2.2.3. Analyzing data plan 22 CHAPTER 3: RESEARCH ANALYSIS AND RESULTS 28 3.1. Data description 28 3.2. Reliability analysis 31 3.2.1. Reliability of TANGIBLE scale 31 3.2.2. Reliability of RELIABILITY scale 32 3.2.3. Reliability of RESPONSIVENESS scale 33 3.2.4. Reliability of ASSURANCE scale 34 3.2.5. Reliability of EMPATHY scale 35 3.2.6. Reliability of CUSTOMER SATISFACTION scale 37 3.3. Exploratory factor analysis 38 3.3.1. Exploratory factor analysis of service quality scale 38 3.3.2. Exploratory factor analysis of customer satisfaction scale 41 3.3.3. Research model – Version 2 42 3.4. Regression analysis 43 3.4.1. Regression analysis 43 3.4.2. Research model and hypotheses tested result 48 CHAPTER 4: FINDINGS AND DISCUSSION 50 4.1. Assessment of perceived service quality at Military Bank 50 4.2. Impact of service quality factors on customer satisfaction at Military Bank 51 4.3. Suggestions to improve service quality and customer satisfaction at Military Bank 52 4.3.1. Improve RESPONSIVENESS and RELIASS components 52 4.3.2. Improve EMPATHY component 54 4.3.3. Improve other components of service quality 55 CONCLUSION 57 References 58 Appendix: Questionnaire 61  

FACULTY OF BUSINESS ADMINISTRATION

-ASSESSING THE USABILITY OF MARKUS WEBSITE

BY HOANG THI CHANG

A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE BACHELOR DEGREE OF BUSINESS

ADMINISTRATION (INTERNATIONAL STANDARD PROGRAM)

Hanoi, May 2016

VNU- UNIVERSITY OF ECONOMICS AND BUSINESS

FACULTY OF BUSINESS ADMINISTRATION

-ASSESSING THE USABILITY OF MARKUS WEBSITE

BY HOANG THI CHANG

A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE BACHELOR DEGREE OF BUSINESS

ADMINISTRATION (INTERNATIONAL STANDARD PROGRAM)

Hanoi, May 2016VNU- UNIVERSITY OF ECONOMICS AND BUSINESS

FACULTY OF BUSINESS ADMINISTRATION

-ASSESSING THE USABILITY OF MARKUS WEBSITE

Supervisor’s name: Dr Nguyen Thi Huong Lienn Student’s name: Hoang Thi Chang

Student’s ID: 11050013 Intake: QH2011-E Program: International Standard Program

Hanoi – May 2016

First and foremost, I wish to express my sincere thanks to my advisor Dr.Nguyen ThiHuong Lien, who was thoughtfully guiding me in writing this thesis Besides, I want toexpress my grateful to director and staff at faculty of business and administration whohave support me in this thesis

Secondly, I would like to thanks Markus School that gave me an internship Iespecially thank Mr Do Xuan Khoa who was directly introducing me the websiteusability and all aspects of Markus School Besides, I thanks to all staffs of MarkusSchool for their whole-hearted guides during my internship in Markus School Last butnot least, I’m grateful to the numerous people who participated in my research

Student Hoang Thi Chang

Table of Contents

LIST OF FIGURE 7

Purpose of study 10

Research questions 10

Object of study 10

Task/ Process 11

Contribution 11

CHAPTER1: LITERATURE REVIEW 12

1 1 Usability 12

1.1.1 Concept and element of usability 12

1.1.2 Website usability concept and its elements 14

1.2 Usability evaluation method, model and tool 15

1.2.1 Category of usability evaluation method 15

1.2.2 Model of Usability evaluation 16

1.2.3 Usability measure tool 17

1.2.4 Educational website usability measurement on user’s perspective 18

CHAPTER 2: METHEDOLOGY 21

2.1 Analysis framework 21

2.2 Research model 22

2.3 Designing questionnaire and performance test 23

2.2.1 Questionnaire design 23

2.2.2 Designing website performance test 24

2.4 Sample and data collection 24

2.5 Data analysis methodology 25

2.5.1 Questionnaire analysis 25

2.5.2 Website performance analysis 26

CHAPTER 3: ANALYSIS 28

3.1 Markus School introduction 28

3.1.1 Marketing school introduction 28

3.2 Markus website operation 29

3.3 Data description 30

3.3.1 Questionnaire based method 30

3.3.2 Website performance test 33

3.4 Data analysis 34

3.4.1 General usability point: 34

3.4.2 Usability points in term of factor factors 36

3.4.3 Website performance usability 39

CHAPTER 4: FINDING AND DISCUSSION 40

4.1 The usability of Markus website (thinkmarkus.com) 40

4.2 Suggestion for development 43

CHAPTER 5: CONCLUSTION 45

REFERANCE 46

APPENDIX 49

Questionnaire – English 49

Questionnaire – Vietnamese 52

LIST OF TABLE

1 Table 2.1 The merit of website usability 25

2 Table 2 2: The usability points and corresponding level 26

3 Table 3.1: Website performance test data 34

4 Table 4.1: The usability score of Markus website 41

LIST OF FIGURE 5 Figure 3.1: Markus operation system 28

6 Figure 3.2: Demographic user at thinkmarkus.com (Source: Google analytics, 2016) 29

7 Figure 3.3: Demographic information of respondents 31

8 Figure 3.4 Usability of Markus website 34

9 Figure 3.5: Usability based on gender 36

10 Figure 3.6: Usability based on gender 37

11 Figure 3.7: Usability based on number of course 38

Internet is much more important nowadays which new business environment is Everyentrepreneur has its own website to present themselves, communicate and cooperatewith stake holder An efficiency website contributes to the success of entrepreneur.This study aim to evaluate the usability of Markus website on user’s perception to learnhow satisfaction the website serve user The usability would be studied based onWammi model with two tools: questionnaire evaluation and performance evaluation.The result of this study will answer how well it satisfy customer, it meet theexpectation of Markus The result of the study show that Markus website score a goodpoint at usability Base on, the result, the researcher have recommend three point assuggestion for website development This suggestion is developed logically to improvelimited point and enrich the good point at Markus website Although the research solveresearch problem, answered the big and sub research questions as well as achieve thepurpose of this study, it still have several limitation that can improve in the futurestudy

Internet in Vietnam was formed and developed in 1997, since then, Vietnam hasalways been the world rated as one of the countries with the fastest annual increase inthe number of internet users The rushing speed development boots up the e- commerceenvironment in Vietnam According TO VECITA (Vietnam’s e-commerce andinformation technology agency), in 2014, B2C e-commerce sales amounts is more than

$ 2.97 billion Such a rapid increase have gain a lot attention of entrepreneurs Moreand more company or entrepreneur have invested in e-commerce environment One ofgood example is the increasing number of website in Vietnam The internet hasmodified the traditional communication process between organization and stakeholders(Mentes & Turan, 2012) For organization, the web gives access to a lot of people andenhances functional performance Web sites are becoming key factors of anorganization's success in the global market According to VECITA, 2014, 69, 7 %active user use internet for educational website every day in which 59 % active website

in Vietnam belongs to health, education and training website Therefore, educationwebsite is not an exception Academic website not only means technique to connectwith various stakeholder but also means for an institution to shape its image Gordonand Berhow (2009) claims that most of academic institutions use their website as a keypublic relation and marketing tool to reach potential students Whether students wouldapply to institution, the final decision may base on how useful and comprehensiveinformation on institution’s website is On the other word, the customer generallysatisfied with institution if the website can help them achieve their goal in a specificcontext Otherwise, they would move to competitive complement Therefore it ispivotal to evaluate the satisfaction of customer in general and student in academicinstitution in detail on website operation which attract customer at the first sight.Aldwyn (2013) argues that usability is one factor that satisfies user when surfinginternet or access website Web design must conform to specific user needs, and mustalso make sure that users are satisfied to successfully complete tasks with web withoutfacing any difficulties (Yan & Guo, 2010) Therefore, in such a rush development andcompetitive e-commerce currently, it is really important to satisfied customer firstlythrough website which mean, the firm or institution should pay more attention toevaluate and improve the website usability It is much more important to educationindustry since more and more active user choose distance or e- learning as a money –time saving education, otherwise there are no more inactive user who only know tofind information directly in representative office Nowadays, internet booming save

their time and make it easy to assess information via website, social network In case ofMarkus School which is an academic institution, where they ignite new marketingthinking and train marketing skills Their website means brand image, present theirupdated course information and useful marketing blog which support their student.This website is technique to connect lecture, student an operational support Entereducation market have been 3 three years, however their website have been investedfor one year with the aim of enhancing business Therefore, it is necessary to assess theusability of website to be sure whether or not the website have done well in term ofsatisfying customer The result is also the foundation for next investment: e-learning.

Purpose of study

In context of this study, the researcher want to evaluate the usability of Markus websitebased on user’s perspective then use the result to suggest the improvement methodwhich increases the usability as well as satisfaction of user at present when they accessMarkus website and foundation for future feature development To achieve thatpurpose, the researcher must firstly understand deeply what usability is and itselements From investigated literature, the researcher would assess the usability ofMarkus website to know how easy the user feel when the use Markus website based onusability element and practical accession From user’s perspective, based on theirfeeling of Markus website, the researcher would recommend some method to improveand raise the usability of Markus website for user based on their strategy in future

What is the element to measure the usability in general and website usability?

How to measure usability and website usability?

Object of study

In this thesis, researcher only focus on usability of website on user’s perspective Awebsite includes one or more web page that have a common theme such as people,

organization, business (Nur, Adzhar, Norrozila, 2013) There are many methods,elements to evaluate website Usability is one of the major factor contribute to thesuccessfulness of a website There are a lot of definition or term about usability In ISO

9241 – 11 (1992), usability is “the extent to which a product can be used by specifiedusers to achieve a specified goals with effectiveness, efficiency and satisfaction in aspecified context of use” According to HCI (Human computer Interration), usability

is designing a computer system that support people to carry out activities productivelyand safely This definition reflect a change that focus shift to human Whatever the aim

of website is, but final goal is communicating effectively the information of content tocustomer Therefore, a website design firstly base on customer’s need and the websiteonly successful if its usability satisfy actual user On the other word, assessing usability

on user’s perspective reflect the true quality of website

The study could only focus on Hanoi area in which, the researcher concern mostly ofStudent experience and staffs at Markus, who response for building up website

Markus has start for more than 3 years However the website was built for about 1years to meet the demand of development In this thesis, all data of usability of Markuswebsite is only applied in year from staring time

Task/ Process

This thesis has 3 main steps: (1) secondary research (2) primary research (3) analysisand summary In secondary research, the researcher investigate conceptually tounderstand research problem and figure out the solving method In Primary researchstep, the research applied theory and follow a proven method to approach objective orrespondent to collect primary data Finally, research combine the result of 2 previousstep to answer the researcher questions

Contribution

By conducting this thesis, the researcher want to gain attention of developer andbusiness organization about new trend in website design, which is focusing on theuser’s perspective, taking user-experience to be cornerstone The result will contribute

a comprehensive conceptual of usability for academic researchers and a easy and cheapmethod for assessing usability This method can be easy to apply by all kind ofbusiness, special start up

CHAPTER1: LITERATURE REVIEW

1 1 Usability

1.1.1 Concept and element of usability

Usability is simply the “ease of use” which is a catch phrase that describe a productthat work better for user (Whitney Quesenbery, 2001) Usability can be applied innumerous objects, such as: software application, book, website, tool, machine, process,etc In different context usability has a different specified definition, however all ofthem focus toward make product and system easier to use and match them closely touser’s need and requirement

The term ‘usability’ was introduced in the early 1980s to refer to what was then anumber of vague and subjective attributes of a product, collectively known as “userfriendly elements” (Bevan et al 1991) However, the cornerstone of usabilityconceptual is usability standards provided by (ISO) the International StandardsOrganization which has two main categories: (1) product-oriented standards (ISO

9126, 2001; (2) process-oriented standards (ISO 9241, 1992/2001; ISO 13407, 1999)

In ISO 9241-11, 1992 usability is defined as: “The extent to which a product can beused by specified users to achieve specified goals with effectiveness, efficiency andsatisfaction in a specified context of use” ISO 9241-11 explains how to identify theinformation that it is necessary to take into account when specifying or evaluatingusability in terms of measures of user performance and satisfaction It also explainshow measures of user performance and satisfaction can be used to measure how anycomponent of a work system affects the quality of the whole work system in use Inthis definitions, the element of usability are: “effectiveness, efficiency andsatisfactions” It focus more on human interaction for software product In the field ofSoftware Engineering (SE) in ISO 9126-1, usability is defined as “the capability of thesoftware product to be understood, learned, operated, attractive to the user, andcompliant to conditions” It has 6 main elements: “functionality, reliability, usability,efficiency, maintainability and portability and more sub elements: understandability,learnability, operability, attractiveness and usability compliance Usability is not barelyfunctionality and increasing functionality does not mean improved usability Youshould focus on 3 usability component: user, their goal and usage situation which isaffected to improve usability

Base on ISO 9126 AND 9241, some researcher has created their own model includingdifferent usability element For example, Benbunan-Fich defined usability as “howwell and how easily a user, without format training can interact with an informationsystem of a website” In context of website, Benbunan-Finch use learnability,efficiency, aesthetics, navigation, content, functionality, accuracy, consistency,technical adequacy, help and documentation and error removal for his usability.Similar, in 2005, in term of digital library, Judy Jeng use effectiveness, efficiency,satisfaction and learnability as usability elements Usability consultant Jakcon Nielsenand computer science,Professor Ben Shneiderman has define usability as usefulness,which composed of learnability, efficiency, memorability, errors, satisfaction In thisdefinition, Nielsen concentrated on memorability, which related to cognitive aspects,and highlighted error tolerance, a elements that some authors neglected

Beside some common previous elements, some researchers concerned on therelationships between users and Web technologies (Yu-Hui, Abebe, Carol, 2015) Forexample, Andreas Lecerof and Fabio Paternò’s definition mentioned the relevance of asystem to the users’ needs, efficiency, learnability, and a system’s safety features (undoaction to prevent error) Whitney Quesenbery highlighted user experience and affect byincluding elements: engagement, pleasure in information seeking and online servicecontext Furthermore, many researcher addressed information organization andstructure as crucial usability elements Jonathan Palmer accentuated websiteconsistency, ease of reading, information organization, speed and layout Many expertsalso regard information organization and structure as crucial usability elements.Jonathan Palmer emphasized website consistency, ease of reading, informationorganization, speed, and layout In addition, many researcher believed that content is acore component of website usability Beside, usability characteristic, researcher alsodefined usability testing in a specified environment in which a particular system is usedand tasks undertaken to achieve a user’s goals For example Shackel emphasizedenvironment in relation to user support in fulfilling a specified range of tasks TefkoSaracevic built an iterative information retrieval model in which users (with a host ofvariables of their own) are related to a situation (task, problem-at-hand) within anenvironment, each having a number of elements and dynamics Nicholas Belkindeveloped information-seeking models which consists of three components: the enduser, the information objects, and intermediaries (such as humans, tools, or both) thatsupport the interaction between the user and the information objects Anotherinformation-seeking behavior model proposed by Tom Wilson addressed theinterrelation among cognitive, physiological, and affective needs His modelaccentuated the environment factor, focusing on work, sociocultural, politico-

economic, and physical aspects Wilson noted that a user’s social role in tandem withthe environment would impact the user’s needs.

From precious review, it is clearly to claim that usability is much more pivotal inhuman interaction and communication Usability has various elements in differentfiled, which is influenced by many factors: web technology, user, context, andenvironment and so on Day by day when the environment changes cause usabilityelements change to adopt a new environment Nowadays, the internet booming andtechnical digital tool development bring the control closer to user or customer, whichcause a change in usability change There is an obvious shift toward human interaction

as well as user perspective Usability only work if it meet user’s need, facilitate user toachieve their goal Usability is not only focus on software any more Specially, website is use widely for medium of information and service communication and usabilityprinciples was implementing in web environment and not for software only User onlyuse the web site if the web site can achieve their task or goal for searching theinformation or using it services more quickly, easily and effectively This issue hasgain attention of a lot researcher in website usability on user’s perspective

1.1.2 Website usability concept and its elements

Web evaluation has been used in decades to evaluate how well the website work.Website is a series of page which has a common theme in user, business ororganization The front page is called home page which composed a lot of hyperlinkrelated to the others webpage There are many method to evaluate websiteperformance However, usability is a key factor whose test result tell developer andorganization how well the website work From international standard definitionproposed by ISO, many researcher has developed new concept of website usability Forexample, Benbunan – Finch, 2001defined usability as: “how well and how easily a userwithout formal training can interact with information system of website” Thisdefinition developed base on the key term of usability: “ease of use” Bazar, 2009 andJadhav, 2010 has separately defined that website usability is a quality attribute toassess how easy user interfaces to use This definition clarified website usability in twoterm: design process and approach to use In 2013, Samsur Rahman and definedwebsite usability as an approach to make end-user for using and learning withoutrequirement training Usability play a pivotal role in website quality control A website

is consider usable if it encompass four factors: usefulness, effectiveness, learnability,user satisfaction (Rubin, 1994) According to Benbunan- Fich, 2001, the element ofwebsite usability are learnability, efficiency , aesthetics and navigation, content andfunctionality, accuracy and consistency, technical adequacy help and documentation

and error removel Zaphiris and Darin (2001) claimed that web usability as “anyoneusing any kind of web browsing technology must be able to visit any site and get acomplete understanding of the information, as well as have the full and completeability to interact with the site if that is necessary.” In other words, the web siteusability is a test on the successfulness of web site’s user in doing some task or findinginformation in the web site (Yusof et al, 2010) Mebrate (2010) defined web usability

as the mixed several design goals that make website easy to learn, remember, andunderstand find and effective to use Nielsen, 2012 defined that ‘usability is a qualityattribute that assesses how easy user interfaces are to use’ Nielsen’s definitioncomposed five quality components: learnability, efficiency, memorability, error,satisfaction Jabar, Usman, Sidi, 2014 defined website usability as four maincomponents: Content, organization and readability; navigation and links; Userinterface design; performance and effectiveness

There are many website usability definition cover various dimension, however thereare two common elements: (1), ease of use, (2) human interaction From 1994, JakobNielsen has claimed that usability is about basic human capabilities and user’s need,which do not change nearly as rapidly as technology He also claimed that humanfactors remain the same decade after decade Therefore, the use center usability trend isindispensable More and more research interested in website usability on user’sdimension

1.2 Usability evaluation method, model and tool

1.2.1 Category of usability evaluation method

Usability can be evaluated in different dimensions, such as technology, developer, anduser There are different types evaluation method used to examine usability indifference aspects According to Mack and Nielsen (1994), the evaluation methods can

be classified into four following categories: (1) Automated – usability was measured

by running a user specification through evaluation software (2) Empirical – usability isevaluated by testing the interface with real users (3) Formal – using models andformulas to calculate usability level (4) Informal –usability expert use their knowledgeand experience to assess usability measures Benbunan-Fich (2001) on the other hand,clarified usability evaluation methods into four following categories: (1) Objectiveperformance – measuring how the visitors using the website in terms of time for taskcompletion (2) Subjective user preferences – measures the users’ likability by askingthem give opinions or use questionnaire to rate the system (3) Experimental – testhypotheses about design and impact on user performance and preferences based onexperiment (4) Direct observation – inspect and monitor the users’ behavior while they

are interacting with the system to detect usability problems Al-Radaideh (2011)simply clarified usability measure method into 3 categories: (1) The inspection basedmethods without end users (2) The test based methods with end users and the toolbased automated evaluation method (3) Usability inspection methods often referred to

as professional evaluation where experts’ knowledge is used to assess the usability ofthe website, are used to identify usability problems and improve the usability ofinterface design by inspecting it against recognize standards

1.2.2 Model of Usability evaluation

A lot of usability testing model was developed for various dimensions However, thereare only several well-known models which is used and adapted to measure usability ondifferent field Esacon Model (1984) is the earliest usability measurement model,which is proposed by Kennneth Eason in issue of behavior and information technology.Eason Model defined usability into three aspect: Task, user and systems In Taskaspect, researcher addressed the frequency and openness of product or system Easonconcerns knowledge, motivation and discretion A usability system in Eason modelhave to be ease of learning, use and task match Eason model could not measure userand their targeted task This model can applied to measure usability of product andsystem From this model, many researcher has improved and developed new differentmodel for various purpose and context For example, Shackel Model (1986) has 4elements: effectiveness, learnability, flexibility and attitude The researcher did notweight aspect of dimension like Eason model Because the researcher believed that thedimension would change on different project However, this model accentuated onhuman factor Booth adapted this mode to a new usability definition which composed 4elements: usefulness, effectiveness, learnability (ease of use) attitude (likability) Heeliminated flexibility since it is difficult to specify and measure flexibility Booth’sdefinition has been accepted generally by usability community, especially likabilitywas considered to be important Until 1998, ISO 9241 built up an internationalstandard for guidance on usability based on process oriented This model has 3 mainelements: effectiveness, efficiency, satisfaction Effectiveness describes the interactionbased on processed viewpoint, efficiency emphasized on results and resources.Satisfaction means user viewpoint The advantage of this model is objective measures

of usability The disadvantage is abstract However this usability model is accepted andapplied in academic research field In addition to ISO model, there are another practicaland applicable to real business, which is Wammi WAMMI has been scientificallyproven whose reliability and data rating of between 0.90 and 0.93 (Marzanah, Usman,Aisha, 2013) The unique point of this model is Wammi questionnaire which has 20statement focus on user –experience of visitors 20 questions covers five factors to

assess the usability of websites (http://www.wammi.com/demo/graph.html) According

to wammi official service, explanation of 5 factors as following:

Efficiency

When respondents give a high Efficiency rating they feel they can quickly locate and

do what is of interest to them in a effective and economical manner They feel that theweb site responds (possibly, the pages load) at a reasonable speed

Helpfulness

A site which is high on Helpfulness corresponds with the respondents' expectationsabout its content and structure A site low on helpfulness can be misleading about itslayout and content

Learnability

When Learnability is high, respondents feel they are able to start using the site with theminimum of introductions Everything is easy to understand from the start WhenLearnability is low, respondents feel that the site may be using concepts or terminologywhich are unfamiliar More explanations are needed

Besides, there are many other famous model like Nielson Model (1993), ISO 9126(2001), QUIM (2006)

1.2.3 Usability measure tool

There are a lot of questionnaires developed to support usability measurement.Usabilitynet.org has listed down several outstanding questionnaire For example, Sumiwhich is a mature questionnaire which is good for desktop products, but has also beenused to evaluate command-and-control applications It is a commercial product whichcomes complete with scoring and report generation software Factors Research Group

at University College Cork designed and sold it Specially, applicable widely Wammiquestionnaire, designed to evaluate the quality of use of web sites It is backed up by

an extensive standardization database It is the result of a joint development project byJurek Kirakowski and Nigel Claridge Besides, there are many other tools like: SUSwhich is very robust and has been extensively used and adapted It is public domainand nobody has published any standardization data about it; QUIS questionnairedeveloped by Kent Norman that has been modified many times to keep it current sinceits first appearance It is commercially available and is championed by BenShneiderman in his book “Designing the User Interface”; USE Arnie Lund developed

it and last updated 11/11/1998 It attempts to create a three-factor model of usabilitythat can be applied to many situations However, no reliability or validation data arepresented; IsoMetrics This questionnaire is produced by Guenter Gediga and his team,which attempt to produce a way of measuring ISO 9241 part 10, with reference tospecific software features It is therefore good both for summative and formativeassessments The questionnaire is well researched and detailed statistical information isgiven

1.2.4 Educational website usability measurement on user’s perspective

Although there are various usability evaluation method generally or specified websitecontext, there are few studies in context of educational website (Samsur Rahman andZabed Ahmed, 2013) Lautenbach (2006) used 2 element or element: survey ability(user perceptions of satisfaction lay out) , find ability ( observed ease of use) Mustafaand Al-Zoua’bi (2008), applied 5 usability factors for university website content,organization and readability; navigation and links; user interface design; performanceand effectiveness; and educational information A questionnaire evaluation techniquewas used in this study, consisting 24 questions which was divided into four parts Theresults showed the good and the bad of the usability aspects of these websites in terms

of the usability element, the result of the study can be used by website designers anddevelopers to improve the websites Abdul Aziz, 2010 investigate the usability ofMalaysian university website including 120 samples of higher education institutionwebsites from the online portal of the Ministry of Higher Education(http://www.mohe.gov.my) This study focused on page size, broken links and web

performance are the usability features that the study focused on Bairamzadeh &Bolhari, 2010 on the other hand investigates factors affecting students' satisfaction ofuniversity websites The model composed usability, trust, perceived usefulness,satisfaction and website innovation The study by (Şengel & Öncü, 2010) investigatesthe usability of the Website of Uludag University through questionnaire which inquired

on how students uses the university’s website It was distributed to 445 students indifferent grade levels and enrolled in various faculties of Uludag University Therewere dissimilarities in the responses of both males and females The study providesvital directions for how future In other words, Yusof, 2010, web site usability was test

by doing some task or finding information in the website The results of study provide

a tangible concept on how the aesthetics design and usability impact each other andmethods to balance between aesthetics design and the usability Other studies usedautomated tool, such as Anwar and Tsuji (2011) used HTML Toolbox and Web PageAnalyser, to measure university websites’ internal usability in Bangladesh This studygathered data from users through a questionnaire covering 5 usability element inresearch Mustafa and Al-Zoua’bi (2008) Caglar & Mentes, 2012, the research analysethe effects of the information age on universities and measure usability of EUL's(European University of Lefke) website in terms of student utilization His studiedadapted Website analysis and measurement inventory (WAMMI) questionnaire,consists of 20 questions and use regression and Non–parametric techniques examinethe data The study shows discontentment and other usability problems of a EUL which

is situated at Northern Cyprus Mentes & Turan, 2012 tested usability of the usabilitylevel of Namık Kemal University (NKU) website based on Caglar and Menttes’sresearch Hasan (2012) assessed the usability of nine educational website with 5elements: navigation; architecture/organization; (3) ease of use and communication;design; and content Several studies measure usability based on page size, speed andbroken links Recently, in 2013, Sharmistha, Prasant, Rajib Mall combined Wammiquestionnaire method and performance method to evaluation usability of academicwebsite on user perspective to the 23 usability element which were grouped into fivecategories and each of the category deals with one usability aspect

Another study by (Hasan, 2012) developed element for evaluating the usability ofeducational websites and consists of 25 design issues divided into five majorcategories: content, design, ease of use and communications, organization/ architectureand navigation Then the relative importance of the developed element in theevaluation of the usability of educational websites was investigated from theperspective of 237 students The study also investigates based on the major/specialization and gender of the students that participated in the research (wherestudents were chosen from two faculties: Administration and Information Technology)

It was observed from the results that the order of the in the evaluation of the usability

of educational websites element from the most to the least important from the studentsperspectives was: content, navigation, ease of use and communications, design, andorganization / architecture The results also reveal that it is only in one category:contents that there was a statistically significant difference between among the gender

As females considered it as the most important category and males considered it as thesecond most important category However, the results showed that there were nostatistically significant differences between the students of the two selected facultiesregarding the relative importance of the element

The literature outlined above shows that there is an interest in investigating users'perceptions of the relative important of different design factors on the usability ofdifferent types of website

Quite a number of questionnaires have been and still being used in the evaluation of thewebsites usability Some of such questionnaire include: SUMI (Software UsabilityMeasurement Inventory), QUIS (Questionnaire for User Interface Satisfaction),

CHAPTER 2: METHEDOLOGY 2.1 Analysis framework

Figure 2.1: Analysis Framework

After doing secondary research, the researcher reviewed all related article andinvestigate company data to finger out researcher model: WAMMI Based on researchmodel, the usability was evaluated by two method questionnaire based evaluation andwebsite performance based method Then the researcher randomly selected sample toparticipated two evaluation method After data collection is done, it will be analysis the

Literature review

Research model

Website performance based evaluation method

-Questionnaire- based method

Sample

Data collection

Data analysis

same method in research of Sharmistha, Prasant, Rajib Mall, 2013 in term ofperformance method and Islam & Tsuji, 2011 in term of questionnaire method

2.2 Research model

Figure 2 2: Wammi Model (source: Human Factors Research in 1999)

After reviewing related article, the researcher decided to choose WAMMI which wasdeveloped by Human Factors Research Group (HFRG) in 1999 This model waschosen for several reason:

WAMMI is a usability evaluation tool for website

 It is based on a questionnaire filled by visitors of a website, and gives a measure

of how useful and easy it is to use in term of user’s dimension

 WAMMI has been scientifically proven and has a reliability and data rating ofbetween 0.90 and 0.93

 Model include 5 elements: attractiveness, control, efficiency, helpfulness, andlearnability They are popular element which was used in a lot of articles

Explain Wammi evaluation tool

WAMMI questionnaire has been thoroughly tested and used on numerous websites toensure maximizing user The 20 WAMMI statements are selected from a large range ofquestions about users' experiences with websites Over a number of iterative test-and-analyse cycles, the current 20 emerged in the form they now have The questionnaireallows visitor to speak their own voice of their real experience The 20 statement inquestionnaire covers 5 following elements:

Attractiveness

An Attractive site is visually pleasant, and also offers much of direct interest to therespondents, whether it be functionality or information.

Controllability

If a site scores well on Controllability the respondents most probably feel they cannavigate around it with ease and do the things they want to do Poor usually means apoorly organised site that disrupts the way they normally expect to do things

Efficiency

When respondents give a high Efficiency rating they feel they can quickly locate and

do what is of interest to them in effective and economical manner They feel that theweb site responds (possibly, the pages load) at a reasonable speed

Helpfulness

A site which is high on Helpfulness corresponds with the respondents' expectationsabout its content and structure A site low on Helpfulness can be misleading about itslayout and content

Learnability

When learnability is high, respondents feel they are able to start using the site with theminimum of introductions Everything is easy to understand from the start WhenLearnability is low, respondents feel that the site may be using concepts or terminologywhich are unfamiliar More explanations are needed

2.3 Designing questionnaire and performance test.

2.2.1 Questionnaire design

Based on Wammi questionnaire, and the purpose of how different user assess theusability of Markus website The researcher developed her own questionnaire includingthree parts: Personal information, usability evaluation, extended question Personalinformation including 4 questions of gender, age, occupation and the course userstudied The age question is defined into 4 main group which has short gap ( 13-22, 23-

28, 29-34, >34) which corresponded to each group targeted customer of Markus, which

is student ( junior and seniorz) graduated and young worker Usability evaluationcomposed 20 statement in Wammi questionnaire Extended questions includes 4question which investigate deeper the user feeling of Markus website, such as what is

the best point, the role of Markus website toward user, their internet skills and finallytheir opinion of new field in Markus website The questionnaire was design in Englishthen translated to Vietnamese to limit the language barrier since almost respondents isVietnamese.

2.2.2 Designing website performance test

In Performance-based evaluation, Remote Usability testing method is chosen toperform usability tests on websites This usability evaluation approach also involvedparticipants answering post-task questionnaire – After-Scenario Questionnaire (ASQ)and post-study questionnaire – Website Analysis and Measurement Inventory(WAMMI) A website performance test is task completion experiment There areseveral task was assigned to user to complete The process user’s performance would

be record in term of the number of completed task, time and the number of clicks Thistest was design to support questionnaire method which still have some limitation With

a real performance test, we can evaluation exactly what user really do compare what

we expected them to do It may be different internet level, satisfaction standard and so

on people can feel the website in different usable level The result of questionnaire justshow how usable the website is generally without a specified metrics Therefore, weneed a practical task completion test to know detail how user use our website in howlong, many click to complete a task A small interview with Markus websitedeveloper and user was done to know what the developer want user to do Based onthat interview, the researcher chosen 4 tasks: (1) find a course that user want to study(2) Register for one course (3) Find useful information (4) Contact with Markus staff.The task was built on developer dimension only in order to evaluate the developmentprocess, whether the design of website with oriented task is suitable to user or not

2.4 Sample and data collection

There are two sample were collected, one sample for questionnaire based method,another for website performance method

Firstly, the questionnaire’s sample is 50 - 60 people The researcher want to have afresh experience which the user still remember so sample chosen respondents whorecently studied or are studying and going to stud Since Markus has only 5 course,each course has around 20 people, totally appropriate 100 people for 5 course there forthe scale is only 50 - 60 people which account 50 % and cover equally among course.The respondents were selected through email then fanpage of Markus The researcherwas allowed to take email of student in 5 main course then sent them email In total,there are 200 questionnaire was spread out via email After that, the research post a

public status on “Welcome to Markus” fanpage to attract more respondents whostudied and are going to study in Markus

Secondary, website performance based method, there is a small sample of 3 -5 peoplewho respectively a new user who want to study, one user who studied one course, one

or two user who studied more than one course, one user who work in Markus Sample

of user there is the same internet level and condition but difference in Markus websiteexperience Sample was gather to participate in a real experiment They was assigned 4tasks: (1) find a course that user want to study (2) Register for one course (3) Finduseful information (4) Contact with Markus staff They was asked to record their PCscreen during experiment test

The result from both process will be collect and analysis based own their own metrics

2.5 Data analysis methodology

The result questionnaire and website performance will be collected and analyzed in aspecified method

(Source: Islam and Tsuji, 2011)

To evaluate the usability of website from the perspective of students, responses will beevaluated according to the merit point adopted by (islam & Tsuji, 2011)

Five options are available for each question The options and corresponding merits areshown in table According to (Abdullah & Wei, 2008), usability point for a factor x, isdefined:

x = YΣ (Merit for each question of the factor)] / Ynumber of questions]

Where factor is:

Attractive: statement 1, 6, 11, 16 in part II of questionnaire These statementinvestigate the customer feeling and opinions on how website visually pleasant, and

also offers much of direct interest to the respondents, whether it be functionality orinformation.

Controllability: statement 2, 7, 12, 17 in part II of questionnaire These statements askrespondent to express their feeling when they can navigate around website with easeand do the things they want to do

Efficiency: statement 3, 8, 13, 18 in part II of questionnaire These statements wouldshow how productive the website is based on can quickly locate and do what is ofinterest to them in effective and economical manner They feel that the web siteresponds (possibly, the pages load) at a reasonable speed

Helpfulness: statement 4, 9, 14, 19 in part II of questionnaire These statements expressthe respondents' expectations about its content and structure A site low on Helpfulnesscan be misleading about its layout and content

Learnability: statement 5, 10, 15, 20 in part II of questionnaire These statementsconsiders respondent’s feeling they are able to start using the site with the minimum ofintroductions Everything is easy to understand from the start

The usability level of Markus website was determined by using the correspondingmerit values from 0.000 to 1.000 The greater the value shows greater satisfaction fromthe respondents whereas the lower the value determined lower respondents’satisfaction Table 2.2 shows the usability points and the corresponding level for eachpoint The overall website usability is the mean value of usability points for the fivefactor

Table 2 2: The usability points and corresponding level

(Source: Islam and Tsuji, 2011)

2.5.2 Website performance analysis

The performance metrics used in this study to evaluated collected data includes:

Task success which is used to measure whether the users were able to complete the

tasks successfully Since all the tasks that the participants had to perform on the