Quality measurement and analysis of web based information systems

1.1 An Overview of the Service Quality Assessment in Information Systems 1.1.1 Evolutionary Development of Information Systems Information systems IS used in an organization are very

CHAPTER 1 INTRODUCTION

This research focuses on quality measurement and analysis of web-based information systems (WIS) In this introductory chapter, the evolutionary development of information systems and the service quality assessment in the previous research are described to give the research background of this current study The objectives and scope of this dissertation are then highlighted

1.1 An Overview of the Service Quality Assessment in Information

Systems

1.1.1 Evolutionary Development of Information Systems

Information systems (IS) used in an organization are very critical for its successful operation (Eriksson & Torn, 1991), which lead to higher productivity, better customer service, and better quality of working life (Iivari & Ervasti, 1994) Since their first introduction in the 1960s, the role of information systems has undergone a drastic change

in organizations, as shown in the literature and practice

In the 1960s and the 1970s, IS were used as tools for the data processing In the 1980s they evolved to support managers’ needs to make better decisions In the early 1990s, IS were used to support business goals of organizations and help to create competitive advantages, which is a crucial role of the strategic planning in decision support systems

(DSS) In line with the above discussion, Ward et al in 1990 classified IS into three eras,

namely, Era of Data Processing (1960s), Era of Management of Information Systems (1970s and 1980s), and Era of Strategic Information Systems (1990s) However, since

1993 with the birth of the World Wide Web (WWW), the role of IS has been widely broadened A new era called Web-based Information Systems (WIS) has been paid much

attention in the literature (for example, Lindroos, 1997; Lederer et al., 1998; Tenenbaum, 1998) WIS facilitate the intra- and inter- business process and system integration WIS enable e-business and e-commerce to flourish

Companies are rushing to use the Web to enhance their competitiveness Expenditures

on WIS are at an ever-increasing rate Despite the huge expenditure, WIS have often failed to support business effectively, in particular to meet web users’ needs How to analyze and improve the quality of WIS comes to be a critical issue WIS should be strategically managed

1.1.2 Determinants of Information Systems Success

In the literature, there are three important issues in any IS, that is, products, processes, and services Information products delivered to users include the software and hardware systems, user documentations, training courses, etc Information processes creating information systems refer to system analyses, technical designs, program codes, final handover tests, and so on Information services deal with answering questions, solving problems, addressing concerns and aspirations of end users, and so forth (Whyte & Bytheway, 1996)

Studies into the determinants of IS success have been conducted from the perspective

of products, processes, and services The measure of IS success is a multi-dimensional construct In DeLone and McLean’s (1992) IS success model, six measures of IS success are identified, namely, system quality, information quality, use, user satisfaction, individual impact, and organizational impact The basis of the D & M IS success model is

product-oriented (Pitt et al., 1995) ‘System quality’ measures technical success;

‘information quality’ measures IS outputs; ‘use, user satisfaction, individual impacts’ and

‘organizational impacts’ measure system effectiveness However, with the emergence of end user computing in the mid-1980s, especially with the development of WIS, the role of

IS department within the organization has changed It has been broadened from product developers and operations managers to information and service providers Recent research has agreed that service quality measure is a part of IS success (For example, Kettinger &

Lee, 1995; Pitt et al., 1995) The updated D & M IS success model (2003) has integrated

service quality into their conceptual model

1.1.3 Achieving High Quality in Customer Service

High quality in customer service is the key factor leading to business success in today’s competitive world The delivery of high service quality has been increasingly considered

as a prerequisite for success in the marketplace However, people’s awareness of the importance of quality and the exploration of formal methods for quality control and improvement is an evolutionary process (Montgomery, 1996), especially in the domain of service quality Unlike product quality, which can be measured objectively by such indicators as the number of defects/nonconformists, service quality is an abstract and elusive construct due to the following three unique characteristics:

(1) Intangibility, where a mismatch can occur between the service quality perceived

by the information producers and consumers (Parasuraman et al., 1988)

(2) Heterogeneity, where the performance of service delivered may vary from user to user, system to system, and from day to day (Kettinger & Lee, 1995)

(3) Consumer participation, where service consumers will participate in the consumption of the service

In the absence of objective measures, developing an appropriate approach to assess service quality has raised a great deal of interest among researchers, from both theoretical and empirical perspectives The SERVQUAL model is one of the most widely used models in the service quality literature The SERVQUAL model is based on the concept of

service quality gaps (Parasuraman et al., 1988) In this dissertation, service quality is

defined as a function of the gap between customers’ expectations of a service and their perceptions of the actual service delivered Over the past two decades, the SERVQUAL model has been modified to suit the service nuances of specific industries, e.g., health-care, tourism, information systems, festivals, and the automobile industry, etc Since there are numerous differences between the traditional one-way communication channels and the new two-way communication via web-based exchanges, the SERVQUAL model developed for the traditional customer service should be modified to better suit the context

of web-based customer service Research on this modification should be useful for web designers and information service providers (ISP) It will also make some contributions to the information systems literature

1.2 Objective

The proposed research focuses on quality measurement and analysis of web-based information systems Since the 1980s, the quality revolution in manufacturing had a profound impact on the competitiveness of companies Nowadays the service industry is beginning to understand that quality does not improve unless it is measured The main objective of this research is to develop a strategic and quantitative approach to identify elements of superior web-based service quality, to measure online customer satisfaction, and to analyze the determinants of high service quality for continuous and strategic improvement Using the SERVQUAL model as a starting point, some conceptual and methodological issues involved in the management of web-based service quality are discussed This research will provide guidance for practitioners on how to assess web-based service quality, how to implement service quality measures, and finally how to achieve online customer satisfaction

To achieve the stated objective, the research topics in this thesis include:

(1) To explore the differences between web-based customer service and traditional communication channels, then identify the shifts in service quality dimensions in the context of this new information age

(2) To propose a general framework to assess web-based service quality, to implement measures, and finally to achieve service quality, from the perspective of web users instead of the system development related to information technology (IT)

(3) To conduct a worldwide and online survey to develop a parsimonious measurement instrument for web-based service quality with sound psychometric properties in terms of reliability and validity

(4) To conduct a campus-wide survey to understand users’ perceptions of web-based service quality A variation analysis is to be carried out in terms of web users’ experience

(5) To identify existing problems and limitations of the well-established service quality instruments, such as the SERVQUAL model that conceptualizes a linear and symmetric relationship between the service quality gaps and the overall service quality This research is to investigate the asymmetric and nonlinear nature

of this relationship

(6) To seek rigorous and data-driven approaches to develop robust constructs on which to categorize web users In this dissertation, new approaches to user pattern identification and categorization are to be proposed

(7) To highlight some opportunities for further development and in-depth research on web-based service quality measurement, analysis, and improvement Some suggestions on the strategic information systems planning (SISP) in WIS are to be proposed for continuous improvement

1.3 Scope of the Research

The proposed research focuses on quality measurement and analysis of WIS The relationship among various concepts is configured in Figure 1.1 as below:

Use r

Fe edback

User Information Satisfaction

Behavioral Intentions

Affect

Business Interface

Web Users

Web-based Information Systems

Figure 1.1 Concepts and Relationships in this Dissertation

The contents of the main body of this thesis and the relationships among chapters are described in Figure 1.2

Chapter 7 Prioritization of

Service Attributes

Chapter 5 Case Study on

Managing Web-based Service

Quality

Chapter 6 Case Study on

Variation Analysis of Web

Users

Chapter 4 A General Framework to Develop Quality

Measures

Chapter 9 Using Alternating

Least Squares Scaling in Web User Categorization

Chapter 8 A Fuzzy Set-based

Approach to Web User

Categorization

Web-based Service Quality

Improvement

Web User Satisfaction

Chapter 3 Identification of Shifts in Web-based

Service Quality

Figure 1.2 Scope of this Research

The whole dissertation is divided into three parts Part I includes Chapter 2, which is a literature review of the service quality assessment in web-based information systems, from both theoretical and empirical perspectives This brief overview on the existing literature

provides the research background for the current study This introductory chapter also pinpoints the limitations in the existing literature, which elucidates the motivation for the current study

Part II comprises five chapters (Chapters 3 - 7) This part deals with both a conceptual and operational framework to manage web-based service quality After an exploration of the differences between web-based and traditional customer service, Chapter 3 notes that the SERVQUAL model, developed for the traditional customer service, might require adaptation for the use in this information age Confirmatory Factory Analysis (CFA) is implemented in Chapter 3 to serve the purpose of identifying the shifts in service quality dimensions Chapter 4 proposes a general framework as an effective and operational procedure to assess web-based service quality, to implement measures, and finally to improve service quality Field studies are reported in Chapters 5 and 6 Chapter 5 describes an extensive empirical field study conducted online and worldwide It focuses

on developing a parsimonious measurement instrument with sound psychometric properties Chapter 6 intends to understand users’ perceptions of web-based service quality, regarding a local university as a campus-wide information system This study not only develops a measurement scale which gives a good indication of users’ perceptions of web-based service quality, but also further conducts a variation analysis in terms of web users’ experience Chapter 7 is to develop an operational procedure that would prioritize customer service attributes in a simple, inexpensive, and accurate manner This study discusses problems with the well-established instruments such as the SERVQUAL model

in the prioritization of service attributes for improvement By addressing the existing problems, Chapter 7 investigates the asymmetric and nonlinear nature of the relationship

between the service quality gaps and the overall service quality A model is developed to advance utility theory into the prioritization at the service attribute level as well as at the service dimensional level In the service literature, this is a first attempt at integrating utility theory into the prioritization of service attributes to help achieve quality in customer service

Part III includes Chapters 8 and 9 The focus of this part is to propose two new approaches to web user identification and categorization User-centered information systems represent the recent effort of delivering information more effectively in the modern electronic age It is important for ISP to accommodate individual differences in web users and to tailor the web content according to the needs of a particular user or a set

of users This is in order to achieve system applicability, efficiency, and effectiveness Although user profiling is a principal element of web personalization, surprisingly, not enough attention has been paid to it either in intranet design methodology (IDM) or in Internet commerce development methodology (ICDM) Most of the earlier approaches in the IS literature tend to model web users in terms of broad linguistic terms This has resulted in the use of inconsistent theoretical constructs and limited applicability across studies Chapter 8 presents a data-driven approach to web user categorization with the help of fuzzy set theory A multi-attribute structure is developed to enable fine distinctions among web users Chapter 9 focuses on developing a user categorization process model by using the alternating least squares scaling (ALSCAL) method for categorical data In this model, numerical quantifications can be assigned to each category of the categorical variable The predefined measurement levels are adjusted accordingly Hence the optimal scale can be defined for each categorical variable Both methods discussed in Chapters 8

and 9 would be useful for web service providers to customize the content and structure of their web site, regarding to users’ specific and individual needs

Chapter 10 concludes this dissertation by outlining the major contributions of the current study The limitations of this dissertation and other potential research topics are also presented in this concluding chapter

CHAPTER 2 LITERATURE REVIEW

Since the birth of the WWW in 1993, WIS have provided a dynamic and distributed platform for interactive business applications WIS facilitate intra- and inter-business processes and systems integration WIS have enabled e-business and e-commerce to

flourish (Lindroos, 1997; Lederer et al., 1998; Tenenbaum, 1998) The two-way

communication via WIS has become increasingly popular in electronic transaction, publication, broadcasting, and other service applications The literature has paid much attention to developing a framework to identify elements of superior web-based service quality, to adapting well-established quality tools to measure online customer satisfaction, and finally to implementing measures to analyze and improve service quality

A close look at the literature suggests that the advancement of WIS has been along two major lines From a theoretical point of view, researchers have attempted to explore issues concerning the characteristics and nature of WIS in order to derive conceptual models or measurement instruments On the other hand, WIS have undergone a widespread dissemination as witness of the enthusiasm of numerous companies rushing to use the Web

Literature surveys on WIS from the above two perspectives are desirable and would benefit both practitioners and researchers The purpose of this chapter is to: 1) provide a thorough review on the WIS literature, 2) highlight some deficiencies and limitations in

the existing service management and information systems literature, and 3) hence stress motivations for the current study of managing web-based service quality This chapter is organized as follows First, an overview of the WIS literature is presented, including the definition and classification, the evolutionary development and features, and the dissemination and applications of WIS Thereafter is a survey on the development of the service quality assessment and its applications in traditional information systems Since there are some similarities between web-based and traditional information systems, the existing literature on traditional information systems will shed some light on the development of new quality measures Finally, limitations of the existing literature are discussed Several directions for the further improvement and refinement are highlighted

2.1 An Overview of Web-based Information Systems

The public-domain Internet Protocol (IP) network and the WWW provide a dynamic and distributed platform for interactive business applications Using the Internet and Web technologies, WIS have begun to remove the barriers between different forms of communication

2.1.1 Definition and Classification of WIS

Due to its short history and diverse applications, there is no standard definition of WIS in the literature Generally speaking, WIS are information systems based on web

technologies (Isakowitz et al., 1998) A web presence is generally a part of WIS Web

browsers serve as a common interface Web technologies, such as related protocols and standards, support the basic functions of an information system

WIS are systems, not a set of web pages The major difference between these two is that web pages are designed to be browsed like magazines whereas WIS enable users to perform work, which is usually tightly integrated with other non-WIS such as databases and transaction processing systems (Wang, 2001) WIS are linked systems of entities made up of both humans and computers The human entities consist of: 1) users who interact with each other and the information device, 2) designers who create the device, and 3) information specialists who select designers and organize the data In general, WIS

are information systems with characteristics of web systems (Terveen et al., 1999)

WIS are classified according to their scopes, from those locally tailored to those with a world-reach They range from being simple web pages to those that are highly complex

and integrated (Isakowitz et al., 1998) A classification of four general types of WIS

includes:

(1) Intranets used to support internal work

(2) Web-presence sites such as marketing tools designed to reach external consumers (3) Electronic commerce (e-commerce) to support online transactions

(4) Extranets as a blend of internal and external systems to support the

business-to-business communication

It is necessary to clarify two terms, i.e., intranet and extranet Both are based on IP technologies and Web technologies An intranet is a private and secure IP network, deployed inside an organization IP and Web technologies are used to reduce or eliminate internal communication problems by providing easier accesses to information (Dunn & Varano, 1999) An extranet is a hybrid that integrates intranet with the public-domain

Internet Extranets can support partners, suppliers, distributors, contractors, and others that operate outside the physical boundaries of the organization A comprehensive WIS is very critical to the success of the business, with the Internet supporting public uses, the intranet serving internal business needs, and the extranet serving as a bridge between the public-

domain Internet and the private corporate intranet

2.1.2 Stages and Features of WIS

McMillan (1999) built a four-stage model to describe the evolution of web-based

communication At stage 1 there were static web-based information systems that comprise

corporate brochures to attract audiences, promote products and services, or perform persuasive but non-selling communication functions Such WIS were composed of static hypertext documents In 1993, having a server on the Web was enough to show that one

was a pioneer (Lindoors, 1997) At stage 2 dynamic web-based information systems

facilitated in-depth searchable content such as the database or the archived information There was a two-way communication, such as the customer feedback gathered via emails,

direct orderings, and online surveys At stage 3 adaptive web-based information systems

focused on active environments that facilitate the execution of online transactions commerce was the typical application at this stage, including online auctions, individual

E-negotiations, shopping cart applications, and so forth At stage 4 commercial

manifestation of virtual communities focused on the application of agent-mediated

architectures, such as service brokerage

Among the various features and characteristics of WIS discussed in the literature, two

common and distinguishing features are identified They are super-connectivity and

hypermedia (Grosky, 1997; Isakowitz et al., 1998; Bimbo, 2000; Wang & Head, 2001)

The term super-connectivity (coined by Turoff & Hitz, 1998) has two meanings: 1) the phenomenon of a perfect transmission throughout the human habitations of the universe via computer networks, and 2) the interconnection of all social and economic institutions

as a result of the communication via computer networks It can be referred as a strong power of WIS for connection and interaction WIS can be regarded as a new medium of the human communication Since there is no restriction separating information providers from information receivers, WIS can provide close connections between users, between users and systems, and between systems

Hypermedia is concerned about two aspects, i.e., hyperlinks and multimedia The linear assumption concerning the structure and presentation of the printed text is violated when users access the electronic media Hyperlinks provide the connection among related content and web sites However, although they improve the ease of implementation and scalability in a networked environment, they also result in user disorientation (Lowe & Hall, 1999) Multimedia is used to provide information through various media, such as text, graphics, audio, and video Information can, thus, be presented in such a way that more closely match its existence in the real world Customers may then receive the information more efficiently and effectively, and should be able to evaluate alternatives better (Wang & Head, 2001)

2.1.3 Benefits of Implementing WIS

Since the rush to use the Web has been enthusiastic, researches have started to dig out the motivation and potential benefits that companies are seeking from their web-based

information systems Dunn & Varano (1999) argued that the investment on WIS can be

returned by such strategic advantages as: (1) lower operational costs; (2) better

collaboration with customers, suppliers, and other partners, both domestic and global; and

(3) reduced business cycle time According to the statistics in the surveys (Lederer et al.,

1998), enhancing competitiveness tops the list of 32 potential benefits sought from the

applications of WIS Table 2.1 lists the means of WIS to enhance competitiveness in the

mentioned three aspects

Table 2.1 Means by WIS to Enhance Competitiveness

Aspects Means Lowering costs 1 Eliminating sales persons and outlets by letting

customers instead of employees to enter order information

2 Expensive Proprietary electronic data interchange (EDI) could be replaced with WIS

Focusing on customer

groups

1 WIS facilitate the individualized tailoring of outputs to users based on the recent purchases, information requirements, and voluntary demographics

2 Building users mailing lists based on some criteria

3 Using automated agents or search engines to search newsgroups for potential customers

Market differentiations 1 Global market coverage

2 Efficiency of marketing channels

3 Opportunities for new services based on web technology

Tenenbaum (1998) suggested two major competitive advantages achieved by WIS: 1)

the potential of WIS for transforming commerce; and 2) the ability of WIS for building

links between buyers, sellers, and organizations in an innovative way The traditional

controlled retail-sales process, that is, retailers set the price, the terms, and the operating

hours; buyers could take it or leave it, has been totally changed by WIS Consumers can

scour the Internet using shopping agents to find the cheapest flight, the closest restaurant,

or the best health plan Web sites are available across the world, 24 hours 7 days The whole marketing environment has largely changed, which promotes a more dramatic

competition

2.2 Development of the Service Quality Assessment

Interest in service quality has increased substantially in the recent decades as being evidenced by numerous books and academic papers We have confirmed this point by estimating the number of papers that have been published in professional and academic journals on this subject For example, a search for ‘service quality’ in the Sciencedirect database reveals that the number of publications on this concept is steadily growing throughout the past decade, especially in recent years (see Figure 2.1) It should be noted that many other publications appeared in other sources that are not registered in the Sciencedirect database

Achieving quality in customer service is to enhance a company’s competitiveness With the growth of information on the Internet and the development of more sophisticated searching tools, it is appropriate to monitor and enhance service quality It is necessary to establish some basic definitions by answering questions such as ‘what is service quality? ‘,

‘who are our customers?’ and ‘how can service quality be measured and improved?’ Clearly there has been a profound interest and an increasing need to develop tools for the assessment of service quality The following sections are focused on the definition of

service quality, the conceptual models and measurement instruments, along with conceptual and empirical problems

Figure 2.1 Growths of Publications on Service Quality

2.2.1 Definition of Service Quality

Service is regarded intrinsically with characteristics of intangibility, perishability, simultaneity, and heterogeneity (Dotchin & Oakland, 1994) Therefore, service quality is

an elusive and abstract construct that is difficult to define and measure (Cronin & Taylor, 1992) There is no value in a product/service until it is in the hands of its customers Service quality is determined by the interaction of all those factors that affect the process

of making a product/service available to its customers (Christopher, 1998) The debate on

the definition of service quality has various contributions on the nature of service quality (Harvey & Green, 1993; Rust & Oliver, 1994; Russell & Miles, 1998)

In view of different perspectives, quality can be defined as:

(1) Transcendent perspective: Quality is distinctive and linked to notions of excellence that is unattainable by most

(2) Product perspective: Quality is a set of attributes that determine product suitability and is a function of the distance between actual and ideal attributes

(3) Customer perspective: Quality is the ability of a product/service to fulfill customers’ requirements

(4) Manufacturing perspective: Quality is a product’s degree of conformance with the engineering and design specification

(5) Value perspective: Quality in terms of return on investment is that a better outcome can be achieved at a lower cost

Service quality falls into the third category, which is focused on customers’

requirements Parasuraman et al (1985, 1988) defined service quality as a function of the

gap between customers’ expectations of a service and their perceptions of the actual service delivered For the use of further illustration, it is necessary to classify the definition of expectations, perceptions, and perceived service quality

Expectations are the pre-trial beliefs about a product/service (Boulding et al., 1993)

Based on their cumulative experiences with a particular product/service, customers have some forms of expectations, ranging from ill-defined in unfamiliar situations to well-

defined in familiar situations Perceptions are the reality of service that customers have experienced Perceived service quality can therefore be defined as the discrepancy between what the customer feels a service provider should offer (expectations) and what the service provider actually offers (perceptions) Achieving quality is to minimize this discrepancy

The World Wide Web Consortium (W3C) defines a Web service as ‘a software system designed to support interoperable machine-to-machine interaction over a network It has

an interface described in a machine-processable format (specifically WSDL) Other systems interact with the Web service in a manner prescribed by its description using SOAP-messages, typically conveyed using HTTP with a XML serialization in conjunction with other Web-related standards’ Here, WSDL means ‘Web Service Description Language’ XML means ‘Extensible Makeup Language’ HTTP means ‘Hyper Text Transfer Protocol’ SOAP refers to the formal set of conventions governing the format and processing rule of a SOAP message These conventions include the interactions among SOAP nodes that generate and accept SOAP messages for the purpose of exchanging information along SOAP exchange Obviously this definition of Web service is focused on engineering-oriented performance instead of customer service

Currently, there is another term QoS (Quality of Service) widely used in the IP network literature Three approaches to QoS definition are proposed by International Telecommunications Union (ITU), European Telecommunications Standards Institute (ETSI), and The Internet Engineering Task Force (IETF), respectively All of these approaches are based on a general QoS model developed by Hardy (2001) In this general

model, three notions of QoS are defined: intrinsic, perceived, and assessed The definition

of perceived QoS is parallel to perceived service quality discussed above The perceived QoS reflects the gap between customers’ expectations and perceptions of a particular information service The intrinsic QoS pertains to service features stemming from technical aspects, which is determined by the network design and provisions of network access, terminations, and connections The assessed QoS is to assess the customers’ decision on whether to continue using the service It is influenced by perceived QoS, service price, and so on Other terminology related to QoS, such as class of service (COS), grade of service (GOS), service level agreement (SLA), and service level specification

(SLS), are clarified and compared in Gozdecki et al., (2003) and Oodan et al., (2003)

2.2.2 The Gap Model for Service Quality

In line with the hypothesis that service quality is the discrepancy between customers’ expectations and perceptions, conceptual and empirical service quality models have been developed Among those models, the conceptual gap model by Parasuraman, Zeithaml and Berry (1985) made a substantial contribution to the understanding of service quality

concept (see Figure 2.2) Starting from the PZB gap model, Parasuraman et al (1988) first

introduced the SERVQUAL measurement instrument as a means of measuring service quality, which was designed to investigate the discrepancy between customers’ expectations and perceptions By addressing the problems with the SERVQUAL measurement instrument, both conceptually and empirically, the improved SERVQUAL measurement instruments were stemmed from it in the service quality literature

The description of the five gaps is presented as below:

Gap 1: Between customers’ expectations and management perceptions of customers’ expectations This gap consists of discrepancies that arise because executives do not understand what customers consider to be high quality This may result from a lack of marketing research and demand analysis, inadequate upward communication, and too many organizational layers, and so on

Gap 2: Between management perceptions of customers’ expectations and service specifications This is a rather wide gap in many companies, which occurs when what management perceives to be customers’ expectations are not consistent with the actual established service quality specifications Some reasons for this gap can be planning mistakes or insufficient planning procedures, absence of goal setting, and insufficient support from top management, and so on

Gap 3: Between service quality specifications and the actual service delivery The difference between service specifications and the actual service delivery is the service-performance gap caused by employees who are unable or unwilling to perform the service

at the desired level Even when the guidelines exist for performing the excellent service, its delivery may not be up to the standard due to a large variability in the employee performance, which results from role ambiguity, role conflict, poor employee job fit, and inappropriate supervisory control systems, etc

Gap 4: Between the actual service delivery and the external communications to customers about service delivery Customers’ expectations are affected by the media

advertising and other forms of communication It is essential to deliver services that customers perceive as high in quality through accurate and appropriate company communication, advertising, and public relations that do not over-promise or misrepresent

CONSUMER

GAP1

Management Perceptions of Consumer ExpectationsGAP2

Service Delivery (Including pre- and post-contacts

Translation of Perceptions into Service Quality Specs

External Communications

to Consumers GAP4

GAP3

Perceived Service

Personal Needs Past Experience

Expected Service GAP5

MARKETER

Word of Mouth

Communications

Figure 2.2 The PZB Gap Model for Service Quality

(Reproduced from Parasuraman et al., 1985)

Gap 5: Between customers’ expectations and perceived service This gap results when one or more of the previous gap occurs A good service quality is one that matches or exceeds customers’ expectations Judgments of high or low service quality depend on how

consumers perceive the actual service performance in the context of what they expected

2.2.3 The SERVQUAL Measurement Instrument

Designed to investigate Gap 5 in the PZB gap model, the SERVQUAL measurement

instrument was first introduced by Parasuraman et al (1988) Later the SERVQUAL

measurement instrument has been widely tested as a valid means of measuring service quality in the service management literature

A close look at the literature suggests that the advancement of the SERVQUAL measurement instrument has been along two major lines From a theoretic point of view, the SERVQUAL measurement instrument itself has experienced its progressive advancement along with a number of conceptual difficulties, such as low reliability, unstable factor structure, and poor validity On the other hand, over the past two decades, the SERVQUAL measurement instrument has been modified to suit the service nuances of

specific industries, e.g., health-care (Lim & Tang, 2000; Lee et al., 2000), tourism (Armstrong et al., 1997), information systems (Kettinger & Lee, 1997), festivals (Childress & Crompton, 1997), and the automobile industry (Andaleeb & Basu, 1994)

2.2.3.1 The Development of the SERVQUAL Measurement Instrument

The development of the SERVQUAL measurement instrument began with a set of consumer focus group interviews 97 items were generated and grouped into identified 10

dimensions to estimate service quality The dimensions were summarized below

(Parasuraman et al., 1985):

(1) Tangibles, which refers to the physical environment in which the service is presented

(2) Reliability, which is the consistency of performance and dependability

(3) Responsiveness, which is the willingness to help the customer

(4) Competence, which is the processing of the required skills and knowledge to perform the service

(5) Courtesy, which refers to the supplier’s behavior, e.g., politeness, consideration and kindness

(6) Credibility, which means trustworthiness, believability, and honesty of the service provider

(7) Security, which means freedom from danger, risk and doubt

(8) Access, which is the ease of making contact with the supplier

(9) Communication, which is the ability of talking in a way which is understandable to the customers

(10) Understanding/Knowing the Customer, which involves making the effort to understand the customer’s needs

The further research and scale purification procedures led to the deletion of some items and reassessment of dimensions As a result, the number of dimensions was reduced

to five as listed below (Parasuraman et al., 1988):

(1) Tangibles: physical facilities, equipment, and appearance of personnel

(2) Reliabilities: ability to perform the promised service dependably and accurately (3) Responsiveness: willingness to help customers and to provide prompt service (4) Assurance: knowledge and courtesy of employees and the ability to convey trust and confidence

(5) Empathy: caring, individualized attention that an organization provides to its customers

The SERVQUAL measurement instrument provided a basic skeleton through its expectations/perceptions format encompassing statements for each of the five service quality dimensions The skeleton, when necessary, could be adapted or supplemented to fit the specific research needs or the characteristics of a particular organization (Kettinger &

Lee, 1994)

2.2.3.2 Difficulties with the SERVQUAL Measurement Instrument

The difficulties with the SERVQUAL identified in the literature can be grouped into two main categories, namely, conceptual and empirical, although the boundary between them

is blurred because they are closely interrelated (Van Dyke et al., 1997)

2.2.3.2.1 Conceptual Difficulties with the SERVQUAL Measurement Instrument

Many problems associated with the SERVQUAL measurement instrument stem from the operationalizaiton of a service quality construct that is theoretically grounded in the gap

model (Van Dyke et al., 1997) Rather than develop an instrument to directly measure the

perception of service quality that is the outcome of the cognitive evaluation process, the

SERVQUAL measurement instrument (Parasuraman et al., 1988, 1991) separately

measures the expected level of the service and the experienced level of the service The service quality is the difference between customers’ expectations and perceptions The appropriateness of operationalizing service quality as a difference or gap score is certainly worthy of empirical evaluation because the implicit assumption that subtraction accurately

portrays this cognitive process seems overly simplistic Van Dyke et al (1997) suggested

that the direct measurement of customers’ perceptions of service quality seems more likely to yield a valid and reliable outcome than the SERVQUAL measurement instrument

Another issue of interest about the difficulties associated with the SERVQUAL measurement instrument is concerned with the ambiguity of the construct ‘expectations‘

In the SERVQUAL measurement instrument and follow-up studies, expectations are variously defined as desires, wants, what a service provider should possess, normative expectations, ideal standards, desired service, and the level of service a customer hopes to

receive (Parasuraman et al., 1985, 1988, 1991) The multiple definitions and

corresponding operationations of ’expectations’ in the SERVQUAL literature result in a

concept that is loosely defined and open to multiple interpretations (Van Dyke et al.,

1997) These various interpretations of the ‘expectations’ construct lead to a number of measurement problems and result in even opposite impacts on the perceived service quality Fortunately, findings of the SERVQUAL developers’ follow-up studies have proved that the use and interpretation of the ‘expectations’ construct is appropriate in

terms of diagnostic value (Parasuramn, et al, 1994a, b)

In some industrial settings, such as web-based information systems, Zeithaml et al

(2000) found that customers can articulate with ease the nature and sources of their expectations for traditional service quality, but they have difficulties in expressing preconception standards when buying/owning technological products Their expectations are subject to change as life circumstances shift There is a mutual influence between

customers’ expectations and perceptions (Boulding et al., 1993, 1999) Therefore the

traditional service quality measures based on a service gap between customers’ expectations and perceptions may not be applicable in this new circumstance It is necessary to build an online customer service framework based solely on web users’ perceptions

2.2.3.2.2 Empirical Difficulties with the SERVQUAL Measurement Instrument

Empirical problems are, by and large, the result of those conceptual difficulties, most notably the use of the difference scores, in conjunction with the theoretical nature of the processes used in the construction of the original five dimensions of service quality In sum, among the difficulties related to the use of difference scores discussed in the literature are low reliability, unstable dimensionality, and poor validity

Peter et al (1993) clearly indicated that by using Cronbach’s alpha, researchers tended

to overestimate the reliability of difference scores, especially when the component scores were highly correlated

Another problem with the SERVQUAL measurement instrument concerns the poor divergent and convergent validity of the model The study carried out by Cronin & Taylor

(1992) confirmed that divergent and convergent validity of perception-only score was superior to that of the SERVQUAL scores They therefore proposed the SERVPERF

model based on customers’ perceptions only O’Neill et al (2003) addressed this debate

and concluded that both techniques are reliable and valid measures of online quality But the SERVQUAL measurement instrument is more favored by practitioners because of its ability to pinpoint service failures and direct continuous quality improvement efforts

Another criticism on the SERVQUAL measurement instrument is on its unstable dimensionality Carman (1990) and Babakus & Boller (1992) proposed that the number of service quality dimensions might be a function of the particular service industry, i.e is contextualized Modification of the SERVQUAL dimensions can be either theoretical or operational At the beginning, the five dimensions remained with modified items to suit the various industries Gradually, research has further developed service quality measurement with refinements suitable for various circumstances Table 2.2 depicts the applications of the SERVQUAL measurement instrument in various industries and their related dimensions

The number of empirical studies listed in Table 2.2 supports the conclusion that the five-dimensional structure of the SERVQUAL measurement instrument can only serve as

a meaningful conceptual framework Modification of dimensions and attributes should be subject to the specific contexts

Table 2.2 Applications of the SERVQUAL Measurement Instrument and Their

Related Dimensions

2.2.4 Users’ Expectations

One criticism associated with the SERVQUAL measurement instrument is concerned with the ambiguity of the term ‘expectations‘ Now it is necessary to discuss the nature and determinants of customers’ expectations Since the whole dissertation is focused on WIS, and customers in WIS are web users, we will pay special attention to users’ expectations

in WIS

2.2.4.1 Determinants of Users’ Expectations

Pitt et al (1995) proposed a determinant model of users’ expectations in traditional

information systems (see Figure 2.3) There are five factors affecting users’ expectations that is, word-of-mouth communications, personal needs, past experiences, IS department communications, and vendor communications

The study of Ryker et al (1997) adopted and expanded the Pitt’s model In Ryker’s

models, determinants of users’ expectations were classified into three categories: 1) sources internal to the organization, 2) sources external to the organization, and 3) past experiences And they carried out an empirical study on the relationship between the sources of users’ expectations and the overall user information satisfaction

Figure 2.3 Determinants of Users’ Expectations

Word-of-mouth Communication

IS Communications

Expected Service

Perceived Service

Personal Needs

Past Experiences

Vendor Communications

(Reproduced from Pitt, et al., 1995)

2.2.4.2 Nature of Users’ Expectations

Studies into the nature of users’ expectations have evolved from the perspective of a point estimate to a range of levels, and finally to a distribution based on users’ cumulative experiences In the marketing literature, beginning with the SERVQUAL measurement

instrument originally invented by Parasuraman et al in 1988, service quality models have

universally focused on a point estimate of expectations Later the literature has tried to distinguish between ‘should expectation’ and ‘will expectation’ (Van Dyke et al., 1997) Then the discussion of customers’ expectations has shifted from a single point estimate to

a zone of tolerance (Zeithaml et al., 1993) In the model of zone of tolerance, there are

three types of service expectations, namely, the desired service, the adequate service, and the predicted service The desired service is defined as the level of service that customers hope to receive, whereas the adequate service is defined as the minimum standard of the acceptable service This is because customers realize that the desired service may not be possible, and they will form a lower level of expectations The discrepancy between desired and adequate service results in the ‘zone of tolerance’

In the development of a dynamic process model of service quality, Boulding et al

(1993) first used the Bayesian decision theory to explicate and test the relationships among customers’ expectations, perceptions, and intended behavior Under the suggestion

of Anderson & Sullivan (1993) that a distribution rather than a point estimate should be

implemented in the updating approach, Rust et al (1999) developed a Bayesian updating

theory in conjunction with a diminishing returns utility function Bordley (2001) furthered the study by reinterpreting utility as a function of the uncertainty-adjusted gap between the actual performance and the expectation Bordley also discussed the impact of information uncertainty on the customers’ repurchase intentions In these studies, it was assumed that customers formed a certain distribution for their expectations based on their cumulative experiences with the product/service

2.3 Service Quality Assessment in Web-based Information Systems

In the literature, there are three important issues in any information system, that is, product, process and service Studies into the determinants of the success or failure of information systems have been conducted from the perspective of products, processes, and

services, respectively Earlier research focuses more on the observable and tangible attributes of information products, such as response time, data volume, and the extent of systems usage When studies shift from information products to information processes, focuses are still on the tangible attributes such as the number of errors occurring within the process, the level of user involvement, and the milestones at which user approval is given Due to the similarities between the IS department and other service operations, more recent research has sought to borrow the concept of service quality from the marketing

research (Pitt et al., 1995)

WIS are information systems with characteristics of web systems Thus, the measures

or instruments developed to measure the service quality of traditional information systems will shed some insight on the way to develop new instruments to measure WIS quality A literature review on IS reveals that among various quality instruments user information satisfaction (UIS) and the modified IS SERVQUAL measurement instrument, have received considerable attention However, due to some obvious differences between traditional and web-based information systems, those existing measures may no longer be appropriate for WIS Consequently there is a pressing need to adapt those well-established quality instruments to suit this new context

2.3.1 User Information Satisfaction (UIS)

In the literature, user information satisfaction (UIS) is a frequently used measure of information systems effectiveness UIS has been proved as a valid and subjective measure

as a substitute for objective determinants of information systems effectiveness (Ives et al.,

1983)

2.3.1.1 Definition of UIS

In the IS literature, UIS is the most frequently-used measure of IS effectiveness However, there is no agreed definition of UIS UIS has been variously associated with the terms such as ‘felt needed’, ‘system acceptance’, ‘perceived usefulness’, and ‘feelings about a

system’ (Ives et al., 1983) The underlying assumption is that a user satisfied with a

system will perform better than a user with a poor or neutral attitude towards the system

In other words, an effective system must have some positive influence on user behavior Results from Gatin’s research (1994) indicated that a relationship does exist between satisfaction and the behavior of different user groups UIS has shown to be a valid

measure (Bailey and Pearson, 1983; Ives et al., 1983)

Bailey & Pearson (1983) defined that satisfaction in a given situation was the sum of

one’s feelings or attitudes toward a variety of factors affecting that situation Ives et al

(1983) defined UIS as the extent to which users believed that the information system available to them met their information requirements Kim (1990) regarded UIS as a function of organizational factors and the discrepancy between the expected information service quality and the perceived information service quality This definition paralleled UIS with the concept of service quality in the marketing research In the marketing research, service quality involves a comparison between customers’ expectations and perceptions of the service delivered

UIS has also received criticisms, both theoretical and empirical One criticism, of a conceptual nature, is that user information satisfaction is a good surrogate for system success Melone (1986) contended that satisfied users alone are not indicative of a

successful business There is no theoretical basis that satisfied users would perform better than unsatisfied users Numerous empirical studies (Rivard & Huff, 1988; Doll & Torzadeh, 1991; Magal, 1991; Whyte & Bytheway, 1996) have suggested a plethora of questionnaires to use in determining user information satisfaction The wide variety makes

it difficult for practitioners to adopt a generalized model based on some common grounds

2.3.1.2 Conceptual Models of UIS

Based on different theoretical backgrounds and different viewpoints, there are several conceptual models developed in the IS literature Among them, two conceptual models of UIS have received much attention, that is, Kim’s model (Kim, 1990) and Shirani’s model

(Shirani, et al., 1994)

2.3.1.2.1 Kim’s Conceptual UIS Model

Kim (1990) developed a conceptual UIS model, applying the relationship observed in the consumer satisfaction research (shown in Figure 2.4) In Kim’s model, UIS was defined as how well the information service level delivered matched users’ expectations before implementation The discrepancy, in turn, was determined as a series of gaps which were related to management of information systems (MIS) development process (GAP1 and GAP2) and MIS delivery process (GAP3) In the process of building the conceptual model, the general systems theory was applied that inputs were processed into outputs as a result of the interaction among subsystems

GAP1 is related to the first stage of the MIS development process, when determining what kinds of information are needed from the proposed MIS to meet their requirements

Three reasons outlined by Davis & Olson (1985) summarize the difficulties for MIS personnel in obtaining a correct and complete set of information needs from users This results in a discrepancy between users’ expectations and MIS personnel interpretations of users’ expectations, which influence user satisfaction with information quality

GAP3GAP2

GAP1Expected

Information

Service

MIS Personnel Interpretation of User Expectations

Quality of EstablishedMIS

Delivered InformationService

Organizational Factors

Figure 2.4 Kim’s Conceptual Model of User Information Satisfaction

GAP2 is related to the MIS design and the installation stage of the MIS development process This process translates the MIS interpretation of users’ expectations into physical MIS A variety of factors, such as resource constraints, market conditions, and top management commitment, may result in a discrepancy between MIS employees’ interpretations of users’ expectations and the actual information systems established

GAP3 is related to the information delivery process, when information service is delivered to user primarily through the user interface process (interaction between human users and the MIS) Users in information systems may be heterogeneous (users could be either novices or experts); therefore, a well-designed interface is critical to the success of a system

Organizational factors can play a significant role in UIS by influencing the above three gaps during the MIS development and information delivery processes The gaps can be reduced by the extensive user participation in the information requirement determination

process (GAP1) (Ives & Olson, 1984; Baroudi, et al., 1986) The support from top

management can reduce GAP2 by providing sufficient organizational resources Proper user training is an important factor to reduce GAP3

The feedback loop from the MIS effectiveness to UIS implies that MIS effectiveness directly influences users’ perceptions, and thereby influences users’ future expectations on the information service quality In the MIS research, expectation should be based on experience norms, that is, what users will expect from a given service provider is affected

by their experience with that specific type of service organization (Woodruff et al., 1983)

Users learn from experience In other words, users’ perceptions play a roll in the formation of expectations

Kim (1990) defined UIS as the discrepancy between users’ expectations and perceptions of information service quality This discrepancy is derived from the gaps

associated with information needs determination, MIS design and installation, and delivery of information service processes A conceptual model is generated as below:

UIS = f (GAP1, GAP 2, GAP3 and Organizational factors)

The magnitude and direction of each gap will have an impact on UIS The functional form of relationship needs to be investigated to determine whether it is additive or multiplicative

2.3.1.2.2 Shirani’s Conceptual UIS Model

Shirani argued that the IS literature has paid too much attention to the delivered products, but too little attention to the organizational context and users’ individual characteristics

Therefore Shirani et al (1994) suggested a conceptual UIS model in which user

information satisfaction/dissatisfaction with the system is a function of a user-perceived gap between users’ expectations and actual outcomes of an implemented system This gap model is consistent with the gap model in marketing literature derived as the theoretical

background of the SERVQUAL measurement instrument by Parasuraman et al (1988) In

Shirani’s model, user expectations are affected by both users’ characteristics and organizational characteristics Users’ characteristics include users’ expertise in computer-

based technology and in the functional area for which they expects system support Organizational characteristics include the environment of organization, such as organizational structure, culture, and politics, etc Compared with Kim’s model, Shirani’s model does not take the feedback loop into account

2.3.1.3 UIS Measurement Instrument

To measure the perceived UIS, Bailey & Pearson (1983) made the first step by developing

a 39-item instrument, followed by a variety of UIS measures Table 2.3 depicts those

studies and components of their instruments

Table 2.3 UIS Studies and Dimensions of Their Instruments

Studies Dimensions

Ives et al (1983)

Baroudi et al (1986)

1 The quality of information products being supplied

2 The quality of systems personnel and service

3 The knowledge and involvement of systems personnel in the business

Doll & Torkzadeh

(1988, 1991)

1 Content of systems

2 Accuracy of systems

3 Format of reports

4 Ease of use of systems

5 The timeliness of systems Rivard & Hull

(1988)

1 Degree of user independence from systems department

2 Satisfaction with the set-up of the information center

3 User friendliness

4 User attitude

5 Satisfaction with the degree of support received from the information center

Magal (1991) 1 Quality of the information center service

2 Quality of user-developed applications

3 Degree of user self-sufficiency

UIS has received numerous criticisms, both theoretic and empirical The most critical

problem is the questionable assumption that user information satisfaction is a good

surrogate for system success Satisfied users alone may not be an indicative of a successful

system, or of business success (Melone, 1986) There is no theoretical basis that satisfied

users perform better than dissatisfied users Similarly, if information system helps user

perform better, the system is successful From the empirical perspective, numerous studies

have resulted in the widespread confusion, such as, what set of measure scales to use, and