To be more specific, an IS-approach framework included over 75% of the technology- oriented factors to evaluate websites, such as usability, accessibil- ity, navigability, or information [r]
Trang 1A strategic framework for website evaluation based on a review of the
literature from 1995–2006
Wen-Chih Chioua, Chin-Chao Linb,*, Chyuan Perngc
b
Department of Marketing and Distribution Management, Hsiuping Institute of Technology, 11, Gongye Rd., Dali City, Taichung County 41280, Taiwan, ROC
c
Department of Industrial Engineering and Enterprise Information, Tunghai University, Taiwan, ROC
1 Introduction
A website offers a business not only a platform to promote
products or services but also another avenue to generate revenue
by attracting more customers Unfortunately, not all websites
successfully turn visitors into customers The effective evaluation
of websites has therefore become a point of concern for
practitioners and researchers During recent years, different
website evaluation approaches have been introduced These deal,
for example, with website usability and design [1,11,47,65] ,
content [4,16,24,73] , quality [6,18,42,54,99] , user acceptance
[17,44,71,79] , and user satisfaction [22,58,61,86] being the most
common outcomes measured to evaluate websites From a
tactical viewpoint, these approaches were good by assessing
user attitude towards the website and could be considered as an
external user’s view From a strategic viewpoint, however, little
attention was given to evaluating the consistency between web
strategy and web presence, which can be considered as an internal
evaluation, from the company’s view point A strategic website
evaluation should first be conducted and then an external
evaluation (user-based survey) can follow to validate user acceptance of the website.
A sound web strategy acts as a guideline for developing websites in the virtual marketplace [20] Thus, using it in an evaluation framework helps the website manager measure how well the website satisfies the firm’s goals and objectives For example, the goal of ‘‘Google.com’’ is to have users leave its website
as quickly as possible [31] , i.e., the objectives of a clear and simple interface and instant page load This leads to an intriguing point that while many studies [60,72,74,95] consider the multimedia capability (e.g., graphics, video, or animation) of a website as important for a quality website, ‘‘Google.com’’ did not recognize this as crucial, because its goals and objectives were different An organization needs to be able to measure how successful their web strategies are with respect to its goals Choosing the ‘‘right’’ metric depends on what must be measured [70]
Our objectives therefore were to:
1 Investigate the trend of website evaluation through a compre-hensive review of literature published between 1995 and 2006;
2 discuss the evaluation frameworks and criteria proposed in the literature and develop a criteria pool;
3 propose a strategic conceptual framework for website evalua-tion and introduce a five-stage process for showing how to evaluate a website using this framework.
A R T I C L E I N F O
Article history:
Received 18 May 2007
Received in revised form 24 February 2010
Accepted 23 May 2010
Available online 4 June 2010
Keywords:
Website evaluation
Strategic evaluation framework
Website strategy
Strategy consistency
Evaluation factors and criteria
A B S T R A C T
Many studies have proposed new website evaluation frameworks and criteria We have attempted to understand and improve website evaluation through the analysis of 83 articles by classifying them into
IS, marketing, and combined-approaches Our findings showed that most early studies adopted the IS-approach but that later ones (after the burst of the dot-com bubble) shifted to a combined-IS-approach Our study also revealed that most papers analyzed the evaluation factors via a ranking list
Our review showed that most studies conducted user-based surveys to examine a website, but that very few addressed strategic issues of website evaluation We therefore proposed a strategic framework
as an internal evaluation to ensure consistency between web strategy and actual website presence The framework involved analysis of web strategy and a hybrid approach that included evaluation during three transaction phases; the framework was designed to be applied by a specific website vis-a`-vis its goals and objectives through a five-stage evaluation process
ß2010 Elsevier B.V All rights reserved
* Corresponding author Tel.: +886 4 24961100; fax: +886 4 24961187
Contents lists available at ScienceDirect
Information & Management
j o u r n a l h o m e p a g e : w w w e l s e v i e r c o m / l o c a t e / i m
0378-7206/$ – see front matter ß 2010 Elsevier B.V All rights reserved
Trang 22 Literature review
2.1 Methodology
To find the articles on website evaluation, we adopted a
structured approach following the suggestions of Webster and
Watson [93] This involved four steps:
1 Search a particular keyword in leading journal databases.
The databases used in our study included EBSCOhost, Emerald,
ProQuest, and ScienceDirect OnSite (SDOS) The candidate articles
were identified by searching titles with the keyword ‘‘website’’ and
abstract with additional keywords ‘‘evaluation,’’ ‘‘assessment,’’
‘‘measurement,’’ ‘‘framework,’’ and ‘‘model.’’ We chose 1995 as the
starting date when we found that the first few academic articles
were published that year [14,63]
2 Select the articles from important journals using the ‘‘Web of
Science’’ (the electronic version of the Social Sciences and
Science Citation Indices).
Those which were published in the list of the Web of Science,
were considered to be possible articles.
3 Scan these articles by reading their titles and abstracts to select
those relevant to website evaluation.
When an article’s title or abstract seemed relevant for
evaluation, the full text of the article was reviewed through a
consensus process to ensure that an evaluation framework or
model was provided, and the content was relevant to website
evaluation The consensus process was conducted by the three
authors of this study through iterative group discussion First, they
set the rules to be followed for classification Second, they read the
articles to decide whether each was relevant to the study or not.
Third, if there was a discrepancy between their opinions,
discussions were held until a consensus or agreement was
obtained After this, the qualified articles were retained in the
article list but irrelevant articles were eliminated.
4 Determine the number of citations of each of the remaining articles.
The number of citations was searched in February 2008 using SCOPUS (the electronic citation database of research literature) To improve the quality of the review list, only articles with at least one citation were included.
The database search identified 1668 candidate articles from 335 journals After removing articles that were not published in the
‘‘Web of Science’’, the number of prior articles was reduced to 882, and the number of journals to 175 Then in step 3 the number of articles relevant to our subject dropped to 114, and the number of journals dropped to 25 After the final step, we had 83 articles from
23 journals in our review list.
Table 1 shows the 23 journals and their articles, identified by the number of the paper in our set of references In particular, Information & Management published the most articles (19%) in our list for a total of 16 articles, followed by Internet Research (12%) with 10 of the articles Among the set articles, various authors used different terms to evaluate websites, such as criteria, factors, attributes, metrics, features, characteristics, and measures From our review of the literature, we found the term ‘‘factors’’ to be most commonly used; possibly because the authors adopted factor analysis to identify different groups of evaluation items or variables Thus, we define factor as the term be used as the set
of relevant criteria For instance, ‘‘Ease of use’’ can be a collection of criteria such as user-friendly interface, easy access to the site, or ease of navigation In addition, we defined criterion to be a specific item or variable, such as loading speed, search capability, shopping cart, up-to-date information, or FAQ.
The selected articles were classified into three sets: IS, marketing, or a combination of the two To be more specific, an IS-approach framework included over 75% of the technology-oriented factors to evaluate websites, such as usability, accessibil-ity, navigabilaccessibil-ity, or information quality In contrast, the marketing factor included over 75% of the marketing related factors, such as advertising, promotion, online transaction, order confirmation, or customer service The combined framework was a mixture of using
IS and marketing factors Under these rules of classification, a consensus process was conducted by the authors to determine the
Table 1
The website evaluation papers from selected journals
a
Trang 3study’s approach As a result, 41% of the selected articles were
attributed to IS, 12% were marketing, and 47% had a
combined-approach.
2.2 General trend of the reviewed literature
To analyze the pattern of study during the different time
periods, we used year 2001 as a splitting point to compare ratios
before and after the burst of the dot-com bubble (year 2000),
because most articles took about one year to be published As
illustrated in Table 2 , the data showed that before the burst, 59% of
studies adopted the IS-approach, 14% o adopted the
marketing-approach, while the remaining studies (27%) adopted the
combined-approach when assessing a website Using the
IS-approach to assess websites was widely accepted by most studies
at that time However, after the burst, using the IS-approach
dropped to 34%, and the marketing-approach dropped to 11%,
while the combined-approach increased to 55% (see Fig 1 ) The
pattern is consistent with the essence of e-Commerce wherein a
website should be regarded as a technology-oriented distribution
channel and media to accomplish an organization’s marketing
strategies In addition to the shift in the dominant approach, the
number of our selected studies during the 1995–2001 time-frame
numbered 22, but this rapidly increased to 61 during 2002–2006.
Faced with the explosion of evaluation frameworks, managers
and researchers may have difficulty grasping what alternatives are
Fig 1 The pattern of study approaches
Table 2
The number of different approach studies between 1995 and 2006
No of studies
studies
studies
Table 3
Website evaluation studies
Section I IS-approach studies
Table 3 (Continued )
Section II Marketing-approach studies
Section III Combined-approach studies
Trang 4available and to determine their appropriate framework Table 3
presents the IS, marketing, and combined-approach studies
included in our review As section I reveals, the most frequently
cited among the IS-approach studies is McKinney et al., which
developed theoretical constructs for measuring web-customer
satisfaction of information quality and system quality In section II
the most frequently cited is Ho [37] occurring 39 times; it
introduced a process-value matrix framework to evaluate the
customer value creations throughout the marketing processes of
e-Commerce websites In section III, the most frequently cited study
was that of Szymanski and Hise occurring 139 times; this
examined the factors that make consumers satisfied with their
e-retailing experiences.
The research methodologies were based on the data collection
method The frequency of using these methodologies in the three
approaches is shown in Table 4 It indicated that the survey ranked
as the most frequently used in all studies In most, surveys were
conducted through e-mail or online questionnaires Most of the
respondents were web users (either students or online shoppers)
except for one questionnaire proposed by Liu and Arnett, used
experts (webmasters) as its population of respondents.
Experimental evaluation was the most used methodology in IS
studies (32%) and the second most used in overall studies (23%).
More specifically, 11 out of 19 total experimental studies used the
IS-approach An experimental evaluation was generally conducted
under controlled settings The participant was asked to accomplish
a specific task by following a detailed set of instructions Data were
collected by interaction logs and protocol analysis, which has been
recognized as the most efficient way to identify the usability
problem [35] when the interface is being developed or the web is
being designed.
Content analysis is another major research technique used in
understanding the design and functions of websites [59] In our
study, 17% used the IS-approach to evaluate the websites of related
industries It is worthwhile to mention that only automatic
evaluation was adopted in IS-approach studies It generally uses a
software tool to capture the characteristics of web information
system [10] through a set of operational criteria or metrics for
classifying the website or measuring its accessibility.
2.3 Review of evaluation frameworks from the three approaches
To increase the applicability of evaluation frameworks, we condensed each study’s dimensional factors into the 12 unified factors suggested by Park and Gretzel [67] : ease of use, information quality, responsiveness, visual appearance, security/privacy, in-teractivity, trust, fulfillment, personalization, advertising/persua-sion, playfulness, and technology integration Using unified factors
process for assigning each study’s proposed criteria into one of the
12 unified factors We analyzed the evaluation factors of the different studies; Table 6 presents the percentages of the studies using each factor.
The IS-approach studies adopted technical factors, such as ease
of use, security/privacy, visual appearance, and information quality, as the major evaluation criterion Nearly every IS study (97%) included ease of use as a factor; the most frequently used criteria were navigation, logical structure, user-friendly interface, loading speed, well linkage, searching mechanism, ease of access, and ease in finding targeted information Information quality was the second most used factor in IS studies (82%) Its criteria included relevancy, usefulness, comprehensive coverage, currency, read-ability, and accuracy Visual appearance also often appeared in IS-approach studies (53%), with factor mostly relating to interface design criteria (aesthetics, consistent style, and proper multimedia presentation).
In the early days of e-Commerce, IS was considered to be the driving force behind achieving a successful commercial web However, its role should not be over-emphasized and should never today be the starting point of online activities As Ivory and Hearst
[40] pointed out, ‘‘There is a wide gap between a heuristic and the implementation of this advice.’’ Each website has its own goals The evaluation frameworks and factors proposed by most studies will not match all website strategies.
Table 5
Description of unified factors
Table 6 Ranking and frequencies of unified factors used in three approaches
Table 4
Frequency of using research methodologies in the three approaches
Trang 5The marketing-approach framework evaluates a website, that is
generally ignored by most technical webmasters [92] : the typical
web visitor is not merely a web user but also a potential customer
who needs product information, attractive promotional activities,
convenient order process, and after-sales services As e-Commerce
involves a series of complex interactions and Internet marketing
strategies, companies have learned that successful e-Commerce
should not only provide technical aspects.
Fulfillment, which acts as a transaction-oriented factor, was
used by over 90% of the marketing studies Most IS studies did not
consider it as a crucial dimension The most frequently used
criteria for fulfillment were online transaction and order status
tracking Information quality was another factor mostly used in
marketing studies (90%) This factor emphasized more
commer-cial-oriented criteria, such as product and price details
Advertis-ing/persuasion was also a commonly used factor in marketing
studies (80%) but not in IS or combined studies Its criteria were
mainly promotional campaigns (i.e., discount or free trial), and
company/brand recognition.
e-Commerce systems can be regarded as a sequence of market
transactional Marketing-approach studies have adopted different
transaction processes as part of the framework for evaluating
e-Commerce activities and functionalities For example, Ho classified
the business purposes of a commercial website into three
categories: promotion, provision, and processing These were also
used by Doolin et al [23] in their extended model of Internet
commerce adoption Liang and Lia [50] proposed a model that
divides consumer decision processes into five stages: problem
recognition, search for information, evaluation of alternatives,
choice, and outcome evaluation Schubert [77] claimed that the
transaction process included information, agreement, settlement,
and after-sales phases Wan [91] modified Rowley’s [75] five
components of online transaction to focus on four processes:
promotion, pricing, transaction, and services In general, these
models are somewhat different, but they share a degree of
similarity Ho and Wan used a process-value matrix framework in
their evaluation of websites; it was designed to assess customer
value additions in the transaction process Although customer
values and processes may be somewhat different in the studies, a
process-value framework shows areas where customers may be
concerned with all business processes and thus help managers
improve their websites to provide value added services and
products.
Combined-approach studies emphasized the degree of
impor-tance of both IS and marketing factors in the framework The top
three ranking factors were: ease of use (97%), information quality
(92%), and responsiveness (87%) Information quality is defined as
either an IS or a marketing factor because it involves both accuracy,
currency, or variety, etc and price details and product description,
etc Responsiveness mostly relates to customer service support.
Interestingly, the unified factor’s ranking of total studies is the
same as that in the combined studies.
Szymanki and Hise’s study of e-satisfaction is a good example of
using a combined-approach to examine the factors that make
consumers satisfied with their e-retailing experiences Several IS
criteria (e.g., ease of browsing, easy-to-navigate, etc.) were placed
as factors of site design, convenience, and financial security On the
other hand, the marketing criteria (procure offerings, variety of
product offerings, quality of product information, etc.) were
included under convenience and merchandising Their study
results showed that convenience, site design, and financial security
were the dominant factors in consumer assessments of
e-satisfaction.
In the combined-approach framework, simply joining IS and
marketing criteria will result in confusion because some criteria
can be both IS and marketing oriented For example, complete
product description was attributed to information quality for IS by Liu and Arnett but to marketing by Dutta and Biren [25] Another example is customized search engine, which increases the customers’ ability to find information on the web It was attributed
to the learning capability as a marketing factor, but it is strongly supported by technology, and thus it could also be considered to be
an IS dimension.
After reviewing the frameworks, we identified three issues that need to be further addressed:
1 The review of marketing-approach studies corroborates the importance of transaction processes For this reason, an evaluation framework needs to be process oriented to identify crucial activities in each transactional phase.
2 A hybrid approach that considers the role of IS as a support factor for marketing rather than a combined-approach suggests that IS factors should be embedded into marketing factors as facilitating e-Commerce By doing this, the confusion in classifying criteria would no longer exist.
3 The existing studies proposed a variety of frameworks with extensive factors and criteria Unfortunately, none of them addressed the issue of the relationship between web strategy and evaluation factors A framework with strategy consideration would ensure that the website presence was consistent with its predefined goals and objectives.
3 A conceptual framework The Web-Marketing Mix (WMM) model can be used to identify the critical elements for designing and developing online commerce while addressing the strategic, operational, organiza-tional and technical issues In practice, each business has its own strategies, which should be considered when developing an evaluation framework.
We therefore collected representative criteria from our studies
as the initial pool and analyzed it to eliminate repetitive items, merge similar items, and condense sub-attributes to higher level criteria After a consensus process by the three authors, 53 criteria remained in the criteria pool These criteria were then categorized into five marketing oriented factors: (1) Product, (2) Promotion, (3) Price, (4) Place, and (5) Customer Relationship (4PsC), which were slightly changed from the Marketspace model of Dutta and Biren.
An overview of the 4PsC factors with a description of each is given
in Table 7 We considered them as a good operational tool for appraising how well the marketing mix concepts converted into web presence rather than as a strategic tool The assessment result
of 4PsC provides an opportunity for managers to examine the strength and weakness of each factor from a marketing perspec-tive Table 8 shows the frequency of the 53 criteria in the studies
we reviewed and classified; criterion needing IT support were shown with a superscript ‘‘T’’ The criteria were to be used as a selection pool for choice by website managers to address their strategic objectives and its frequency.
We considered the three-phase transaction (information, agree-ment, and settlement) as the basis for extending our framework The Table 7
Description of 4PsC evaluation factors
purchase
buyer–seller relationship
Trang 6information phase starts when the prospective buyers enters the
e-Commerce system and lasts until they decide to place an order or
leave In this phase, prospective buyers identify their needs, search
for specific product, ask for price and condition, and evaluate
alternative sources Thus all five marketing factors of the 4PsC will
have relative impact on the customers’ decision to move to the next
phase The agreement phase involves negotiations between
prospective buyers and sellers, finalized by a contract [41]
Important criteria include ease of online transaction, interactive
communications, and online assistance and help Eventually, the
contract is executed in the settlement phase according to the
stipulated conditions; delivery of the product and after-sales
interaction takes place in this phase Important criteria may include
convenient payment methods, order status inquiry and tracking, and
ease of returning a product The assessment of identified criteria
about the strategy provides diagnostic information and managerial implications for improving a website’s internal consistency.
We therefore proposed a strategic evaluation framework, as depicted in Fig 2 At the start, the framework will be constructed using web strategies Following this, a hybrid approach will have a transactional process orientation with information, agreement, and settlement phases The criteria of each phase will be selected from our criteria pool, depending on the goals and objectives of the marketing factors of 4PsC in the three phases Thus IS factors are embedded into each marketing factor throughout the entire process.
To implement this strategic evaluation framework, a five-stage website evaluation process was defined (see Fig 3 ).
In Stage one, users identify the website strategy (with goal, objectives, and actions) through an in-depth interview with senior
Table 8
The criteria pool for website evaluation
13
10
4
37
28
25
23
19
17
7
T
The criterion is supported by information technology
Trang 7managers Website evaluation criteria are then selected from the
proposed criteria pool A hierarchical structure is constructed to
delineate the relationship between goal, objectives, actions, and
criteria Each criterion weight is then determined by the website
manager, as he/she knows the priority of these criteria.
Then in Stage two, they develop a web-based evaluation
instrument with two main sections:
Section I briefly introduces the summary of the interview with
the website manager as background knowledge and reference for
evaluators.
Section II lists the questionnaires created based on the selected
criteria.
In Stage three, they conduct a website evaluation with the aid of
experts, using fuzzy linguistic terms to express their agreement or
disagreement with the statement of each questionnaire.
In Stage four, they compute the criteria weights and scores in
three steps.
Step 1 transforms the linguistic terms into numbers.
Step 2 normalizes the criteria weights to compare the relative importance of each criterion.
Step 3 calculates the weighted scores of the objectives and criteria.
In Stage five, they conduct data analysis and discuss it from three perspectives.
First, each criterion’s importance weight and presence score are listed to compare their strengths and weaknesses In general, a criterion with a low score should be considered by managers to improve it However, this may mislead managers to allocate more resources to a low score criterion that is less important To generate a better assessment result, the gaps calculated by subtracting the average score from the weight can be used to decide which action should be taken The issue of how well the website satisfies its strategic objectives is addressed through the weighted scores of the objectives.
Second, a criteria performance matrix is constructed to provide managers with the performance information and to rank the priorities of improvement plan The result of the criterion presence score is plotted against the criterion weight The matrix has nine cells if the axes are equally divided into three sections involving the three regions between four scales [0,1/3, 2/3, 1].
Third, the selected criteria are classified into the 4PsC dimensions
by following the criteria pool classification Afterwards, each dimensional criteria weight and score is added to obtain the average weight and score of the 4PsC dimensions A radar chart can then be constructed for identifying the strengths and weaknesses of the 4PsC dimensions By comparing each dimension’s weight and score, we can determine any dimension which is important but has poor presence A drill-down analysis can be used to help managers further discover the criteria that have relatively low scores.
4 Conclusion and suggestion Our review of related research has shown that website evaluation studies can be made from an IS, marketing, or a combination perspectives We identified 83 studies from 23 journals from 1995 and 2006 From our analysis of these studies,
we found a pattern showing that an IS-approach was used by the majority of web evaluation studies) before 2001 Since then, the combined-approach emerged as dominant.
To analyze the evaluation factors used by most studies, we ranked the unified factors of the different study approaches Generally, ease of use and information quality was common in most studies.
Based on our review, there was very limited research performed
to explore the web strategic issue in website evaluation, and none included strategy as part of its framework.
To explain how to evaluate a website using our new framework,
a five-stage evaluation process was introduced Meanwhile, the collected data were analyzed from three perspectives to determine the improvement priority and to show how well the website satisfies its strategic objectives Our framework was designed to focus on how a specific website applies its goals and objectives The framework could help website managers to understand causal links that show ‘‘how’’ and ‘‘where’’ a website is consistent with its strategy.
A limitation of our framework is that the 4PsC factors are pertinent for commercial websites but not for non-commercial ones, since ‘‘Price’’ is not then relevant.
To fulfill a strategic evaluation, we recommend that domain experts have a better understanding of the target website’s strategies and evaluate the site according to these.
Fig 3 The five-stage website evaluation process
W.-C Chiou et al / Information & Management 47 (2010) 282–290 288
Trang 8[1] R Agarwal, V Venkatesh, Assessing a firm’s Web presence: a heuristic evaluation
procedure for the measurement of usability, Information Systems Research 13
(2), 2002, pp 168–186
[2] J.S Ahuja, J Webster, Perceived disorientation: an examination of a new measure
to assess web design effectiveness, Interacting with Computers 14 (1), 2001, pp
15–29
[3] A.M Aladwani, P.C Palvia, Developing and validating an instrument for
mea-suring user-perceived web quality, Information and Management 39 (6), 2002,
pp 467–476
[4] S Baloglu, Y.A Pekcan, The website design and Internet site marketing practices
of upscale and luxury hotels in Turkey, Tourism Management 27, 2006, pp 171–
176
[5] S.J Barnes, R Vidgen, An evaluation of cyber-bookshops: the WebQual method,
International Journal of Electronic Commerce 6 (1), 2001, pp 11–30
[6] S.J Barnes, R Vidgen, Measuring website quality improvements: a case study of
the forum on strategic management knowledge exchange, Industrial
Manage-ment and Data Systems 103 (5), 2003, pp 297–309
[7] S.J Barnes, R Vidgen, Data triangulation and web quality metrics: a case study in
e-government, Information and Management 43, 2006, pp 767–777
[8] C Bauer, A Scharl, Quantitative evaluation of website content and structure,
Internet Research: Electronic Networking Applications and Policy 10 (1), 2000,
pp 31–43
[9] H.H Bauer, T Falk, M Hammerschmidt, eTransQual: a transaction process-based
approach for capturing service quality in online shopping, Journal of Business
Research 59 (7), 2006, pp 866–875
[10] H Bell, N.K.H Tang, The effectiveness of commercial Internet websites: a users
perspective, Internet Research: Electronic Networking Applications and Policy 8
(3), 1998, pp 219–228
[11] R Benbunan-Fich, Using protocol analysis to evaluate the usability of a
com-mercial website, Information and Management 39, 2001, pp 151–163
[12] G Bhatt, Bringing virtual reality for commercial Web sites, International Journal
of Human-Computer Studies 60 (1), 2004, pp 1–15
[13] B.F Blake, K.A Neuendorf, C.M Valdiserri, Tailoring new websites to appeal
to those most likely to shop online, Technovation 25 (10), 2005, pp 1205–
1214
[14] E Boling, Usability testing for websites, Proceedings of the 7th Annual
CLES/usability/usability.main.html
[15] G Chakraborty, V Lala, D Warren, What do customers consider important
in B2B websites? Journal of Advertising Research 43 (1), 2003, pp 50–
61
[16] W.M Cheung, W Huang, An investigation of commercial usage of the World
Wide Web: a picture from Singapore, International Journal of Information
Management 22 (5), 2002, pp 377–388
[17] J Chung, F.B Tan, Antecedents of perceived playfulness: an exploratory study on
user acceptance of general information-searching websites, Information &
Man-agement 41 (7), 2004, pp 869–881
[18] M Cao, Q Zhang, J Seydel, B2C e-commerce website quality: an empirical
examination, Industrial Management and Data Systems 105 (5), 2005, pp
645–661
[19] N Cho, S Park, Development of electronic commerce user-consumer satisfaction
index (ECUSI) for Internet shopping, Industrial Management and Data Systems
101 (8), 2001, pp 400–405
[20] E Constantinides, The 4S Web-marketing mix model, Electronic Commerce
Research and Applications 2002, pp 57–76
[21] P Davoli, F Mazzoni, E Corradini, Quality assessment of cultural web sites with
fuzzy operators, Journal of Computer Information Systems 46 (1), 2005, pp 44–
57
[22] S Devaraj, F Man, R Kohli, Antecedents of B2C channel satisfaction and
preference: validating e-Commerce metrics, Information Systems Research 13
(3), 2002, pp 316–333
[23] B Doolin, L Burgess, J Cooper, Evaluating the use of the web for tourism
marketing: a case study from New Zealand, Tourism Management 23, 2002,
pp 557–561
[24] X Dre`ze, F Zufryden, Testing Web site design and promotional content, Journal
of Advertising Research 37 (2), 1997, pp 77–91
[25] S Dutta, B Biren, Business transformation on the Internet: results from the 2000
study, European Management Journal 19 (5), 2001, pp 449–462
[26] A.E Ellinger, D.F Lynch, J.D Hansen, Firm size, web site content, and financial
performance in the transportation industry, Industrial Marketing Management
32 (3), 2003, pp 177–185
[27] J E´thier, P Hadaya, J Talbot, J Cadieux, B2C web site quality and emotions
during online shopping episodes: an empirical study, Information and
Manage-ment 43 (5), 2006, pp 627–639
[28] J.R Evans, V.E King, Business-to-Business marketing and the World Wide Web:
planning, managing, and assessing Web sites, Industrial Marketing Management
28 (4), 1999, pp 343–358
[29] H Evanschitzky, G.R Iyer, J Hesse, D Ahlert, E-satisfaction: a re-examination,
Journal of Retailing 80 (3), 2004, pp 239–247
[30] F.J.M Gonza´lez, T.M.B Palacios, Quantitative evaluation of commercial
web-sites: an empirical study of Spanish firms, International Journal of Information
Management 24, 2004, pp 313–328
[31] Google.com, Ten things Google has found to be true, January 8, 2008 Available
[32] S Hackett, B Parmanto, A longitudinal evaluation of accessibility: higher edu-cation web sites, Internet Research: Electronic Networking Appliedu-cations and Policy 15 (3), 2005, pp 281–294
[33] S Hackett, B Parmanto, X Zeng, A retrospective look at website accessibility over time, Behaviour & Information Technology 24 (6), 2005, pp 407–417 [34] L.C Harris, M.M.H Goode, The four levels of loyalty and the pivotal role of trust: a study of online service dynamics, Journal of Retailing 80 (2), 2004, pp 139–158 [35] R.D Henderson, M.C Smith, J Podd, H Varela-Alvarez, A comparison of four prominent user-based methods for evaluating the usability of computer soft-ware, Ergonomics 38 (10), 1995, pp 2030–2040
[36] E Herrera-Viedma, G Pasi, A.G Lopez-Herrera, C Porcel, Evaluating the infor-mation quality of Web sites: a methodology based on fuzzy computing with words, Journal of the American Society for Information Science & Technology 57 (4), 2006, pp 538–549
[37] J Ho, Evaluating the World Wide Web: a global study of commercial sites,
jcmc.indiana.edu/vol3/issue1/ho.html [38] S Hong, J Kim, Architectural criteria for website evaluation—conceptual frame-work and empirical validation, Behaviour & Information Technology 23 (5),
2004, pp 337–357
[39] E.K.R.E Huizingh, The content and design of websites: an empirical study, Information and Management 37, 2000, pp 123–134
[40] M.Y Ivory, M.A Hearst, Improving website design, IEEE Internet Computing 6,
2002, pp 56–63
[41] J Kim, J Lee, Critical design factors for successful e-commerce systems, Behav-iour and Information Technology 21 (3), 2002, pp 185–199
[42] S Kim, L Stoel, Dimensional hierarchy of retail website quality, Information and Management 41 (5), 2004, pp 619–633
[43] S.E Kim, T Shaw, H Schneider, Website design benchmarking within industry groups, Internet Research: Electronic Networking Applications and Policy 13 (1),
2003, pp 17–26
[44] M Koufaris, Applying the technology acceptance model and flow theory to online consumer behavior, Information Systems Research 13 (2), 2002, pp 205– 223
[45] Y.-F Kuo, A study on service quality of virtual community websites, Total Quality Management & Business Excellence 14 (4), 2003, pp 461–474
[46] H.M Kuo, S.L Hwang, E.M.Y Wang, Evaluation research of information and supporting interface in electronic commerce websites, Industrial Management and Data Systems 104 (9), 2004, pp 712–721
[47] T Lavie, N Tractinsky, Assessing dimensions of perceived visual aesthetics of web sites, International Journal of Human-Computer Studies 60 (3), 2004, pp 269–298
[48] K.C Lee, N Chung, Cognitive map-based web site design: empirical analysIS-approach, Online Information Review 30 (2), 2006, pp 139–154
[49] Y.N Li, K.C Tan, M Xie, Measuring web-based service quality, Total Quality Management 13 (5), 2002, pp 685–700
[50] T.-P Liang, H.J Lai, Effect of store design on consumer purchases: van empirical study of on-line bookstores, Information and Management 39, 2002, pp 431– 444
[51] N Lightner, Evaluating E-Commerce functionality with a focus on customer service, Communications of the ACM 47 (10), 2004, pp 88–92
[52] J.C.-C Lin, H Lu, Towards an understanding of the behavioural intention to use a web site, International Journal of Information Management 20, 2000, pp 197– 208
[53] C Liu, K.P Arnett, L.M Capella, R.D Taylor, Key dimensions of web design quality
as related to consumer response, Journal of Computer Information Systems 42 (1), 2001, pp 70–77
[54] C Liu, K.P Arnett, C Litecky, Exploring the factors associated with website success in the context of electronic commerce, Information and Management 38,
2000, pp 23–33
[55] Y Liu, Developing a scale to measure the interactivity of websites, Journal of Advertising Research 43 (2), 2003, pp 207–216
[56] M.D Marsico, S Levialdi, Evaluating web sites: exploiting user’s expectations, International Journal of Human-Computer Studies 60 (3), 2004, pp 381– 416
[57] M.B Mateos, A.C Mera, F.J.M Gonzalez, O.R.G Lopez, A new Web assessment index: Spanish universities analysis, Internet Research: Electronic Networking Applications and Policy 11 (3), 2001, pp 226–234
[58] V McKinney, K Yoon, F Zahedi, The measurement of web-customer satisfaction:
an expectation and disconfirmation approach, Information Systems Research 13 (3), 2002, pp 296–315
[59] S.J McMillan, The microscope and the moving target: the challenge of applying content analysis to the World Wide Web, Journalism & Mass Communication Quarterly 77 (1), 2000, pp 80–98
[60] M.M Misic, K.L Johnson, Benchmarking: a tool for website evaluation and improvement, Internet Research: Electronic Networking Applications and Policy
9 (5), 1999, pp 383–392
[61] S Muylle, R Moenaert, M Despontin, The conceptualization and empirical validation of website user satisfaction, Information and Management 41 (5),
2004, pp 543–560
[62] S Okazaki, J.A Rivas, A content analysis of multinationals’ Web communication strategies: cross-cultural research framework and pre-testing, Internet Research
12 (5), 2002, pp 380–390
[63] L Olsina, D Godoy, G Lafuente, G Rossi, Assessing the quality of academic websites: a case study, New Review of Hypermedia and Multimedia 5 (1), 1995,
pp 81–103
Trang 9[64] A.A Ozok, G Salvendy, How consistent is your web design? Behaviour &
Information Technology 20 (6), 2001, pp 433–447
[65] J.W Palmer, Website usability, design, and performance metrics, Information
Systems Research 13 (2), 2002, pp 151–167
[66] H.-S Park, S.J Noh, Enhancement of web design quality through the QFD
approach, Total Quality Management 13 (3), 2002, pp 393–401
[67] Y.A Park, U Gretzel, Success factors for destination marketing Web sites: a
qualitative meta-analysis, Journal of Travel Research 46, 2007, pp 46–63
[68] B Parmanto, X Zeng, Metric for Web accessibility evaluation, Journal of the
American Society for Information Science & Technology 56 (13), 2005, pp 1394–
1404
[69] M Petre, S Minocha, D Roberts, Usability beyond the website: an
empirically-grounded e-commerce evaluation instrument for the total customer experience,
Behaviour & Information Technology 25 (2), 2006, pp 189–203
[70] A Phippen, L Sheppard, S Furnell, A practical evaluation of Web analytics,
Internet Research: Electronic Networking Applications and Policy 14 (4), 2004,
pp 284–293
[71] T Pikkarainen, K Pikkarainen, H Karjaluoto, S Pahnila, Consumer acceptance of
online banking: an extension of the technology acceptance model, Internet
Research: Electronic Networking Applications and Policy 14 (3), 2004, pp
224–235
[72] C Ranganathan, S Ganapathy, Key dimensions of business-to-consumer
web-sites, Information and Management 39, 2002, pp 457–465
[73] S.S Robbins, A.C Stylianou, Global corporate websites: an empirical
investiga-tion of content and design, Informainvestiga-tion and Management 40 (3), 2003, pp 205–
212
[74] D Rosena, E Purintonb, Website design: viewing the web as a cognitive
landscape, Journal of Business Research 57, 2004, pp 787–794
[75] J Rowley, Retailing and shopping on the Internet, Internet Research: Electronic
Networking Applications and Policy 6 (1), 1996, pp 81–91
[76] M.C Roy, O Dewit, B.A Aubert, The impact of interface usability on trust in Web
retailers, Internet Research: Electronic Networking Applications and Policy 11
(5), 2001, pp 388–398
[77] P Schubert, Extended Web Assessment Method (EWAM) evaluation of
electron-ic commerce applelectron-ications from the customer’s view point, International Journal
of Electronic Commerce 7 (2), 2002, pp 51–80
[78] C Shchiglik, S.J Barnes, Evaluating website quality in the airline industry, Journal
of Computer Information Systems 44 (3), 2004, pp p17–25
[79] H.P Shih, An empirical study on predicting user acceptance of e-shopping on the
Web, Information and Management 41 (3), 2004, pp 351–368
[80] N Singh, D.W Baack, Web site adaptation: a cross-cultural comparison of U.S
and Mexican Web sites, Journal of Computer-Mediated Communication, 9(4),
[81] N Singh, H Matsuo, Measuring cultural adaptation on the Web: a content
analytic study of U.S and Japanese Web sites, Journal of Business Research
57 (8), 2004, pp 864–872
[82] A.G Smith, Applying evaluation criteria to New Zealand government websites
Source, International Journal of Information Management 21 (2), 2001, pp 137–
149
[83] J.A Song, F.M Zahedi, Theoretical approach to Web design in e-Commerce: a
belief reinforcement model, Management Science 51 (8), 2005, pp 1219–1235
[84] S.S Srinivasan, R Anderson, K Ponnavolu, Customer loyalty in e-commerce: an
exploration of its antecedents and consequences, Journal of Retailing 78 (1),
2002, pp 41–50
[85] M Supphellen, H Nysveen, Drivers of intention to revisit the websites of
well-known companies, International Journal of Market Research 43 (3), 2001, pp
341–352
[86] D.M Szymanski, R.T Hise, E-satisfaction: an initial examination, Journal of
Retailing 76 (3), 2000, pp 309–322
[87] G.J Udo, G.P Marquis, Factors affecting e-Commerce website effectiveness,
Journal of Computer Information Systems 42 (2), 2002, pp 10–16
[88] R van der Merwe, F Bekker, A framework and methodology for evaluating
e-commerce websites, Internet Research: Electronic Networking Applications and
Policy 13 (5), 2003, pp 330–341
[89] J van Iwaarden, T Van der Wiele, L Ball, R Millen, Perceptions about the quality
of web sites: a survey amongst students at Northeastern University and Erasmus
University, Information and Management 41 (8), 2004, pp 947–959
[90] P van Schaik, J Ling, Five psychometric scales for online measurement of the quality of human–computer interaction in Web sites, International Journal of Human-Computer Interaction 18 (3), 2005, pp 309–322
[91] C.-S Wan, The websites of international tourist hotels and tour wholesalers in Taiwan, Tourism Management 23, 2002, pp p.155–160
[92] H.A Wan, Opportunities to enhance a commercial website, Information and Management 38, 2000, pp 15–21
[93] J Webster, R.T Watson, Analyzing the past to prepare for the future: writing a literature review, MIS Quarterly 26 (2), 2002, xiii-xxiii
[94] M Wolfinbarger, M.C Gilly, eTailQ: dimensionalizing, measuring and predicting etail quality, Journal of Retailing 79 (3), 2003, pp 183–198
[95] K.D Wulf, N Schillewaert, S Muylle, D Rangarajan, The role of pleasure in website success, Information and Management 43 (4), 2006, pp 434–446 [96] Z Yang, C Shaohan, Z Zhou, N Zhou, Development and validation of an instrument to measure user perceived service quality of information presenting Web portals, Information and Management 42 (4), 2005, pp 575–589 [97] W.L Yeung, M.-T Lu, Gaining competitive advantages through a functionality grid for website evaluation, Journal of Computer Information Systems 42 (2),
2004, pp p.67–77
[98] P Zhang, G.M von Dran, Satisfiers and dissatisfiers: a two-factor model for Website design and evaluation, Journal of the American Society for Information Science 51 (14), 2000, pp 1253–1268
[99] P Zhang, G.M von Dran, User expectations and rankings of quality factors in different Web site domains, International Journal of Electronic Commerce 6 (2),
2001, pp 9–33
[100] M Zviran, C Glezer, I Avni, User satisfaction from commercial websites: the effect of design and use, Information and Management 43, 2006, pp p.157–178
Wen-Chih Chiou is an associate professor in the Department of Business Administration at the National Chin-Yi University of Technology (Taiwan) He received his PhD degree in business management from National Sun Yat-Sen University His research interests include Internet marketing, consumer behavior, and website and mobile usability test
Chin-Chao Lin is an assistant professor at the Depart-ment of Marketing and Distribution, Hsiuping Institute
of Technology, Taiwan He received his PhD degree in industrial engineering and enterprise information from the Tunghai University His research interests include electronic commerce and information system
Chyuan Perng holds a PhD in industrial engineering from Texas Tech University and is an associate professor at the Department of Industrial Engineering and Enterprise Information, Tunghai University His main research areas are e-commerce, production scheduling, and information technology application