The
most experienced and successful E-commerce companies are beginning to
realize that key determinants of success or failure are not merely a
web presence or low price but delivering on a high quality website. To
attain the desired quality of website software, it is necessary to
produce a framework and model that enables evaluation of a website’s
quality. This paper defines and categorizes the quality factors for
measuring an E-commerce website. The software is an essential part of
the E-commerce website. The ISO 9126-1 standard for software
engineering product quality states that the main purpose of software
quality evaluation is to supply referential quantitative results for
software products that are reliable, understandable and acceptable.
According to Larsson (2004), quality factors are primarily attributes
of the software that are often labeled as “non-functional
requirements.” The key challenge of these attributes is the lack of a
formal specification of a means of measurement. This paper provides a
framework to establish quality factors in terms of attributes, in
addition to their level of importance based on the opinion of
highly-skilled professionals. The primary goals are identifying,
qualifying, categorizing and ranking these factors.
The
quality of a website is a property difficult to define and capture in
an operational way, yet everybody feels it when it is missing. In fact,
for a website there can be as many views of its quality as there are
usages. Quality may depend on task-related factors affecting end users
such as presentation quality and appeal, content and function adequacy,
and navigability. It may also depend on performance-related factors
that affect the efficiency of end users and the economics of the
website within the company running it. These factors include response
time, transaction throughput, reliability and robustness. It may depend
on development-related factors that affect developers and maintainers
of a website. These include code complexity, code readability, code
flexibility, portability, page coupling and modifiability (Darie and
Balanescu, 2008). The foundation model used to identifying quality
factors and attributes is based on research by Albuquerque and Belchior
(2002). The model is extended with further research investigation and
expert reviews and interviews.
Albuquerque and
Belchior have
organized a comprehensive set of software quality attributes into
objectives where each objective is composed of a set of quality
factors. Each quality factor is further decomposed into sub-factors.
According to Albuquerque and Belchior (2002), three broad objectives
formulate the model, as illustrated below which enables the evaluation
of an E-commerce website’s quality.
Fig 1:
E-commerce quality objectives
Usability
is a quality objective that refers to the characteristics that allow
the use of the E-commerce site in the most diverse situations, not only
during its development process, but also during its operation and
maintenance. This objective builds on the reliability of the web site.
Reliability is composed of two aspects, according to Albuquerque and
Belchior (2002).
Conceptual
reliability is concerned with the
E-commerce site’s capacity to implement, satisfactorily, what was
specified and designed. Representative reliability refers to the
E-commerce site’s representation characteristics that affect its
understanding and manipulation through its lifecycle (Cox and Dale,
2002). Albuquerque and Belchior’s identification of the factors under
these objectives was derived from their extensive questionnaires to
both the user and developer communities in E-commerce.
Identifying
Further
quality factors and attributes were researched to ensure having a
comprehensive list of quality factors. In particular, scalability and
availability were added as, according to Suh et al. (2009), E-commerce
website software is large and complex, but quality requirements demand
the key performance of factors such as availability, performance,
scalability, and security. This, in essence, provides the biggest
influence on the effective implementation of a website.
Scalability is the website readiness to meet rising demands of users
and usage. Thus, website software applications must be prepared to grow
quickly both in terms of users serviced and in terms of services
offered. According to Firesmith (2003), the need for scalability has
been a driver for much of the technology innovations in the past few
years. Industry has developed new software languages, new design
strategies, and new communication and data transfer protocols, in part
to allow web sites to grow as needed. According to Firesmith (2003),
availability is of two parts. The first relates to the website
accessibility and being up and running 24 hours a day, 7 days a week
and 365 days a year. The second relates to the web software and it
accessibility by diverse types of web browsers. Using the common
denominator of features among all browsers as the baseline and removing
any use of features unique to a particular browser ensures
cross-browser availability. To be available in this sense requires
significantly more knowledge and effort on the part of the software
engineers.
Table 1 lists the factors used with a short explanation of
each factor. At this stage, and based on the academic
research
exercised, it was felt that a list of sixteen factors within three
objectives satisfy an assessment of the quality of the operational
software of an E-commerce website. The next step was to extend the
factors with sub-factors that allow measurement to qualify the
assessment.
Table 2 displays the complete set of those quality
attributes. They are a total of eighty four sub-factors organized
within the quality factors.
- Table 1: E-commerce Quality
Factors -
Usability
Efficiency
The
timeliness of which the website responds to the user.
User-Friendliness
The
user interface capabilities to which the website provides a supportive
experience to the user.
Navigability
The
browsing extensibility which the website’s software allows.
Maintainability
The
reduced effort which the website’s software requires for its upkeep,
enhancing its ability to be kept up to date and usable
Involvement
Capacity
The
measure of which the website can adapt to and attract each user’s
individuality.
Conceptual
Reliability
Functionality
The
extent of the operational aspects of the website software and its
fitness of use.
Security
The
extent of safety assured against malicious or accidental intrusion of
unauthorized users when using the website.
Reliability
The
extent of which the website remains available and working.
Integrity
The
reliability, consistency and correctness of stored data.
Trustworthiness
The
extent to which the user perceives the website to behave consistently,
reliability and correctly, building a trusting relationship.
Content
Adequacy
The
extent to which the information presented is contextually applicable to
the user and sufficient for the user’s needs.
Scalability
The
website readiness to meet rising demands in users and usage.
Availability
The
extent of website accessibility to users through different browsers in
differing times.
Representative
Reliability
Readability
The
appropriate application of the written language within the website
Standards
Conformance
The
extent of consistency applied within the user interface of the website.
Ease
Of Manipulation
The
extent of help provided to operate the website and the software
underneath it.
- Table 2: Identified Quality
Sub-Factors –
Quality
Sub-Factors
1.
Usability
1.4.3
Analyzability
2.5.2
Completeness
1.1
Efficiency
1.4.4
Changeability
2.5.3
Necessity
1.1.1
Time Behavior
1.5
Involvement Capacity
2.6
Content Adequacy
1.1.2
Purchase Process Performance
1.5.1
Attractiveness
2.6.1
Updated Content
1.1.3
Page Generation Speed
1.5.2
Aesthetic Attributes
2.6.2
Correctness
1.2
User-Friendliness
1.5.3
Client Profile Identification
2.6.3
Intelligibility
1.2.1
Understandability
1.5.4
Simulation
2.6.4
User Oriented
1.2.2
Products Information Availability
1.5.5
Additional Services Availability
2.6.5
Respectability
1.2.3
Interactivity
2.
Conceptual Reliability
2.6.6
Concise Content
1.2.4
Learn-ability
2.1
Functionality
2.6.7
Completeness
1.2.5
Information Localizability
2.1.1
Accuracy
2.6.8
Compatibility With Real Store
1.2.6
Response Time Uniformity
2.1.2
Client Support
2.7
Scalability
1.2.7
Forms Of Payment Availability
2.1.3
Information On Product Delivery
2.7.1
Multiprocessor handling
1.2.8
Storage Of Purchase List
2.1.4
Suitability
2.7.2
Farming capabilities
1.2.9
Help Availability
2.1.5
Flexibility
2.8
Availability
1.2.
10 Products Comparison
2.1.6
Interoperability
2.8.1
24/7/365Readiness
1.2.11
“Shopping Cart” Metaphor
2.2
Security
2.8.2
Partial Availability
1.2.12
Printing Facilities
2.2.1
Payment Systems Security
2.8.3
Browser version compatibility
1.2.13
Download Facilities
2.2.2
Vulnerability
2.8.4
Cross Browser Support
1.3
Navigability
2.2.3
Site Authentication
3.
Representation Reliability
1.3.1
Absence Of Navigation Errors
2.2.4
Access Control
3.1
Readability
1.3.2
Minimal Path & Shortcut Facility
2.2.5
Confidentiality
3.1.1
Language Correctness
1.3.3
Drawback
2.2.6
Privacy
3.1.2
Style Uniformity
1.3.4
Navigation Structure Taxonomy
2.3
Reliabilitty
3.1.3
Clarity
1.3.5
Links Visibility
2.3.1
Recoverability
3.1.4
Conciseness
1.3.6
Links Visualization Consistence
2.3.2
Maturity
3.1.5
Terminology Uniformity
1.3.7
Alternative Paths
2.3.3
Fault Tolerance
3.1.6
Abstraction Uniformity
1.3.8
Navigational Prediction
2.4
Integrity
3.2
Standards Conformance
1.3.9
User Level Adaptability
2.4.1
Data Integrity
3.2.1
Interface Standards
1.3.10
Interaction Storage Capacity
2.4.2
Data Entry Signalizing
3.2.2
Programming Standards
1.3.11
Mobile Devices Accessibility
2.4.3
Robustness
3.2.3
Navigation Standards
1.4
Maintainability
2.4.4
Audit Trail
3.3
Ease Of Manipulation
1.4.1
Stability
2.5
Trustworthiness
3.3.1
Up-To-Date
1.4.2
Testability
2.5.1
Correctness
3.3.2
Ability To Trace
Qualifying
To
establish a rating system for the factors, a standard statistical
rating scheme based on frequency of expert rating was used to reflect
the relative importance of the different sub-factors within a factor
(Wang, 2003). The weighting system was generated based on questionnaire
results from expert specialists in E-commerce development and
representative consumers of E-commerce.
Participants in the
questionnaires were selected based on meeting qualifying criteria. The
qualifying criteria were derived from the industry best practices. The
selection of the sampled participants was based on the following:
The number of E-commerce
websites developed by the expert / respondent
(at least 5 websites
to be considered).
The level of experience for
each expert / respondent
The complexity and size of
the developed E-commerce websites.
A
total of twenty experts were surveyed from largest IT companies. The
questionnaires were administered by the first researcher visiting these
companies and interviewing the respondents. Each question gave the
respondents the option to select one answer from multiple answers. Each
answer was assigned a score from 0 to 7 depending on the question. The
answers were recorded by the first researcher.
A score of 20
or above was used as the qualification to be considered as an expert (A
score of 20 was chosen for convenience to get a range of experience of
building websites with several different companies). The first
researcher was able to define 13 experts/respondents out of 20 sampled,
most of whom used to be software engineers and
developers.
Once
identified as an expert, the participants were asked to respond to the
questionnaire that asked them to rate each of the sub-factors in order
of importance of their contribution to the factor. The questionnaire
also provides an explanation of how each sub-factor influences the
factor it belongs to.
Initially, one randomly selected expert
was identified for a pilot run to validate the content and style of the
questionnaire. When the answers had been received from the pilot run,
they were verified to make sure that the questions were clear, complete
and unambiguous; the questionnaire was then distributed to the
remaining twelve participants. Data generated from the pilot run was
excluded from the final results.
100% of the twelve respondents
gave back their responses to the questionnaire. However, some of those
who were given the questionnaire did not answer all parts of the
questionnaire.
The questionnaire was administered during a
face-to-face interview with the experts. For each interview engagement,
an introduction orientation was given on the questionnaire followed by
a one-on-one question and answers session to gain the answers and
clarified any ambiguities. The answers were recorded by the first
researcher to assure accuracy and consistency, which follows the
guidelines of the Delphi method. In general, the process of soliciting
answers followed the standard Delphi method (Grisham, 2009) in
soliciting expert opinions.
Ranking
The
rating of sub-factors was done within each factor. The rating is
sequentially based on the order of the importance of the sub-factors in
their influence on the factor. So, for factor S having six sub-factors
of SS1 to SS6, each participant rated each sub-factor according to its
importance in influencing S, where 1 was the most important and 6 was
the least.
In handling missing values for sub-factors, the average
of received responses was calculated to fill in the gaps of missing
observations. It was intended that sub-factors that had six or fewer
responses would be removed from the analysis, but this did not prove to
be necessary as each sub-factor had at least ten responses.
Once
all results were collected, a weighting scheme was applied to reflect
the relative importance (rating) of the different sub-factors based on
the following formula:
Sub-factor Percent Importance = 100 – (M /
N)*100
Where M represents the average rating received on a
sub-factor and N represents the total number of sub-factors attributes
for a given factor. The subtraction from 100 is to reverse the rating
scale of the questionnaire so that the questionnaire rating of “1” has
the highest percentage importance. The final rating achieved has the
highest percentage given to the most important sub-factor, proceeding
to the least important in a descending fashion.
Results
The
final rating achieved has the highest percentage given to the most
important sub-factor as the key factors to assessing the qualities of
an E-commerce website, proceeding to the least important in a
descending fashion. Table 3 shows the rating received by each sub
factor.
-
Table
3: Identified Quality Sub-Factors -
Quality
Sub-Factors
Rating
Usability
Efficiency
Time
Behavior
53
Page
Generation Speed
53
Purchase
Process Performance
33
User-Friendliness
Understandability
76
Response
Time Uniformity
74
Interactivity
73
Learn-ability
68
Help
Availability
52
“Shopping
Cart” Metaphor
52
Products
Comparison
48
Forms
Of Payment Availability
33
Storage
Of Purchase List
27
Localizability
26
Printing
Facilities
26
Download
Facilities
24
Products
Information Availability
23
Navigability
Minimal
Path & Shortcut Facility
80
Links
Visibility
72
Absence
Of Navigation Errors
67
Navigation
Structure Taxonomy
66
Links
Visualization Consistence
48
User
Level Adaptability
45
Interaction
Storage Capacity
29
Alternative
Paths
28
Navigational
Prediction
17
Mobile
Devices Accessibility
17
Drawback
16
Maintainability
Stability
60
Changeability
52
Testability
27
Analyzability
10
Involvement
Capacity
Attractiveness
63
Client
Profile Identification
55
Aesthetic
Attributes
53
Simulation
20
Additional
Services Availability
10
Conceptual
Reliability
Functionality
Accuracy
67
Suitability
58
Client
Support
53
Information
On Product Delivery
44
Flexibility
18
Interoperability
10
Security
Payment
Systems Security
58
Site
Authentication
57
Access
Control
51
Privacy
51
Confidentiality
18
Vulnerability
14
Reliability
Recoverability
53
Fault
Tolerance
47
Maturity
0
Integrity
Data
Integrity
58
Audit
Trail
25
Robustness
22
Trustworthiness
Correctness
50
Completeness
0
Content
Adequacy
Updated
Content
68
Correctness
61
User
Oriented
55
Completeness
52
Concise
Content
39
Intelligibility
26
Respectability
25
Compatibility
With Real Store
24
Scalability
Farming
capabilities
54
Multiprocessor
handling
21
Availability
Browser
version compatibility
57
24/7/365
Readiness
53
Cross
Browser Support
53
Partial
Availability
38
Notification
Integrity
13
Representation
Reliability
Readability
Language
Correctness
75
Clarity
67
Style
Uniformity
38
Conciseness
33
Terminology
Uniformity
24
Abstraction
Uniformity
14
Standards
Conformance
Interface
Standards
61
Programming
Standards
42
Navigation
Standards
31
Ease
Of Manipulation
Up-To-Date
44
Ability
To Trace
38
Structure
35
Documentation
Availability
33
Conclusions
This
paper has determined the factors that assess the quality of an
E-commerce website, identifying and rating the main quality attributes
to this application domain. The list of attributes was derived from the
specialized literature. The survey and analysis described in this paper
enabled a greater understanding of the inter-relations and influences
these sub-factors have on the main quality factors. The results provide
an important foundation for the understanding of quality in E-commerce
websites that will allow developers to assess the strengths and
weaknesses of their sites in order to know where to focus further
development to achieve the high quality needed for E-commerce success.
References:
Albuquerque,
A.B. and Belchior, A.D, (2002), ‘E-commerce websites: a qualitative
evaluation’, Proceedings of the 11th International WWW Conference,
Hawaii, ACM Press, pp. 294-300.
Wang, M., (2003), Assessment of E-Service Quality via E-Satisfaction in
E-Commerce Globalization, [Online] Available at:
http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/68/68
