About EPSS & Technology - Research Reports - usability and research reports about how people interact with and react to software, Web sites and technology. 

Can Navigational Assistance Improve Search Efficiency
First Monday: Providing navigational aids to assist users in finding information in hypertext systems has been an ongoing research problem for well over a decade. Despite this, the incorporation of navigational aids into Web search tools has been slow. While search engines have become very efficient in producing high quality rankings, support for the navigational process is still far from satisfactory. To deal with this shortcoming of search tools, we have developed a site specific search and navigation engine that incorporates several recommended navigational aids into its novel user interface, based on the concept of a user trail. Herein, we report on a usability study whose aim was to ascertain whether adding semi-automated navigational aids to a search tool improves users' experience when "surfing" the Web. The results we obtained from the study revealed that users of the navigation engine performed better in solving the question set posed than users of a conventional search engine. Moreover, users of the navigation engine provided more accurate answers in less time and with less clicks. Our results indicate that adding navigational aids to search tools will enhance Web usability and take us a step further towards resolving the problem of "getting lost in hyperspace". (2001-09-16)

Exploring Users' Experience of the Web
First Monday: While browsing the Web is a widespread everyday activity there is a shortage of detailed understanding of how users organise their Web usage. In this paper we present results from a qualitative in-depth interview study of how users browse the Web and combine browsing with their other activities. The data are  used to explore three particular problems which users have with browsing the Web. Firstly, users have problems managing their favourites, and in particular accessing their favourites through a hierarchical menu. Second, users have problems with combining information across different Web sites - what we call the "meta-task" problem. Third, users have concerns with security and privacy, although these concerns seem to change as users become more experienced with shopping on the Web. We discuss three concepts which address these problems: "home page favourites", "Web clipping" and the "Web card". These concepts are attempts at incremental improvements to the Web without affecting the Web's essential simplicity. (2001-09-16)

Effects of Link Arrangement on Search Efficiency
Forty-two subjects completed three tasks using either a 30-link or a 155-link Web page. Three treatments were used for each Web page design -- one-column, two-column and three-column link arrangements. In all of the 30-link Web page treatments, the link arrangements fit on a single page and did not require scrolling. For the 155-link Web page treatments, all link arrangements required multiple pages and scrolling. Subjects completed the study using only one treatment for either the 30- or 155-link Web page. Subjects randomly selected a task list of three links to search for on the assigned Web page. The subjects that used the 155-link Web pages had a significantly faster search time using a three-column link arrangement, while the two-column link arrangement had the slowest search time. Results for the 30-link Web pages did not show a significant difference in search times for any specific link arrangement. (2001-09-16)

HTML E-Mail: Text Font Readability Study
Web Marketing Today: This study found that: "At the 12 point size, Arial is preferred for readability 6 to 4, while two-thirds of respondents see Verdana 12 pt. as too large for body text. But at 10 pt. and below, the readability preference shifts to Verdana. At 10 pt. Verdana is preferred over Arial for readability 2 to 1. And at 9 pt. Verdana is preferred over Arial for readability by a 3 to 1 margin." (2001-09-02)

Bringing Order to the Web: Automatically Categorizing Search Results (pdf)
Microsoft Research: We developed a user interface that organizes Web search results into hierarchical categories. Text classification algorithms were used to automatically classify arbitrary search results into an existing category structure on-thefly. A user study compared our new category interface with the typical ranked list interface of search results. The study showed that the category interface is superior both in objective and subjective measures. Subjects liked the category interface much better than the list interface, and they were 50% faster at finding information that was organized into categories. Organizing search results allows users to focus on items in categories of interest rather than having to browse through all the results sequentially. (2001-08-19)

Empirically Validated Web Page Design Metrics
A quantitative analysis of a large collection of expert-rated web sites reveals that page-level metrics can accurately predict if a site will be highly rated. The analysis also provides empirical evidence that important metrics, including page composition, page formatting, and overall page characteristics, differ among web site categories such as education, community, living, and finance. These results provide an empirical foundation for web site design guidelines and also suggest which metrics can be most important for evaluation via user studies. (2001-08-19)

Comparative Usability Evaluation Reports
CUE-1 is a comparative usability test of a Windows calendar program (Task Timer for Windows, version 2) carried out by four professional teams. The results were published at UPA98 in Washington DC in June 1998. CUE-2 is a comparative usability test of the popular www.hotmail.com web-site. Nine teams simultaneously usability tested this web-site, but eight of the nine teams missed 75% of the usability problems. When you look at the total number of unique problems identified, only one team reported more than 25% of the these problems. (2001-08-12)

Are There Users Who Always Search?
This study found that there wasn't a single user out of 30 who always used the search engine first when looking for product information. None of the users in our study were search dominant. However, we did uncover some link-dominant users. About 20% of our participants chose links exclusively. Then, when we looked at the individual sites, we saw that for 21% of the sites, every single user who visited only used search. It seems that these sites were search dominant, not the users. Thirty-two percent of the sites were link dominant (users only used the links on the site) and 47% were not dominant to search or links. We find it fascinating that on 53% of the sites we tested, each visitor stuck with a single location strategy — the same strategy employed by all the other visitors to that site. This implies that there is something inherent in the site's design that causes users to choose the search engine or the links, not a hard-and-fast preference of the user. (2001-08-05)

Impact of Navigational Models on Task Completion In Web-based Information Systems
This study investigated performance differences between three different web-based navigation models: linear, persistent, and semi-persistent menu structures. Forty-four college students were placed into one of the three navigation conditions and completed information-finding tasks. No significant differences were found among the models with respect to success in completing tasks or overall completion time. Use of other navigation aids built into the browser such as the HOME and BACK buttons was also measured, as well user satisfaction and perceived usability. No differences among models were found here either. Results of this study tentatively indicate that the three navigation models, as tested, performed equally well and further that designers should perhaps give fair consideration to each of the models pending the purpose and audience of the site being designed. (2001-08-05)

A Comparison of Popular Online Fonts: Which is Best and When?
Usability News: Several observations can be made regarding the examined font types. First, no significant difference in actual legibility between the font types were detected. There were, however, significant differences in reading time, but these differences may not be that meaningful for most online text because these differences were not substantial. It may, on the other hand, be helpful to consider using font types that are perceived as being legible. In this study, the font types that were perceived as being most legible were Courier, Comic, Verdana, Georgia, and Times. (2001-07-29)

What is the Best Layout for Multiple-Column Web Pages?
Usability News: Several observations can be made from this study. First, no significant differences between the layout conditions were detected in terms of search accuracy, time, or efficiency. However, significant subjective differences were found that favored the Fluid layout. Here, participants indicated they perceived this layout as being the best suited for reading and finding information, as well as having a layout that is most appropriate for the screen size (for both small and large screens). They also indicated that the Fluid layout looked the most professional, and consequently preferred it to the other layout conditions. Conversely, the Left-justified layout was consistently the least preferred condition. (2001-07-29)

Where Should You Put the Links? A Comparison of Four Locations
Usability News: Several observations can be made from this study. First, no significant differences between the four link arrangements were detected in terms of search accuracy, time, or efficiency. This suggests that the link arrangement for documents within a single frame does not have a great affect on its actual navigability. However, there were significant subjective differences between the links arrangements favoring the embedded links. That is, participants indicated that they believed that embedding the links within a document made it easier to navigate, more easily recognize key information, promoted comprehension, and was easier to follow the main idea of the passages while searching for specific information. Moreover, participants significantly preferred the Embedded link arrangement to the other arrangements. Conversely, placing links at the bottom of a document was perceived as being the least navigable arrangement, and was consequently least preferred.  (2001-07-29)

Online Groceries and Textbooks: Is E-Shopping the Answer for Today's College Student?
Usability News: Results from this series of usability studies demonstrated that the dot.com sites tended to outperform their brick-and-mortar competitors. For both the grocery and textbook domains, the clarity and intuitiveness of each site's home page was critical to participant success. On the albertsons.com site, participants struggled with the task of finding cheese because there was no dairy category identified. On the varistybooks.com site, participants struggled to find the cost of shipping charges because it was not evident on the home page.  Other problems reported included small, hard-to-read fonts, counter-intuitive menu structures, and incomplete information. Such problems are not difficult to fix, but could have a profound impact on whether users continue with a site and more importantly return to it again. Usability testing is a fast, cost-efficient method to help developers find these problems.  It doesn't seem plausible that an e-commerce company can afford not to do it. (2001-07-29)

E-Learning and Knowledge Management at the Crossroads
Are major corporations spending money wisely on their e-training initiatives? According to a study released last year by CapitalWorks LLC, those companies funding formalized e-training programs would have been better off spending their money on less costly informal and self-study methods. That's because, according to the study, informal methods were shown to increase employee knowledge and productivity far more than formal educational methods. (2001-07-22)

Understanding Online Behavior
ecommerce-guide.com: According to Forrester Research, only 23 percent of companies currently improve their online operations by making use of the data associated with how customers use their Web sites. However, as experts agree and the brick-and-mortar world has demonstrated, understanding and reacting to customer behavior is the number one resource for acquiring and keeping customers. Forrester leaves us with the sobering pronouncement that fewer than 200 business-to-business sites will be standing in 2003. Will understanding "E-Metrics" be a key differentiator for survival? (2001-07-22)

E-Commerce Web Queries: Excite and Ask Jeeves Study
First Monday: This paper reports a study of business related queries submitted to the Excite and Ask Jeeves Web search services. We sampled a log of 10,000 Excite queries and 10,000 Ask Jeeves question format queries from 20 December 1999 to examine the business queries. Findings include: (1) business queries often include more search terms, are less modified, lead to fewer Web pages viewed, and include less advanced search features, than non-business queries; (2) company or product name queries were the most common form of business; and, (3) Ask Jeeves business queries in question form were largely limited to the format "Where can I buy ..." or the request "I want to buy ...". The study provides insights into the beginnings of e-commerce Web searching. (2001-07-15)

Computer passwords reveal workers' secrets
ZD Net News: A poll, which questioned 1,200 office workers, revealed four distinct categories of people when it comes to passwords. Nearly half of the employees questioned fall into the "family" group, choosing their own name or nickname or the names of their partners, children or pets for their login. A third of office workers fall into the "fan" category, by choosing sports stars, cartoon characters or pop stars. The more "self-obsessed" employee comprises 11 percent of the British workforce, picking keywords like "sexy", "stud", "slapper" and "goddess". The smallest group--the "cryptics"--with just 9 percent of the total, are also the most security conscious.  (2001-07-01)

Seven tricks that Web users don't know
IBM DeveloperWorks: Carolyn Synder: Web developers have all sorts of browsing tricks that they have gained from years of experience, to the point where they can't even imagine not knowing them -- right-clicking to open a new browser window, for instance, or using the arrow keys to navigate a list. To Web veterans, these things are so familiar that they seem obvious. The fact that many people don't know these tricks -- and can get completely stuck as a result -- comes as a shock. This article describes seven Web site features that typical non-technical users aren't familiar with, based on data collected from the author's own usability studies. (2001-06-24)

Attending to Web Pages (MS Word)
Pete Fraraday, Microsoft: Using a eye tracking devise the researcher found:

Are the Product Lists on Your Site Reducing Sales?
You can increase sales on your site as much as 225% by providing sufficient product information to your customers at the right time. In our recent research, we found that the design of product lists directly affected sales. On sites that did not require shoppers to bounce back-and-forth between the list and individual product pages, visitors added more products to their shopping cart and had a more positive opinion of the site. By understanding your customer expectations and needs, and designing your product lists accordingly, you can significantly increase your sales.  (2001-06-03)

More Research Reports (2001)

 
 
Computer Statistics

Internet and Browser Statistics Sites
 This page contains links to sites the collect and maintain information on internet demographics and software and hardware characteristics.  (2000-10-29)

The StatMarket
"StatMarket presents raw data computed from millions of daily Internet visitors to Web sites monitored by WebSideStory's HitBOX Tracker." (Apr-18-99)

NUA Internet Surveys
 This site contains many links to research reports on Internet and technology related studies.  (Apr-18-99)

Computer Almanac: Interesting and Useful Numbers about Computers
This site has an eclectic collection of facts and figures about the use of computers, software development, demographics and the high technology marketplace.  (1999-06-01)  

Research sites provide stats on Web holiday shopping
There's little doubt the big online retailers are ringing up hefty sales this holiday, but exactly how much? A number of sites provide a glimpse into just how much traffic and money is being generated through the Web this season, but the real numbers won't be known until fourth-quarter sales figures are reported next year.