Web Usability Study
due in two weeks

An organization's website should meet the needs of it various users and constituents.  In some cases, these users and constituents are internal, such as employees who must find organization forms, policies, procedures, inventories, and such.  External constituents might include political leaders, business leaders, and social leaders who must be left with an impression that the organization is making a positive contribution to the community.  Many websites must be designed to meet the needs and wants of current repeat customers and of prospective customers who are often visiting for the first time.  Local, state, and national government policies can sometimes additionally mandate that certain kinds of information be posted on a website and that a certain format be used for accesing this information.  Given the range of these sometimes conflicting demands, it is not uncommon for a website design to result in unintended user issues that aren't apparent to the designers.  It is threfore important to conduct a usability study on a new website design.

Conduct a usability analysis of the tamut.edu website from the perspective of potential new students or their parents.  This usability study is interested in learning of the impressions of potential customers who might be making product comparisons.  The issues in your topic guide are based on a drawing at a comic website xkcd.com/773/.  This drawing makes a good point: the things about which a university website posts for the sake of internal users, potential donors, community leaders, and regulatory agencies are often not what a potential student wants to find.  In the scenerio of a potential student with a CLV (customer lifetime value) in the tens of thousands of dollars in future revenues, it makes good financial sense to ensure that answers to visitor questions can be easily found amongst all of the other things that are contained within the website.

Complete your usability study as follows, informing participants that they are to find all requested information as quickly as possible. 

  1. Conduct the usability study on five people (individually), including a convenience sample of yourself and four other convenient participants who are not and never have been affiliated with the university:

    • Conduct the first session on yourself without first looking through the website.  That is, the session that you run on yourself should be the very first time that you look at the web site.

    • Start each session by briefing the participant:

      "I will be asking you to look at the website of a real oganization.  Let's pretend that you know someone who intends to start college next Fall and is considering several universities in the four states region. 

      "Your task in this session is to find various pieces of information on the website of Texas A&M University-Texarkana as quickly as possible.  While you are doing this, I will be taking notes regarding how you found each piece of information and how long it took you to complete each task. 

      "I am not recording your name or any identifying information about you on these notes.  They will be submitted to my professor with my project report as evidence that I conducted this session, but they will be returned back to me when my report is returned.  An oral summary of observations made by students in my class, however, might be shared with university officials because they really are interested in how easy or difficult it is to use the school website. 

      "We should be done within about ten minutes.  If you want to quit at any time for any reason, just let me know and you may take my notes with you."

    • At the start of each session, open a new browser screen to the URL above. 

    • At the end of each session, debrief the participant with regard to confidentiality, anonymity, and how you will be reporting your summary of the data that you collect.

  2. In conducting the usability study, use the following as your topic guide (to include issues above). 

    • Do not allow the participant to use any website or external search functions except as a last resort - use of a search function is proof that the website has a serious usability problem. 

    • Create a paper form to use in recording your observations of each participant:

      • Observe and record the steps (e.g., menu items used) that each participant takes to complete each task.

      • Record the approximate amount of time that was taken to complete each task. 

      • Record any affective comments that the participant makes.

    • Ask the participant to complete each of the following as quickly as possible, directly reading to the participant anything that is in quotes:

      1. "Find a page that tells you the process for getting into the MBA program.  Print a copy of the page(s) with this information."

      2. "Find a list of business school professors with professor contact information.  Print a copy of this list."

      3. "Find the phone number and office number (location) of the admissions office.  Print a copy of the page with this information."

      4. "Find the school's street address.  Print a copy of the page with this information."

      5. "Find and print a copy of the campus map."

      6. "Find where a visitor may park on campus.  Print a copy of the page with this information."

      7. "Find the cost of living in a campus dorm as a grad student.  Print a copy of the page with this information."

      8. "Go back again and find the phone number and office number (location) of the admissions office.  Print a copy of the page with this information."

    • In addition to tracking the steps and the time to complete each task, note that you must also report measures of likability in your report (below).  This necessarily requires that you take likability measures during the session.

  3. Your written report must contain the following elements:

    1. research objectives
    2. methodology
    3. findings:
      1. utility (does the site do what it is apparently supposed to do?)
      2. usability (can the user easily figure out how to use the site?)
        1. ease of learning (how easily can the user learn to move around the site?)
        2. efficiency of use (e.g., how much time to complete tasks?)
        3. memorability (can the user repeat a task?)
        4. error frequency and severity (how often and how serious are navigation errors?)
      3. likability (user satisfaction)
    4. conclusions
    5. recommendations

  4. In a separate folder, include the forms with your handwritten notes of your observations of each study participant stapled to the pages that each participant printed.  These notes would include comments on the menu items that were selected in attempting to complete the task, the amount of time (in seconds) that the participant required to get through various parts of the task, causes of these times (download times, errors working through links, etc.), the types of errors that a person made while trying to find information, the number of attempts that were made before success or abandonment of the assigned task, comments made by the participant regarding usability, likability, frustration, and etc.  Do NOT put the participant's name on your observation notes.

  5. This is an individual assignment that requires you to make your own observations and to write your own report.


References with which you are expected to be familiar for Quiz 2:

http://www.useit.com/alertbox/980503.html
http://www.useit.com/alertbox/991212.html
http://www.useit.com/alertbox/20000319.html
http://www.useit.com/alertbox/20010121.html