Needs Assessment & Usability Evaluation
 

I worked in a team with the City of Detroit's Office of Innovation and Emerging Technologies (IET) to conduct needs assessments and usability evaluations on the Detroit Department of Transportation (DDOT) website - DDOT.info.
 

DDOT.info is a mobile-focused interactive web tool for checking city bus schedules. The client needed assistance in creating a better mobile experience and making the site more intuitive, accessible, and inclusive for Detroit's public transit users.

Role: UX Researcher & Designer   |     Team Size: 4     |     Duration: 13 Weeks

Tools: Qualtrics (for the survey), InDesign (for lo-mid-fi wireframes), 
iMovie (for video editing)
Skills: UX Field Research, Surveys, User Personas and Scenarios, Qualitative Coding, Heuristic Evaluations, Needs Assessment,
Usability Testing, Interviews

RESEARCH QUESTIONS

From the information gleaned from our client contact, we established these three research questions.
 

  1. How are riders using DDOT now, and what role does technology play?
     

  2. How are riders currently using the bus system?
     

  3. How are riders getting information regarding routes, stops, rides, closures, delays, and other pertinent bus-riding information?

 

INTERVIEWS

We spent a day in Detroit to better understand users' bus experiences and have the opportunity to observe these interactions directly.

Interviewed 1 stakeholder Interviewed 8 Participants

Observed only analog tools being used by riders and staff at the Rosa Parks Transit Center

Followed an intra categorical approach. In-person interviews were limited to 10-15 minutes Phone interviews lasted between 20-30 minutes

Noted common themes and patterns from our interviewees

Information about bus closures and delays are not provided. Most of our interviewees did not use technology tools and applications to find
pertinent bus information

Despite our observations, analytics from the IET, show that 1500 people are using the tool every day, a number they say is steadily increasing since the release of DDOT.info
 

Analog Tools used in the Rosa Parks Transit Center:

PERSONAS

"Simplicity is key"

"Technology should make life easy"

"Don't fix what isn't broken"

Brandon Lewis

Student 

Goals

  • To have a tool with cohesive information, so he doesn't have to use multiple tools to navigate the city 
     

Needs

  • App tools data matches the PDF schedules

  • A tool with both the city and suburban bus route information
     

Frustration

  • Lack of relevant information

  • Inaccurate data

James Burnham

Software Developer

Goals

  • To have a tool with real-time feedback; downloadable content, and cross connectivity 
     

Needs

  • A tool that includes PDF schedules

  • Public transportation be less tedious 
     

Frustration

  • Limited options

  • Inaccurate data

  • Inefficient tools

Shelia Tompkins

Grocery Store Manager

Goals

  • Prefers to use conventional way-finding such as word of mouth
     

Needs

  • Leaves space in her schedule for hiccups when planning trips

  • New technology is intimidating to her, so she prefers her "wait and see" technique
     

Frustration

  • Inaccurate data on various tools

  • Smartphone-based tool

SURVEY

To get a more detailed analysis of the general practices of bus riders, we distributed a survey to Detroit bus riders learning about their experience taking the bus. The survey questions allowed users to share their experiences with riding the bus, including how long they've been bus riders, how they find information about buses, and what technology tools they use in the process.

25 Respondents

Analyzed results with Microsoft Excel

Pinpointed patterns in responses and to previous project deliverables 

Findings
 

  • No official information about bus closures and delays
     

  • Users that do not utilize technology to plan their bus trips have some level of difficulty in finding a route and stop information.
     

  • Longtime bus riders found it easier to find bus information than newer riders.

Recommendations
 

  • DDOT.info includes information about delays and closures for riders that use the service.
     

  • Integration with the SMART bus transit system would be useful for riders that travel outside of the city. 
     

  • Include critical features of the Transit app and Google Maps into DDOT.info to better serve the
    population of bus riders that employ different aspects of each tool.

HEURISTIC EVALUATIONS 

Our team analyzed DDOT.info using the heuristic evaluation method. The method of heuristic evaluations involves assessing ten usability heuristics created by Nielsen (Nielsen, 1994). These ten usability heuristics were used to evaluate the quality of the DDOT.info system.

Goal: generate a well-rounded, unbiased set of observations before looking for overlapping results among the evaluations

Ten Heuristics:

  1. Visibility of System Status

  2. Match between system & real world

  3. User Control & Freedom

  4. Consistency & Standards

  5. Help users diagnose errors

  6. Error Prevention

  7. Recognition rather than Recall

  8. Efficiency of Use

  9. Aesthetics

  10. Help & Documentation 

Severity Rankings:
 

0. Don’t agree that this is a usability problem OR This is a usability strength
 

1. Cosmetic problem, low priority
 

2. Minor usability problem, fix as time allows
 

3. Major usability problem, important to fix
 

4. Usability catastrophe, imperative to fix

Findings
 

  • Some areas of the interface that feel onerous to read (Heuristic 9, Severity 3).
     

  • A disconnect and a lack of consistency between language that real-world users use (Heuristic 4, Severity 3).
     

  • The user can’t make a mistake and easily correct it.
     

  • The system does not support undo and redo (Heuristic 3, Severity 3).

Reflections

Many potential shortcomings exist when performing a heuristic evaluation using Nielsen’s heuristics. The heuristic analysis was performed by four individuals who are already relatively familiar with DDOT.info, and might not interact with it the same way a new user might try to. Furthermore, our technical skills are relatively advanced, so our idea of what is or is not an issue may differ from users with less technological expertise. 

ANALYTICS DASHBOARD

We chose our metrics based on what would be the most helpful for DDOT.info to know. In other words, the analytics that could give the most insight for DDOT.info to evaluate its product and how it can be improved.

USABILITY TESTING

Usability Test The test consisted of three tasks that each user was asked to complete while the present members of our team took notes on the users’ behaviors. Our goal was to discover what usability issues were present in the DDOT.info system, as well as analyze how easily users find departure times.

5 Users

Pre and post questionnaires. Designed three tasks to answer our research questions which tested two of the major features of DDOT.info. In-depth Interview after completing tasks. Participants recruited through convenient sampling and bus riders in Ann Arbor. We conducted a pilot test with 3 users.

Held a debriefing session to analyze the data after tests and pinpointed common themes and issues.

Findings

  • Key areas on the interface that were onerous to read and navigate.
     

  • Users utilized the map to assist them in finding pertinent bus information, but the map provided did not support users in their journey and made users feel confused.
     

  • Three out of our five users expressed confusion in regard to the navigation bar.

Recommendations

  • Eliminate text that may not be relevant to users.
     

  • Create a map that is larger in size and more detailed.
     

  • Enable users to go back to the homepage of DDOT.info when they click on the top left logo.
     

  • Ensure that the navigation bar categories appear when user first arrives
    at the selected bus route page.
     

  • Differentiate the navigation bar categories “Overview”, ‘Bus Stops”, and “Schedule”. Perhaps, separate this category from the main DDOT header. The hierarchy of material may be very useful in
    this regard as well.

FINAL VIDEO

 
  • Black Facebook Icon
  • Black Twitter Icon
  • Black Instagram Icon
  • Black LinkedIn Icon