SAFEcar Usability Testing

April 2019

Skills Used

• Writing Usability Test Script

• Recruiting Users for Testing

• Moderating In-person and Remote Usability Tests

• Analyzing Qualitative & Quantitative Data

• Synthesizing Findings

• Presenting Recommendations

Test Objectives & Hypotheses

Objectives

1. Evaluate the navigation

   • Do users understand what the icons in the fixed navigation menu?

   • Can they navigate to the alert settings to change them?

2. Learn if users can set up video feeds & change alert settings

   • Do they understand the video feed set up instructions? And can they successfully interact with the video cameras?

   • Do they understand what each setting changes?

   • Is it what they expect to see?

3. Assess user confidence

   • Does the user know when a change was made?

   • How confident are they that they correctly performed the task?

Hypotheses

1. Clear instructions will inform users how to set up the video feeds.

2. Multiple points of entry to the alert settings will better enable users to change alert settings

3. Clear feedback will increase confidence that alert settings have been changed or updated.

Approach & Methodology

With these three objectives in mind I combined task based evaluation and qualitative discovery into my usability test.

Five in-person usability tests and interviews were conducted over one week. The usability tests and follow up questions were video recorded as participants completed tasks with the SAFEcar prototype on an iPhone.

To measure the learnability of the video feed set up and changing alert settings I used asked target users to complete three tasks:

1. The first task was to set up the video feeds in the app

   • Since the app is still in development I did not have video cameras for users to interact with during the set up. To replicate the experience, I substituted a video camera with a bluetooth Bose® Speaker that makes a sound when the power button is pressed and it connects to a device.

2. The second task was to enable alerts

3. And lastly, I asked users to change their alert settings.

Because this was the first time I’ve tested the SafeCar app I incorporated qualitative discovery to:

• assess if the design met users’ expectations

• how confident they were in their actions

• and if they understood the iconography in the app

Analysis

Quantitative Data

I used task success tracking with levels of completion to analyze the task based evaluation portion of the usability test.

Findings

After analysis, and reviewing my usability test goals;

• I discovered users easily learned how to set up video feeds and change alert settings but struggled to navigate to the settings.

• Additionally, users’ confidence in their interactions with the app was high but users also expressed how they would need to interact with the app more to gain confidence in the apps ability to alert them.

Task 1: Video Feed Set Up

• Every user started setting up the Video Feed correctly.

• Unfortunately, only 3 out of 5 users completed the task successfully without any assistance because of prototype timing errors.

• Despite prototype malfunctions, 4 out of 5 users were confident that the video feeds were set up at the end of the task.

Task 2: Understanding of Navigation

Users’ understanding of the navigation was hit or miss

• All users correctly identified the settings icon.

• However, only 3 of 5 users knew what the notification icon was

  • and all the users expressed uncertainty about what the car icon would navigate to.

I believe evaluating these features in future usability tests would shed more insights on users’ understanding of the app’s features and the navigation’s iconography.

Task 3: Enable Alerts

3 out of 5 users correctly enabled alerts.

• The users who didn’t correctly enable alerts tapped on the alert settings icon.

• The users who correctly enabled alerts stated they did so because it seemed like it was the easiest thing or because the toggle button was so apparent on the home screen.

4 out of 5 users did not expect to see “Activity within 6 feet” to pop up when alerts were turned on.

Task 4: Change Alert Settings

• Only one user had complete success changing alert settings without assistance.

• I expected users would click on the settings icon to increase the alert detection area but 2 users wanted to tap on “activity within 6 feet” first.

• 4 out of 5 users correctly used the slider bar to increase the alert detection area as I expected. But 3 out of 5 users stated the slider bar did not react the way they expected it would.

• On a positive note, all users understood the functionality and found value in being able to set an alert time window and change the way they receive alerts

Recommendations

Tier 1:

Increase prototype timing during video feed set up

• Despite it being a prototype malfunction, some users were unable to read the instructions in the amount of time provided during the video feed set up. I listed this as a tier one recommendation because if users cannot set up the video feeds they cannot use the SAFEcar app.

Tier 2:

Make “Activity within 6 feet” clickable and another access point to alert settings

• Users were split in how they navigated to the alert settings. True to my hypothesis, giving multiple points of entry or different ways to change alert settings will decrease the number of attempts to get to the settings.

Add labels to navigation icons

• Since there was so much confusion surrounding the icons in the navigation menu, I think adding labels will help clarify to users where each icon leads to.

Tier 3:

Make “activity within” slider bar incrementally change

• to align with users’ expectations

Add “save” button in settings

• so users know they are committing to the setting changes they’ve made