Volkswagen's (vw.ca) Usability Testing
An Unmoderated Remote Usability Testing report for vw.ca Build and Price Feature.
00
PROJECT DETAILS
Goal
The purpose of the research is to find identify user goals, behaviors, attitudes, and pain points while using the Build and Price feature to customize a Volkswagen car by conducting an Unmoderated Remote Usability Test (URUT) and make informed design decisions in the future for a better user experience.
Purpose
Insights
To conduct an Unmoderated Remote Usability Testing (URUT)and prepare a report of Volkswagen Canada’s digital product vw.ca for their online tool that allows users to build and price.
Based on the analysis of the usability test, we made several recommendations for the common issues encountered by the user by understanding their pain points to create a better experience while using the build and price feature to customize a car.
01
INTRODUCTION
About
We interviewed people from Volkswagen’s end user’s for their build and price feature. Analyzed the interview data and created user personas to identify user goals, behaviours, attitudes and pain points while using the feature to customize a volkswagen.
We then conducted an Unmoderated Remote Usability Test with recruited participants, based around the goals in order to identify the features’ problem areas and make informed design decisions in the future for a better user experience.
My Role
I did this case study as a part of my UX Design Graduation Project to get insights into the user's journey while experiencing the vw.ca's build and price feature whilst taking a holistic approach in identifying common issues that the user experiences while customizing a car.
I was involved throughout the process of creating personas, defining user goals, identifying research objectives, creating screeners, shortlisitng participants, conducting an unmoderated remote usability test and creating a report to make recommendations to improve the overall experience of the feature.
The Team
Neha Yadav
User Experience Researcher
(Creating Screeners, Creating URUT, Creating Test Plan,
Analyzing and Reporting URUT result, Making Recommendations)
Aastha Purwar
User Experience Researcher
(User Personas, User Goals, Participants Profiles, Recruiting Participants, Making Recommendations)
Nigel Waite
User Experience Researcher
(Research Objectives, Testing Environment, Recruiting Participants, Identifying Common Issues, Making Recommendations)
Recommendation
Based on the analysis conducted of the usability testing, the following recommendations will provide a better user experience.
-
To ensure users receive an accurate price estimate when building their vehicle.
-
The navigation bar should be made visible by moving it closer to the customization options, which will make it more accessible to users.
-
Within each step of the build and price tool, arrows to scroll through carousel should only be presented when needed.
-
Locating the dealership should be made easy to spot.
02
TEST PLAN
We identified the user goals, research objectives, and testing environment to further help in preparing a Usability Test Plan.
USER GOALS
-
To be able to customize a vehicle based on specifications and price point defined by the user.
-
To be able to locate a dealer nearby to check out the customized vehicle in-person and do a test-drive before buying.
-
Being able to customize with easy comparisons among the build models to select the best possible option with required specifications and price.
RESEARCH OBJECTIVES
-
To identify how smooth and user-centric the build and price feature is for a wide range of customers. This will ensure whether the Build and Price feature is meeting the needs of the probable buyers.
-
To further access how the users proceed about buying the customized vehicle and gather insights to improve the overall experience of the buying process.
TESTING ENVIRONMENT
-
This research will be conducted as a remote unmoderated usability testing session.
-
These sessions will be conducted using loop11, which will provide analytics of the participants completing each task.
-
Participants will need a high speed internet connection, as well as a microphone to participate in this usability testing.
PRODUCT UNDER TEST
What product is being tested?
The website at test is vw.ca. We are testing the Build and Price feature on the website to help identify problem areas.
BUSINESS CASE
Why are we doing this test?
Usability testing will ensure possible customers are able to easily receive a price for their desired vehicles.
TEST OBJECTIVES
What are the goals of the usability test? What specific questions will be answered?
-
Pain points while navigating within Build & Price.
-
Understandability of the payment method.
-
User Expectations regarding the final purchase.
We will be looking at the most common problems identified during the usability testing to give us a broad idea of the most commonly occurring issues users are experiencing while using the build and price feature.
PARTICIPANTS
How many participants will be recruited? What are their key characteristics?
4 to 5 participants will be recruited and they must all be 18 and above, have a driving license, and should be comfortable making an online purchase for a car, all based out of Canada.
EQUIPMENT
What equipment is required? How will you record the data?
Desktop or Laptop setting with chrome installed. The usability test will get automatically get recorded on Loop11 while screen sharing for later analysis.
TEST TASKS
What are the test tasks basis the heuristic evaluation?
-
Task-1: You want to buy a customized Volkswagen car under a budget of $30,000. Try and build a car based on your specification criteria under this budget.
-
Task-2: You are done with the customization of the car. You are wondering about financing the car for a year with monthly payments. Try and figure out the total monthly installments before purchasing.
-
Task - 3: You are satisfied with your customized car and payment criteria. You have decided to go ahead and buy the vehicle.
Usability Test Plan
03
SCREENING
Task
We created user personas based on Volkswagen's clientele in order to recruit participants based on the recruitment screener sheet which helped us in identifying 'Who we want to talk to' and 'Who we don't want to talk to' based on the set criteria to come up with screener questions that would help us in shortlisting the participants for the Usability Testing.
Recruitment Screener Sheet
18 to 60
WHO WE WANT
TO TALK TO
What exact criteria will identify the people you want to talk to?
What screening questions will you ask?
Mix of men and women
Mid to high range in educational background
Driven a car
Level of technical knowledge of vehicles
Quota : 2 - 18-35, 2 - 35-50,
1- 50-60
3 men, 2 women
3 with high-school education,
2 with a College/University education,
Have a driver’s license
1- owned 0 car, 3- at least 1 car, 1- owned more than 5 cars
Age?
Gender? M,F
What is the highest level of education you have completed?
Do you currently have a driver's license (yes,no)
How many cars have you owned in the past?
People who work for a car company
WHO WE DON'T WANT
TO TALK TO
What exact criteria will identify the people you don't want to talk to?
What screening questions will you ask?
Minors
Those familiar with the customization process of car
Driven a car
Direct connection with people in an automobile sector
Employees of a car dealership or manufacturers
People younger than 18 years old
Those who have used the VW build and price tool before
Has a learner's license
Family members and relatives of people who are employed at a car dealership/ manufacturer
If you are employed, who is your employer?
Age?
Have you ever received a price estimate on a personalized car online?
Do you currently have a driver's license (yes,no)
Do you currently have a driver's license (yes,no)
Participant
Profiles
We recruited a mix of male and female participants based out of Canada for this usability test. We ensured that all the participants own a driver's license, aren’t employed with an automobile company, don’t have any friends or family members working for a car company and finally, are comfortable making online purchases for cars to qualify for participation.
AGE
GENDER
EDUCATION
LOCATION
PARTICIPANT 1
PARTICIPANT 2
PARTICIPANT 3
PARTICIPANT 4
57
27
24
24
F
M
F
M
College/ University Education
Summerside, Prince Edward island
Graduate/ Masters/ Phd
Toronto, Ontario
College/ University Education
Ottawa, Ontario
College/ University Education
Halifax, Nova Scotia
04
TEST SETUP
We conducted an Unmoderated Remote Usability Test(URUT) on the Loop11 platform with 4 participants (recruited using a SurveyMonkey screener prior to this test) to accomplish 3 tasks on Volkswagen.ca build and price feature.
Introduction
We shared the usability test link with each participant recruited using the screener on SurveyMonkey. Before the usability test, we did the following to prepare for it.
Gave an Overview
-
Prompted that they would have to complete 3 tasks & then answer 1 question for each task.
-
Should try to clearly & loudly verbalize their thinking & actions.
-
When finished with their task, they must go ahead and click the ‘SHOW' button at the lower-left corner of the screen.
-
Would take only a few minutes to complete.
Seeked participants’ permissions & consent
-
Informed participants about our requirement to record their participation.
-
Asked for their consent to record screen and voice by granting access once they commence the test. We gave them the option to leave the study if otherwise.
Provided interface awareness
-
Study would open in a new tab, automatically taking them to the website being tested
-
Tasks and questions providing instructions for them to follow during the study would also appear over the top of the website being tested.
-
Tasks and questions would always be available via a small circle which, when not active, is positioned in the corner of their browser.
-
To click 'Begin’ whenever they are ready to participate
Tasks
The tasks we asked our participants to perform. We later asked our participants about how easy or difficult they found the following tasks.
TASK-1 : You want to buy a customized volkswagen car under a budget of $30,000. Try and build a car based on your specification criteria under this budget.
Post-Test Question : Overall, how easy or difficult was it to customize your Volkswagen Golf?
TASK-2 : You are done with the customization of the car. You are wondering about financing the car for a year with monthly payments. Try and figure out the total monthly installments before purchasing.
Post-Test Question : How easy was it for you to figure out your monthly installments?
TASK-3 : You are satisfied with your customized car and payment criteria. You have decided to go ahead and buy the vehicle.
Post-Test Question : How easy was it for you to locate a dealership in your city?
05
RESULT
Overview
Our Usability Test on Loop11 helped us uncover the areas the user most commonly encountered problems while trying to achieve the specific tasks provided to them. Based on the data and insights gathered after the usability test we have come up with a color code rating system to identify the difficulty level of the tasks according to the participants:
Easy
Neutral
Difficult
Very difficult
Overall Task Analysis
80%
of participants understood the core premise and overall concept of the build and price feature and were able to complete the tasks successfully.
15%
of participants found it difficult to figure out how to go about a certain task while using the feature or had some technical error encountered during the testing.
5%
of participants experienced difficulty while using the feature.
Task - 1
Navigation within the build and price feature during the customization process.
Easy
100%
of participants were able to navigate within the build and price feature during the customization process.
0%
Participants failed or abandoned the task.
Task - 2
View financing options for customized vehicle.
Easy
50%
of participants understood the financing options available and were able to figure out their monthly financing cost.
25%
of participants failed to locate the options to select the filter to find their monthly costs.
25%
of the participant abandoned the task due to some technical error.
Finding the dealership to purchase the customized vehicle
Task - 3
Easy
75%
of participants were able to locate a dealership by entering their location and selecting the dealership closer to their place of residence.
25%
Participants abandoned the task due to some technical error.
06
ANALYSIS
Common Issues
After analyzing the usability test results we were able to uncover the areas the user most commonly encountered problems while trying to achieve the given tasks.
Task - 1
Most participants did not notice the option to view the cost of the vehicle including sales tax, which would have raised the cost above the allotted budget.
Options within each step were presented on a carousel, even when all options were visible at one time. These arrows were mistaken as the way to reach the next step in the building process.
Task - 2
-
It was not clear to all users which options were selectable in financing tables.
-
Technical language made it difficult for users to quickly understand the information they are trying to access.
Task - 3
Participants missed button for ‘Find a Dealer’ and needed to scroll through page several times in order to locate it.
07
RECOMMENDATIONS
Based on the analysis conducted of the usability testing, the following recommendations will provide a better user experience.
-
To ensure users receive an accurate price estimate when building their vehicle, the sales tax option should automatically be displayed with the price. Especially as the sales tax is on such a large product with a significant amount, it should be automatically provided to users.
-
The navigation bar should be made more visible by moving it closer to the customization options, which will make it more accessible to users.
-
Within each step of the build and price tool, arrows to scroll through the carousel should only be present when necessary. Controls should not be presented to users if they don’t do anything.
-
Typical users use the build and price feature in order to get price estimates prior to speaking with a salesperson. In order to allow customers to reach a salesperson as efficiently as possible, the ‘find a dealer’ option needs to be made more clear on the page.