This article is a report on the “last” stage of the design process for the creation of the FitFeet application: the Evaluation. It describes how the design team conducted a usability test during the development of the FitFeet application. The previous steps in the process are not here included, but involved guerrilla studies, user interviews, information architecture design and prototype creation.
The goals of our test included establishing an expectation regarding the user performance and identifying potential pitfalls to be addressed in the next design iterations. The goal we set for ourselves was to improve the efficiency and ease of use of the application interface.
The finished version includes comments and reasoning on the main design choices and tries to follow not only user feedback, but also the main principles of good design. Below a simple prototype:
The application we designed is targeted towards people that have problems related to the shape and size of their feet. It is composed conceptually of two parts. The first part is an AR application that guides the user in measuring its feet and making a 3D model of it. The model will be used as a reference to find a shoe with a perfect fit, which involves the size and shape of the feet. This part is not easily testable with the tools we were supposed to use (Powerpoint) because they would require more interactivity, especially in the scanning part. The following are some pictures depicting the onboarding/measuring process:
The screenshot is taken from the first part of the app (AR)
The second part of the application is an e-commerce front, that uses your scanned measurement lets you browse for your desired shoe pair given different parameters (such as color, brand or price). After that, the user has the possibility of reserving an appointment in a shop for a try-on (it’s something most of the user really wanted) or alternatively buying the shoes online.
In this final phase of the project (evaluation) we concentrated our testing effort in the second part of the application because of its peculiarity and possibility of testing. The AR application is not testable without a more involved mini-prototype or by using the “Wizard of Oz” technique. All our testing scenarios assume the first part as well and we made sure that each of the participants was fully aware of it. At the start of the test, we, therefore, assume that the feet were already measured.
The test consisted of letting the users interact with the prototype and using their feedback as a basis for proposed design changes. Participants were selected to fit the description of the targeted user group (detailed description follows) and asked to come to the experimental setup in person, fulfill two tasks and answer some questions, which were all prepared in advance to make the process run smoothly.
Choosing subjects for the test
All subjects were chosen because they fit the description of the targeted user group and shared a similar problem. They all struggle to find shoes, both from a physical store and online because their feet are unusually shaped (either very large or small or of two different sizes). Their ages varied from 20 to 30, which is the main user group for the application and they came from different backgrounds. As part of the test, all subjects were asked to disclose some personal information for the purpose of the test, which was taken care of accordingly with the Ethics principles and their rights and therefore only disclosed within the design team. If they agreed, their voices were also recorded during the testing time and a picture of them was taken without including their faces.
The following are descriptions of the main roles that the team members took on. Even though the roles were well defined, for the nature of the experiments and settings, a relative amount of role contamination was intended.
Her goal would be to be a guide for the user, she is the only active member during the test in itself. She will assist in the conduct of participant and observer debriefing sessions. Importantly she will respond to any participant’s requests for assistance, answer questions and explain the task. She will also encourage the participants to think out loud during the testing.
She will provide an overview of the study to participants and make them feel at ease in our test-setting. She will define usability and purpose of usability testing to participants and explain the idea of the application before testing.
They are going to record participants’ actions and comments. We’ll try to record all the interactions in audio form (with the proper consent of course). They are taking care of the technical part of the evaluation and make sure the prototype responds properly.
The experiment was conducted in person, between a team of four experts and one interviewee. The testing took place when possible in a controlled environment of a quiet room to minimize distraction. All the tools required to test the prototype were provided by the researching team and prepared for testing in advance.
The test was conducted between the 10th and 11th of October in multiple iterations. The test subjects were scheduled to arrive at separate time slots, leaving enough time to test and discuss results in between. There were two test subjects per iteration, and three iterations.
The prototype was tested by all team members separately in advance to check for possible flaws and prepare for testing purposes. Time for testing by the design team was then doubled for the time slots of participants. We kept the time it took users to fulfill the task, since taking a long time to finish the tasks could also indicate interface problems. This was done by voice recording each interview (with consent) for a detailed review after the interview.
Between iterations, all of the feedback was taken into account and the design of the interface progressed and changed for the next iteration. Also, the team tested it before every iteration, not merely the first one, following an iterative approach.
Description of the subjects
The test subjects of the test represent our target group of 20-30 and they have different problems regarding their feet shape and size. We selected six participants for the last iterations: George, Sarah, Alba, Davor, Vinny, and Marcel. All the people chosen were contacted for a problem regarding their differently shaped feet.
The first participant George is a 27 years old German marketing consultant with different sized feet. He always has a problem buying shoes because they usually come in the same size. He always ends up with one of the two feet having a poor fit. Sarah is 23 years old engineering student with large and wide feet the problem is not so much finding a pair of shoes, but trying on multiple ones to find a brand with a proper fit. Alba is a 24 biology student with small feet and Davor is also 24 years old having the very large feet (47 and 48). Vinny is a 29 years old musician that online shops routinely. Lastly, Mattia is a 28 years old body-pump trainer in KTH Hallen, tall, athletic guy with big feet (46 and 46).
We tried to find people with shoe problems at KTH Hallen – a gym in the KTH Campus. The reason we chose this place is that it is easy to observe people because there is a sitting area next to the shoe racks.
Each task required that the participant acquires or inputs specific data moves through the interfaces and make some choices to obtain the desired result. There were two tasks, that all users were asked to complete in regard to the part of the interface for shopping for shoes. Due to the time constraint and complexity of the prototype, feet measuring was not tested.
Tasks were as following:
1. Find Adidas shoes and buy the cheapest pair.
2. Reserve Adidas shoes from the store closest to you.
The tasks were written down on sticky notes and presented to the participants separately so they focused on one at a time. The sticky notes helped them remember what they were looking for if they got confused.
The task was completed when the participant indicated the scenario’s goal has been achieved. It would also be completed if the participant received sufficient help as to warrant scoring the scenario with a critical error, which did not happen in this case.
For each iteration, we summarized all the changes that the users suggested to us, then we prioritize them based on importance and frequency. In the end, we draw a conclusion after all iterations and identified 3 main problem categories.
After iteration 1
After the first iteration, the overall feedback was good and we extracted three main problems.
- Checkboxes for filtering. One user suggested that having checkboxes when filtering brands would be a good signifier that this is a multiple-choice menu.
- “BOOK” word is confusing. It implies that the user is going to book the shoes and pay for that, even though we wanted to display that the user is booking an appointment in the store. We changed the word “book” with the word “reserve”, so it would be more clear that the shoes are reserved in the shop.
- A pop-up that shows information about the store in the map view. Initially, when clicking on the location icon of a store on the map, the user would be redirected to the reservation form. This was confusing for the users. We changed so that the users are prompted with a pop-up.
After iteration 2
The main changes that we identified after talking with Davor and Alba were the following:
- “FIND A SHOP” button is not intuitive. We decided to change this into a “RESERVE” button, so this would be more explicit.
- The “List” button on the upper side of the store’s map view was not visible and clear enough. Instead of logos, we decided we are going to use text: “Stores list” and “Stores map”.
- Have the possibility to input e-mail as a way of contact in the reservation form.
After iteration 3 – final version
After the last iteration, there were fewer problems that were noticed by the users. The problems were converting certain details, especially the price.
- Added, “Price range” instead of “Price” for the shoes that have different prices from different shops.
- Added a price filter in order to help the users display only the shoes that are in a certain price range.
- Added the price of the shoes in each store on the map. The users thought it would be very useful to be able to know the price at a shop from the map only at a glance, without having to click on the location icon.
Design-wise, there were three problems that we identified that each user:
- List view vs Map view
When the users are prompted by the shops they can reserve the shoes from, there are two options, a list view, and a map view. In the beginning, we had the list view the first and the map views second. After the first iteration, the users said that they did not notice the “MAP” button from the bottom-right corner and they would rather like to have the shops displayed on a map. We changed the button to be more visible and put the map view first. In the second round of interviews, the users liked that they had a map, but it was hard for them to notice where they are on the map, so using the map was still a problem and one of them preferred using the list view. On the third iteration, we changed the way we display the current location, this way it was easier for the users
- Booking vs Reserve
When the users have to reserve shoes in a store, they have to click on the “Book” button in order to see the map and the list of stores. After the first iteration, the people we interviewed said that the word is not clear enough, because it made them think of the process of booking a room or booking a flight, which implied buying the shoes. We changed the word in “Reserve” and we tested this improvement in the second iteration. After interviewing Alba and Davor, we received the feedback that the Reserve button implies now putting the shoes aside in the store and it is more clear that they are not buying anything online. Therefore, changing the name of the button was a good improvement in the application and we did not change anything about this in the third iteration.
- Filtering vs Searching
When the users want to find something specific in the application, they can use the filter, the sorting, and the searching options. All these options are implemented in the application, but the user interaction with them was different in each iteration stage. Therefore, we noticed that half of the people we interview (in the first iteration) liked to searching for options more because they were looking for something very specific and thought it was more intuitive. Also, they were not usual users of online shopping. However, the other half of the people we interviewed (the second iteration) said that it is easier for them to go to the filter option and choose from there the specific brand or the specific color they are looking for. Therefore, we decided to leave both options, because in usual cases, they are both used.
Execution of the test
The test execution overall was very smooth without any major problems. The biggest issue was the short time frame for testing because it was hard to schedule all of the participants within it. Therefore, one time it happened that there were two participants in the room at the same time, which could have affected the test.
Another problem was that one of the participants could not come to the prepared set up, so that the test took place in a different environment. We tried to make the conditions the same to the greatest extent, but it was still more distracting than the original setup. In the end, however, we did not notice any significant problems with the test results for that participant. The setup we prepared worked well because of its relaxed nature. Participants were calm, they didn’t experience any external technical issues during the test and they had no problems expressing their thoughts, even with the negative feedback.
The task division among members of the team worked well in practice, every team member successfully fulfilled the task. The cooperation was good and discussion efficient and productive. In the beginning, it was hard to approach subjects in the right manner, but the same as with the design progress, the team-work also improved in iterations. After each test, we discussed what could be improved and tried to do better in the next one.
Overall, there was a lot of great feedback from all subjects and the test definitely succeeded in its goal of making the application interface better.
In conclusion, the final proposed version is without any major design flaws. Within the test scope, the focus was on the aspects that limited some users’ ability to complete the task successfully, but the possible minor fixes were also noted as future improvements. Some of the ideas for those interface improvements include adding detailed reviews for each product as a source of additional information for the customer, emphasizing the price of the shoe more in the store list and add a more detailed description of the opening hours. All of these can be easily adjusted in the interface and are worth mentioning because they could increase the overall user experience, but overall this interface proves efficient and ready to use in practice.