Pesquisa quantitativa e testes qualitativos
Category tree and navigation menu revision
Stages /
Card Sorting prescription
Ferramentas /
Tree testing (quantitative)
Prototyping
Usability test (qualitative)
In the context of Tok&Stok's product category tree being reviewed as to bring SEO benefits, our squad was activated as to bring the users' point of view into the new proposal, and also understand what changes would that imply to the navigation menu's front-end of the website.
The two main categories, Furnitures and Accessories, would be separated into 10 new first level categories, such as Furnitures, Accessories, Decoration, Bath and Bedding, Kitchen Utensils, and so on.
Quantitative tree testing
We considered the actual understanding users had over the new category tree a hypothesis to be evaluated in itself:
"The new disposal of the category tree an improvement for the understanding and navigation of the actual users"
The selected research method was Tree Testing, which specifically evaluates how useres navigate through a category listing, without considering the interface. The Tree Testing was conducted in a quantitative and unmoderated way, through a platform named Optimal Workshop.
Research plan
We listed the elementary compontes of a quantitative research in the planning stage:
Cause Variable
Effect Variable
Sample
Control group
Cause variable -
The new disposal of the category tree was considered as the changed variable, and the categories with the most revenue were selected to be seached by the users in the experiment.
Effect variable -
The following parameters were observed:
Total successes;
Direct successes;
Backtracking;
First click;
Sampling -
The research was sent to the customer's email in Tok&Stok's database, as to attain a margin of error of 10%, as well as a level of confidence of 90%, considering the population as being the number of weekly users of the website.
Control group-
Emails were alsi sent to a different set of users (of the same profile), containing the previous category tree ordering.
Results:
The overall performance of each version was paired side to side for an overview. Also, looking in a qualitative way into each category, it was possible to indentify improvements and complications in detail.
By looking at the first clicks, we noticed some categories undesirably and consistently drained the traffic of others, such as Puffs being searched as a decoration instead of furniture, and Kitchen Utensils and Table Utensils competing with each other.
We also discovered a lot of backtracking in correct categories, such as sofa, implying the subcategories weren't named properly.
We also noticed some items were being searched in a different place rather than the ones proposed by the company, such as the bedside table being searched onTable and Beds instead of cabinets.
Through these findings, it was possible to identify several oportunities of further improving the SEO team's new category tree proposal, which were presented to them for reviewing, such as the necessity of renaming some categories and adjusting the positioning of some items.
We also prescribed several items that had ambiguous results and would need to be duplicated into more than one category when the category tree revision actually reflects into the navigation menu's front end.
Protoyping and usability testing
A mid/high fidelity prototype was elaborated as to understand the changes in the navigation menu caused by the category tree.
Given the increase in the amoun of primary categories from 2 to 10, we indentified the necessity of creating an all including key word containing all of them as it would't be possible to list all of them openly. There was also the possiblity of keeping the same 2 categories as primary level on the menu, Furniture and Accessories, but reordering every other category inside Accessories in itself.
The new menu desktop version would also need to have 3 unfoldings, instead of only 2. The two versions bellow were ellaborated through benchmarking.
Research questions
To guide the stage of usability testing the solution, a few questions were elaborated as research questions that would need to be answered at the end of the stage, and were further adapted into script questions.
- What sorts of effects were caused over the user when grouping every single category inside the "All categories" keyword (previously validated through a survey), compared to having "Furnitures" and "Accessories" shown openly?
- Are the users able to open several foldings of te menu and scroll inside them?
- What is the user's reaction when first looking at the option with more elements shown at once?
- Are there any further complications in the mobile version usability?
Moderated remote usability test
The usability test was conducted with 6 customers of the e-commerce, who all navigated through both desktop and mobile versions, but from which 3 of them started by a different version. This test was conducted so that the customers received verbal instructions to complete tasks through a remote call, while sharing their screens.
Results
The major discoveries were related to the research questions:
What sorts of effects were caused over the user when grouping every single category inside the "All categories" keyword, compared to having "Furnitures" and "Accessories" shown openly?
There is a trade-off between the feeling of assurance that whatever is being searched can be found, and how actionable the menu is on each version. This trade-off can only be measured through a qualitative and comportamental analysis.
Are the users able to open several foldings of te menu and scroll inside them?
Yes, although notebook users have a bad experience scrolling with the track pad.
What is the user's reaction when first looking at the option with more elements shown at once?
Some users would actually give up searching on this option.
Are there any further complications in the mobile version usability?
No, only the trade-off stated previously
Next steps
After the usability test, it was possible to select which version of the menu to use, given a few corrections were made to reduce the need of scrolling inside it.
An A/B test was prescribed to better understand the trade-off mentioned previously, tracking down the following product metrics:
Menu clicking - Understand how actionabel each version is.
Access to product pages after using the menu and without using the menu, in each version - Understand how assertive the menu navigation is in each version.