Before design, the team already had
With enough research ammo, they began to build the app. The plan was to introduce it during the EURO 2020 where millions of fans crazy about football would be active. Of course, these plans were made long before the COVID-19 outbreak.
Design and development were going smoothly and genuinely fast. By May 2019 the team already had a lab-validated test app with an early version of all the major features. We as researchers constantly supported the team with insights.
But we had zero knowledge of real-life usage. What will happen when the first user downloads and touches this app? How will they react?
The team decided that the product has to leave the convenient and familiar lab environment, so the researchers were asked to go onto the field to collect insights:
Luckily, the finals week of the UEFA Nations League appeared to be a convenient time and occasion for testing our application live. The team adapted the product to the competition and we flew to Porto, Portugal. ✈️
Our goal was set: evaluate our app in a real-life context. But how? We listed the questions we wanted to get answers for and divided them into two categories:
In order to find answers, we’ve defined two separate research streams for these groups of questions. These research streams operated independently on the field to ensure unbiased findings. Stream 1 focused on longer adoption of the app while Stream 2 collected intercept insights about first-time users.
Method: Diary study
With this stream, we wanted to find answers to these questions: What do people think about this app? How do they adapt to it? How does it align with their habits? Is it really something that fulfills the needs of tournament goers in the long run?
Method: Intercept field study
With this stream, we wanted to find answers to these questions: How do tournament goers understand our current app’s offering? How do they complete the given tasks? How easy or difficult is it to use the app? What journey steps do they go through when they start using it?
I organized and led Stream 2: UX evaluation.
Probably the most time-consuming part of our initial planning was to brace ourselves for every possible scenario – both in terms of research and logistics. We basically sat down on a daily basis and gave a lot of thought to how to best cover everything.
In order to have a large variety of different users, we had to set up our criteria.
We identified two main user groups for the testing: Travelling fans (unfamiliar with the city, living 50+ km away or abroad) and Local fans (living in or nearby the city). In order to adapt our most suitable scenario, we also wanted to ask about their favorite teams, days of staying in the city, basic demographics, and some of their planned programs.
But how can one verify these on the field? How can we screen them if we only have a few moments before they lose their interest?
We have printed a screener spreadsheet with all the necessary information we wanted to learn. However, instead of approaching them with our boring questions upfront, we immediately started with the fun part: showing them the app. Since people don’t usually go alone to watch football games, the second researcher could target another group member (Secondary user) and ask the screening questions from them.
After a few minutes, once our primary user got enthusiastic about the app, we could slowly ask our questions from them as well, while completing the tasks.
This way we were able to have an intercept and one or two short interviews from a group.
We aimed to intercept football fans in environments most suitable for tests and short interviews. In order to do so, we had to run through our initial assumptions about locations and cross-check them with our football experience. So I ran in-head mind-palace Q&A sessions with myself on a daily basis. It sounded something like this in my head:
→ I guess airports, train stations, fan meeting points, football events in the city center, and so on.
→ Probably yes. Let’s get written permission from officials to do research on the location sites.
→ Well, fans are likely to get directions to the city center when arriving at the airport and train stations. They’ll possibly wait at baggage claim. I assume they would be interested in football-related programs close to their hotels and the city center. They probably want to read news and facts about the events to kill time while they are waiting.
→ Check arriving flights from supporters’ countries. Check train schedules between cities where games take place. Build a plan to follow the fans’ journey throughout the week.
…and so on.
Planning: check. Now let’s turn your plan upside down. If you’ve ever done any field research, you know planning isn’t everything.
You get there, suddenly realize there’s construction on your desired location, potential users are nowhere close by and your app keeps on loading due to a poor mobile network. What do you do?
These are exactly the reasons that make us go and observe the real context of our product in the first place. Alright, none of the above really happened to us this time, but it could have. Here’s something we experienced though:
By the time we got to the first location (airport), we already had a spreadsheet printed with arrivals of countries with upcoming matches. But as we tried catching people coming out of the arrival gates almost everyone refused to help us.
Makes sense, right? You get off a plane, you collect your luggage, you’re probably tired and just want to get to your accommodation.
So we revised our tactics and used our Beaver scout observational senses. Instead of catching people on the move, we started looking for people who waited for something. Where do people wait at the airport? Taxi pickups, public transport stops cafés. We followed the crowd and realized that the train from the airport was leaving every 10-15 minutes.
So we have spent a day and a half traveling back and forth between the airport and the city center, talking to a lot of people both at the stop and on the train. It’s quite a repetitive way to learn “Next station” in Portuguese if you ask me! 🤷♂️
Football fans not stopping for our test appeared to have another reason: they did not know who we were. People don’t trust some random dudes going around asking a bunch of questions with a phone in their hands.
Luckily, we had initially planned to spend our first morning piloting and fine-tuning our research process. We had applied to get a basic research accreditation for the game a few weeks in advance, which we received the next day. It came with a large badge – displaying our profile photo and name. Very official! We also bought some visible Nations League merch, so football fans could quickly identify us (and it was also a lot more fun to do research in a footy scarf).
Fortunately, we managed to overcome every one of our challenges and carried out a successful study. The results were invaluable: we have collected insights we wouldn’t have done in any other way. Here’s a handful of them:
Initially, our app presented information about the finals week in a tree-like system: you have main sections, then sub-sections within, then even more granular information on a third level. This system worked on the lab tests, but when we went on the field, our system kind of fell on us.
During the intercepts, a number of users started spending too much time jumping back and forth between sections and submenus to collect the small bits of information in the necessary articles. This was overwhelming, and they couldn’t even find the desired content in some cases. Their mental model was built-up in a horizontal structure: “Show me everything I need to know today!”
So the product team put a new “mix” to the home page with relevant info and interesting details of multiple sections. It served kind of as a feed of ‘most relevant information at the moment. The team kept the original navigation as well but restructured and renamed sections for better findability.
Before the study, we thought ticket holders would spend most of their time using our custom map that had built-in football interests, shortcuts to football activities, route planning and more.
On the field, we learned that although people took their time to use it, they did not find that much value in it. Our football fans, such as other audiences, were likely to use their favorite apps they’re already familiar with – Google Maps, Apple Maps, Maps.me.
On the other hand, they found value in our Tips: a section where we would show them interesting facts, notifications and events going on in the city at the moment that are connected to the games.
We used a variety of names in our article, search, and map to identify locations. Some examples:
Can you quickly tell what these names are? Neither did I. The first one Estádio do Dragão is actually Porto’s home stadium, where most of the games have been played at. São Bento is a train station in Porto city center, while Guimaraes is a city in Porto’s agglomeration.
We’ve learned that users had problems remembering local names, and identifying what is what. They used terms like “Porto stadium” or similar in their mother tongue. So we had to improve our app by adding English alternatives to locations. After a bit of Googling, I found out that this was an issue that Google Maps also identified and solved with alternative street/interest names.
So the app team simply added subcategories and alternative names:
We also came up with new name ideas to almost all sections within the app as users had problems identifying the content behind them.
Every field research is an opportunity to check real-life situations. We had the chance to talk and evaluate our app with more than 50 participants in 10+ scenarios. Then we compared our intercept study with the diary stream to have the full picture.
Our main takeaways from this project:
UX studio works with rising startups and established tech giants worldwide.
Should you want to improve the design and performance of your digital product, message us to book a consultation with us. We will walk you through our design processes and suggest the next steps!
Our experts would be happy to assist with the UX strategy, product and user research, UX/UI design, etc.
Imagine a future where lesson planning takes minutes, not hours—this is the promise of AI-driven…
In UX, most of the attention is on the surface: people are interested in wireframes,…
Slowly but surely, 2024 comes to an end, but the significance of investing in user…
Don’t judge a book by its cover. In the case of e-commerce websites from the…
You are standing in front of the participants, wondering where it went wrong. Why aren't…
The FinTech industry is rapidly growing and it can’t be stopped. As everything becomes more…