UX research has become an integral part of the Product Designer’s toolkit and is here to stay. It is beneficial to involve not only designers and analysts, but to encourage broader participation in the research process. There are numerous methods available, each with its own nuances and may be unclear to management unless the value of research is adequately explained. Why data scientists may struggle to understand extrapolation based on three data points, how outsourcing research differs from an internal UX lab (seemingly not much) — all about it in this post.
It's a series of posts called #overheard. Here you will find captivating and insightful content based on the most interesting excerpts from design podcasts (and not only).
This post is an excerpt of the most interesting thoughts and information from the podcast “Research everyone: UX research in the product”.
Authors and guest:
Nikita Lakeev
Roman Nurgaliev
Mikhail Pravdin
What is qualitative research?
a
To understand well what qualitative research is, you need to go up “one level” higher — what is research in general, why do we need it when creating a product?
b
Market analysis of competitors — this is also research.
c
Therefore, “one level higher”, I would talk about how we can reduce uncertainty, reduce risks that it will not work (solution, product or service). And for this we use different methods: market analysis of competitors.
d
If the product is existing — everyone uses analytics to understand where there are problem areas and how to improve → analytics. This is also research.
Market surveys, market research (marketing) or quantitative research, quantitative surveys. These can be calls, questionnaires, street opinion polls and other options — all this is research.
Market surveys, market research (marketing) or quantitative research, quantitative surveys. These can be calls, questionnaires, street opinion polls and other options — all this is research.
e
Qualitative Research — these are usually studies where small samples, Face to Face studies. But I would also call qualitative research when, so to speak, Face to Monitor. For example, analyzing a user session in a webvisor, for me this is also qualitative research, because we look at a specific user road and even if we do not ask questions, do not sit with him on tests, we also qualitatively study his experience and his path.
f
There is also Remote Research (Observation) in qualitative research — if it is not a webvisor, then you can install a plugin for the user in the browser and we record all his actions on certain sites, all actions. This is even more powerful than the webvisor tool. For example, on all sites where you can make purchases. Of course, we warn the user in advance and he knows that his actions on the site are recorded. We also observe him without interfering with his actions, without asking any questions — and this is also qualitative research.
g
In response to your question, for me, Qualitative Research is one of the methods to better understand our client (user), and it is a way to mitigate the risks of launching something that won’t be successful. Typically, qualitative research involves in-depth study of a significant user group, allowing us to gain valuable insights and make informed decisions.
Why conduct research?
a
Without a researcher and without research, you cannot now make a product that will be successful in the current competitive market. Probably in the 2000s it was still possible to do: there were not so many products and services and users were not sophisticated, so you could do without Research. Now it is very difficult.
b
I like the metaphor about a doctor: when we come to a doctor, can a good doctor make a correct diagnosis without talking to a patient and without taking tests? Most likely not. Can he make a correct diagnosis by simply measuring the temperature or taking some simple smear, blood test and based on this make some diagnosis? He can, but if he also talks to the patient, then he will already have both the data that he collected from the tests and the sensations of the person who comes to him with some pain. And based on this, the chance that he will most likely make the correct diagnosis will be higher.
Here is the same thing we transfer to Product: you can launch a product without collecting tests, without communicating with users, but what is the probability that the “shot” will be correct, that you will assign the “diagnosis” correctly?
Here is the same thing we transfer to Product: you can launch a product without collecting tests, without communicating with users, but what is the probability that the “shot” will be correct, that you will assign the “diagnosis” correctly?
c
Tests — this is what we can collect without contacting a person (user). For example, his scenario, how he went through the site, look in Google Analytics; how many purchases he made; look at the length of his session, etc. That is, everything that we can collect without asking the user.
Interview — collecting some qualitative information, Face to Face, this is exactly when a doctor communicates and asks “What hurts you?”.
Interview — collecting some qualitative information, Face to Face, this is exactly when a doctor communicates and asks “What hurts you?”.
d
Can a doctor make a diagnosis just by talking to a patient? And here UX Designers and UX Researchers often make a mistake — talking to a user, he tells them what hurts him, what he would like and we run to do it. And here it would be cool to reinforce your knowledge obtained from the interview with other tools or ways to do some analysis.
e
A cool case was told by guys from “Party of Food” (Editor: the service was bought by Yandex and later ceased to exist):
when they conducted interviews with clients and asked what they would like to change in those dishes that they send delivery, and many clients said: “Oh, you know, I wouldn’t want this fried, fatty, pork, red meat and so on …”.
Then they decided to back it up with data from analytics and unloaded data “Which dishes are ordered most often?” and in first place was creamy pasta with bacon, and in second place fried pork with french fries 🗿.
when they conducted interviews with clients and asked what they would like to change in those dishes that they send delivery, and many clients said: “Oh, you know, I wouldn’t want this fried, fatty, pork, red meat and so on …”.
Then they decided to back it up with data from analytics and unloaded data “Which dishes are ordered most often?” and in first place was creamy pasta with bacon, and in second place fried pork with french fries 🗿.
This is about what people say and what people do.
How to conduct minimal research? What is required for this?
a
If you do not have a Researcher in your team, there is no budget for research, if the Head does not support this idea and you yourself decided to secretly go out and conduct research: communicate at least with colleagues (corridor tests) or find people at least through Facebook for tests and surveys — it is better to do at least so than not to do at all.
b
Any tool that helps create an objective picture goes into action. Even the most unprepared, unstructured and insecure exit to your audience gives more information than what is only with you, in your worldview, in your representation. This in any case throws you some hypotheses, answers, sheds light on what seemed to you reality.
This definitely prepares you for the next steps. Through such a practice, through not very high-quality approaches to research, we gradually learn, albeit a little, not to be afraid to do this.
This definitely prepares you for the next steps. Through such a practice, through not very high-quality approaches to research, we gradually learn, albeit a little, not to be afraid to do this.
What problems can arise in research (qualitative)?
a
Qualitative research in itself means nothing. The research must have a business goal.
b
We have the first level in Research — the business goal. The second level is the research objectives.
Business Goal and Objectives — what we want to do, what we want to get, what questions we have, what tasks we face now. Based on this, we form the Research Task, and based on this, we select the Research Method.
That is, one of the mistakes is “I will go and conduct an interview!”. You started with the third level, you started with the Method. Maybe you don’t need an interview, maybe you need a survey, or maybe you don’t need research at all, maybe you have the second level and you don’t have any goal in research. And when you start with the goals and objectives of the business, then move on to the goals of the research and then you can understand that you need exactly a qualitative research method.
Business Goal and Objectives — what we want to do, what we want to get, what questions we have, what tasks we face now. Based on this, we form the Research Task, and based on this, we select the Research Method.
That is, one of the mistakes is “I will go and conduct an interview!”. You started with the third level, you started with the Method. Maybe you don’t need an interview, maybe you need a survey, or maybe you don’t need research at all, maybe you have the second level and you don’t have any goal in research. And when you start with the goals and objectives of the business, then move on to the goals of the research and then you can understand that you need exactly a qualitative research method.
c
Qualitative Research Methods — usability testing, user interviews, diaries, user observation using a webvisor.
d
Nikita Lakeev: I’ll tell you a case, Once I asked a girl on Instagram, I asked one question “How do you use Freeletics?” and I got twelve audio messages. She told me everything. Literally EVERYTHING. Well, here is the cheapest research in the world 🙂
Feedback on the service (AppStore, Google Play, Support Service) does reading these reviews belong to qualitative research?
a
When we talk about research, we usually talk about two axes: on one axis we have Qualitative and Quantitative methods, on the other axis we have Behavior and Opinions of Customers. Reviews are a quantitative method about People’s Opinion.
b
What is bad about quantitative methods — we can very poorly answer the question “Why?” from them:
“Your app is shit!”
“I want the app to have a function to upload content to Instagram Stories”
“I want a “Show All” button in my Personal Account”
But why do you want it (the button)?
Ideally, with such feedback, it is cool to work in conjunction with qualitative methods: you collected a group of requests on one topic, unloaded 5-10 clients from this large group and just by calling them you can answer the question “Why?”.
“Your app is shit!”
“I want the app to have a function to upload content to Instagram Stories”
“I want a “Show All” button in my Personal Account”
But why do you want it (the button)?
Ideally, with such feedback, it is cool to work in conjunction with qualitative methods: you collected a group of requests on one topic, unloaded 5-10 clients from this large group and just by calling them you can answer the question “Why?”.
How to deal with management objections on the topic of spending time, human resources and money on research? How to explain the value and benefit of research?
a
I read once on Facebook: “Guys. Get out of companies where they don’t let you spend time on research”.
b
Let’s imagine your leader doubts, is not sure about conducting research, why do we need it, if there is a backlog of 400 ideas. I think the best thing you can do in this case is demonstration. And even better demonstration with empathy.
If we manage to pull the boss out to such approaches (here design sprint, design thinking helps well), work out, then maybe you will be able to “switch” his indecision. When does the “switch” happen? It happens when “I thought we would launch this product” and 10 people who came to the session, to the research say “We are not thinking about this, we do not want this”. And here is where the “click” happens for the management.
How often does this happen? That “not there at all” (idea) — not very often. But that the direction and vision changes a little — yes. That you will bring the management to such a session and nothing will happen there and he will tell you “Well, I told you! We didn’t learn anything! Why did we need research?” — this will not happen because there is statistics on startups and features: 1 out of 10 ideas shoots. And the management will see: “Wow! Good thing we didn’t launch it.”
If we manage to pull the boss out to such approaches (here design sprint, design thinking helps well), work out, then maybe you will be able to “switch” his indecision. When does the “switch” happen? It happens when “I thought we would launch this product” and 10 people who came to the session, to the research say “We are not thinking about this, we do not want this”. And here is where the “click” happens for the management.
How often does this happen? That “not there at all” (idea) — not very often. But that the direction and vision changes a little — yes. That you will bring the management to such a session and nothing will happen there and he will tell you “Well, I told you! We didn’t learn anything! Why did we need research?” — this will not happen because there is statistics on startups and features: 1 out of 10 ideas shoots. And the management will see: “Wow! Good thing we didn’t launch it.”
Examples of bad research
a
Usability tests — the paradigm of “Shut up and watch!”. If you don’t even do this (comply), this is a bad usability test.
Interview — probably the main paradigm of “Don’t prompt and hear (not listen, but hear!)!” (Editor: it’s difficult to translate sense from russian). And again, if you prompt or don’t hear, but listen, then you did a bad interview.
Interview — probably the main paradigm of “Don’t prompt and hear (not listen, but hear!)!” (Editor: it’s difficult to translate sense from russian). And again, if you prompt or don’t hear, but listen, then you did a bad interview.
b
You really need to have practice and a good background of knowledge to do research qualitatively. For good research, you need to learn both the math part and practice research every day.
UX research is expensive?
a
What to do if there is not enough money for a UX Researcher? Research with the forces of Product Designer, UX Designer or Analyst? But their capacity is limited. The designer still needs to draw, the Product Designer still needs to manage, and think about the strategy, and the methods of the Analyst are more quantitative. And then there is an option to involve an external resource — an agency. And there the price is yes, it is expensive.
b
And we decided to create our own group of researchers who reduced the costs of an external resource by three times.
Where do UX Researchers come from?
a
Psychologists and sociologists are somehow not taken — they have too “academic” knowledge in research for the product.
b
What is needed for a researcher in terms of skill:
1. Hear (I try to hear your point of view as clearly as possible). And novice researchers first outline this, and then analyze it. Hear and outline it (point of view) without including analysis.
2. Empathy
3. Observability — heard something, saw what the user did, cling, dig, notice it, feel, cling to some small details, to pull something significant out of it.”
1. Hear (I try to hear your point of view as clearly as possible). And novice researchers first outline this, and then analyze it. Hear and outline it (point of view) without including analysis.
2. Empathy
3. Observability — heard something, saw what the user did, cling, dig, notice it, feel, cling to some small details, to pull something significant out of it.”
Should a UX Researcher defend his point of view in front of management or team?
a
A researcher can prove the voice of the customer to the team, or can involve the team in the Research and then there will be no need to prove anything to the team.
You, for example, went yourself, found some hypothesis / pain, put it in the backlog and it will be #1 in the list because you backed it up with interviews. You don’t impose anything on anyone, your task was done quickly and here is a new functionality in the product.
You, for example, went yourself, found some hypothesis / pain, put it in the backlog and it will be #1 in the list because you backed it up with interviews. You don’t impose anything on anyone, your task was done quickly and here is a new functionality in the product.
b
Nikita Lakeev: Yes, you are very right now that you have to think not about your own KPIs (here I conducted 10 researches in a week!), but about the product. How much benefit did I bring to the product specifically? That’s when you’re cool, when you have such a mindset.
P.S. from Mikhail Pravdin
It’s about the team. Somehow a lot goes to the process, to qualitative research, to discovery, “You can’t see people behind the forest” — it’s important to think about who you work with. Solve human problems first, then product ones.
If you are a manager, try to understand if everything is good with your colleagues in the team, if everything is ok in their life context and then they will cope with the tasks, and processes, and qualitative research themselves.
Let’s not forget about human relationships, we work with our colleagues every day for eight hours and they are often closer to us than some people we think are our relatives. Don’t forget about your colleagues.