There is no shortage of guides published about how to build a design system. However, many of these guides do not pay enough attention to a crucial step in building a design system: user research. I’ll discuss why including user research in your design system process is an absolute must. I’ll also discuss one of my favorite methods for design system user research: the field study. We’ll take a leaf out of our elementary school curriculum and cover the “who, what, when, where, and why” of design system field studies. Plus, I’ll include some specific research questions to investigate when doing your field studies.
NEW RESEARCH: LEARN HOW DECISION-MAKERS ARE PRIORITIZING DIGITAL INITIATIVES IN 2024.
Why Companies Skip Design System Research
The usual reasons for skipping user research still apply to design systems — not enough time, not enough budget, can’t convince leadership, etc. However, design systems introduce some new reasons why people skip research.
Because design systems are internal products built for internal employees, teams discount the importance of user research to the system’s success. Essentially they think, “I work with these people every day — I know what they need and what problems they’re having.” In a worst case scenario, a team might not even think of their colleagues as people with problems and needs. Instead, they relegate them to cogs in a machine, with a job to do and a new tool they should be using.
First of all, design systems are inclusive tools — they serve more than just designers and developers. They’re a resource for marketing, copy writers, and even legal teams. So, unless you’re an expert in all of those things, slow your roll on assuming you know the exact pain points a design system can solve for your company’s employees. In fact, even if you are a genius lawyer-designer-author-developer hybrid-cyborg, you still haven’t lived the lives of the people you’re designing for. Each person will have different experiences, attitudes, expectations, skills, and abilities that will affect how they experience your design system. For example, maybe Jane Doe refers to a dropdown as a “select box” because that’s the verbiage her last company used. Have fun telling Jane she’ll never find a select-box in your design system because you never bothered to chat with her for a few minutes.
Your design system is only valuable if people use it. Imagine if Tim Berners-Lee launched the world wide web and hid it away in his basement. Without buy-in, your design system is nothing. Including multiple teams, partners, and stakeholders in your design process garners more buy-in throughout your organization. Think about it — which tool are you more likely to use?
- A tool that you actively helped create;
- A tool that a co-worker built for you, but it’s annoyingly useless because your co-worker never bothered to ask how you work;
The second approach is a fast track to making people feel invisible. The people you leave out will not only avoid your system, but they will become active detractors and undermine it.
The other main reason people skip design system research is because people still have a general misconception about what a design system is. Many think that a design system is just a library of reusable UI components. But it’s actually much more than that. A design system is more about the people who use it and how they interact than the components inside it. It must establish a governance process and design principles, among other things. However, because people think of a design system as just a component library, they think they can simply copy another public system. They look at material design or whatever the hottest new design system is and view it as a one-size fits-all solution.
However, your company is probably more unique than you think it is. Yes — there is probably a lot of overlap between each design system’s components. Every product is probably going to have buttons somewhere. But when it comes to things like the system’s governance process, different approaches are going to work better at different companies or for different products. Without doing user research, you won’t be able to make an informed decision about how to structure your governance process. For example, you’ll want to understand your team’s general outlook regarding design systems. If they feel that design systems tend to limit their creativity, then your governance process should be more inclusive and less authoritative.
The design system team’s structure is also an important part of the governance process. It defines who has the authority to make changes to the system. The team structure you decide to use will depend on several factors:
- How much time can people devote to the design system?
- How do people feel about being fully or partially dedicated to building and maintaining the design system? Are they interested in such a project?
- How well do your team member’s skills align to design system responsibilities?
- How much exposure does each team member have to different products throughout your organization?
Guess what? Unless you do user research, you can bet you won’t have accurate answers to the above questions.
So, now that we’ve established you’ll be doing user research for your design system (we have established that right? Nod your heads), let’s talk about how to actually do it. First, let’s talk about two different types of user research: generative research and evaluative research.
Generative research focuses on finding answers to your “known unknowns” as well as your “unknown unknowns.” In other words, through generative research you can answer your existing research questions, but you can also discover new insights and questions you didn’t even know you should be looking for.
Next we have evaluative research, which is a bit more straightforward. With evaluative research, your goal is to evaluate something (shocker). That something could be how well your solution solves the problems you set out to solve. Or, it could be evaluating the validity of a hypothesis or risky assumption. Most importantly, good evaluative research highlights why the solution is performing well or not.
So how does this apply to design systems? Your first instinct might be to follow a linear approach: do generative research first, then build the design system, then evaluate it. That approach would still be more valuable than doing no research at all, but it’s not the most optimal approach. To get the most out of your research, you want to do it continuously throughout the entire product development lifecycle. Everytime you release a new version of your design system or start to conceive a new feature, you should do user research.
Methods
There are a lot of user research methods out there that can be applied to your design system:
- Interviews
- Collaborative journey line exercises
- Field studies/contextual inquiry
- Participatory design
- Card sorting and tree testing
And the list goes on. For a great rundown of which methods to use when, I repeatedly refer back to this Neilsen Norman article. But today, I want to focus on one method in particular that works well with design systems user research: field studies.
What is a Field Study?
Think of a field study like your typical research interview, except it’s performed in context. Rather than asking someone to recount how they did something, you actually observe them perform the task in their natural environment and ask questions when necessary. The technical term for this method is “contextual inquiry.” However, I tend to use the term field study because it’s more relatable for participants and others.
Why Field Studies?
Memory is notoriously fallible. If you asked me to recount the exact steps I took to make my breakfast this morning, I wouldn’t be able to accurately tell you. I would leave steps out, recount them in a different order, or even add steps that never happened. This is the drawback of your traditional research interview: what people say they do is almost always different from what they actually do. With a field study, you see the action firsthand. You uncover much richer and more accurate data.
Field studies are my preferred method when doing design system research because users are completing very complex tasks. For example, when I design something, I’m making key decisions every few minutes. Meanwhile, the engineering side is so complex that I can’t even begin to fully understand it (another reason why user research is so important). The more complex the task is, the more likely that a participant will leave out details in a typical interview.
There are drawbacks to field studies as well. For example, you have to get the timing right. Field studies are only valuable if you observe your users in their natural environment, performing tasks as they would usually perform them. This means that your participants need to actually have the tasks you want to observe on their to do list. Artificially recreating a task will introduce issues, because then you move into the realm of what the user “typically” does, instead of what they actually do.
A field study can be more time consuming than your typical interview because you have to observe participants as they fully complete their tasks. Often, completing a task takes longer than explaining how to complete it. Similarly, because you collect richer and more data, the analysis process can take more time.
Lastly, it’s often more difficult to find participants willing to let you shadow them for an extended period of time. Many participants perceive an interview as lower effort than doing a field study. That’s because in a field study we ask the participant to take the lead. They explain everything they’re doing to the moderator, which requires thought and effort. However, unlike an interview, the participant completes their work tasks and remains productive. Be sure to stress this point when recruiting co-workers for field studies.
These challenges may seem tough to overcome, but they are minimized when working on an internal tool like a design system. For one, you’ll have more luck recruiting co-workers than consumers. You’re likely to have an existing relationship with many of your co-workers. Your co-workers also will have more motivation to participate. A consumer’s motivation is usually to receive compensation or to improve the tool they use. For your co-workers, it may actually be within their job duties to help you out.
Who to Research With
We’ve already covered the different teams that a design system can impact. But who within those teams should you observe? Not all engineers are the same and not all designers are the same. You should consider having sessions with senior engineers and junior engineers. Engineers with React skills and engineers with Angular skills. Designers who code and designers who don’t. There are a ton of ways to slice it. The point here is that, right off the bat, you might not know who exactly you should research with or how many people to do sessions with. There’s no hard and fast rule for this. As you start your research, you will start to identify different skills, goals, and personalities for each persona. The rigorous researcher side of me will tell you to continue researching until you hit saturation. In other words, research until you stop hearing new things and discovering new types of people.
Also, a word of caution here. Some design system teams structure their teams using a federated model. In this model, different “representatives” from your various product teams also contribute to the design system. In this case, you might be tempted to rely heavily on these representatives to give you insights on what they need from a design system or how well the design system is working. After all, they’ve set aside the time to specifically work on the design system. However, we need to cast a wider net than just these representatives. You’re looking for different perspectives from people across your company. Plus, the people who are on your design system team are most likely biased. Why? Well, they’re on the design system team, so they have inside knowledge and probably an affinity for the design system that others might not have.
Where to Do a Field Study
This might seem obvious, but you should perform your field studies in the field, or in other words, within the user’s natural environment. This means that if your designers typically work from the office at their desk, you should hold your session there. If your participants do any face-to-face collaboration with others during their work day, try to attend the session in person.
If you have a session with an engineer who is 100% remote, you can try doing the session remotely over screen share. Doing these sessions remotely is possible, but there are some technical challenges. For example, your colleagues might use multiple monitors and most screen share tools only share one monitor at a time. So you run the risk of missing important actions the participant might take, like sending a message to someone on another monitor. In this case, it can be helpful to have the user talk through all the steps they are taking to ensure nothing is missed.
How to Do a Field Study
Your first time running a field study might feel a little foregin, because it’s very different than most common UX research methods. Remember, it’s also probably unfamiliar for your participants. That’s why your first step when doing a field study is to make sure your participant is aware of what to expect. Send them an email ahead of time making sure they will have tasks ready to perform. Make it clear that you’re not doing a typical interview. Also make sure they know the session is an individual session, unless they always perform the tasks in question with other people. You’d be surprised how many people “bring a friend.” For design systems, this is a key point, because often designers and developers will collaborate for long periods of time. A design system can be a key resource during those collaboration sessions.
Before you go into the session, I also recommend creating an intro script to follow. This script can be as fleshed out as you need it to be. Personally, I write out every single word that I say to make sure everyone gets the exact same instructions. In this script, I explain what we’ll be doing during the session. There are a few key points to focus on here:
- Tell the participant it’s important to do their work as they naturally would.
- Assure the participants that they can be honest. Nothing they say or do will be held against them.
- Explain that the session will be different from a typical interview. Walk the participant through how the session will work.
On this last item, there are two different ways you can approach the session. In a more structured session, you will know exactly which tasks you would like to observe and should inform the participant beforehand to have those tasks ready. In a less structured session, you’ll start the session with a brief interview to understand the design system related tasks the participant will perform that day. Once you’ve created a list of tasks, ask your participant to complete them in a natural order. When it comes time for the participant to start their work, I make it clear that we are making a dramatic shift in how the session is run. I say something like:
“Okay, now we’re going to change gears and do something completely different. Up until now, I’ve been doing most of the talking and asking you questions. Now, I’m going to turn it over to you and ask you to walk me through your work tasks. I’d like you to think of this session as if you are a trainer and I’m your apprentice. Assume that I know nothing about the work you do, and you’re trying to teach me how you do it. This means don’t skip any of the details, even if you feel I might think they’re boring. I want to see absolutely everything.”
Even with this introduction, your participants will likely try to gloss over the “boring” details. In such cases, gently remind them you would like to see everything. In the case of a design system, the boring, repetitive tasks are actually the most important to observe. After all, a design system is meant to make design and development more efficient.
Tasks and Research Questions
As mentioned, the tasks that people at your company perform will be unique in certain ways. That being said, below I’ve included a set of tasks you can explore with your team members and research questions for each task. The questions you investigate might be slightly different depending on whether you have launched your design system or not.
Collaboration between Team Members
Design systems are meant to streamline collaboration between your team members by limiting disagreements about how components work and building a shared terminology. Here are some things to look out for when watching teams collaborate:
Design Concepting & Prototyping
Design systems play a big role in making prototyping more consistent and efficient. Similarly, they can introduce consistency into very early stages of the design process as well. Investigate these research questions to make sure your designers get the most out of your design system when using it to concept and prototype:
Design Review or Critique
Hopefully your design team performs regular design reviews to gather actionable feedback. Try to answer the following research questions when observing this task:
Translating Designs into Front-End UI
If your team has designers and developers, there’s a good chance that at some point your developers will reference a design artefact, such as a prototype or mockup, to begin building part of the product.
Updating Existing UI
One of a design system’s biggest benefits is that it makes changing your existing front-end UI much faster. Explore these research questions to understand how you can help your teams achieve this goal:
User Research and Analytics
This is going to get a bit meta, but just as you should perform user research for your design system, your product teams should also perform user research on their products. More broadly, your product teams should inform their decisions with data, including product analytics, to reduce risk.
Most people won’t think of a design system as affecting user research and analytics all that much. However, the design principles you evangelize through your design system can transform your entire design and development process. For example, if your company struggles to do user research but wants to improve, you might include a design principle like “Test early and often.” You could also include best practices for how to use data in the design process. That’s not something you typically see in a design system, but that’s the point — design system user research allows you to make your system your own.
With that in mind, here are some good research questions to explore when observing people perform user research.
Evolving the Design System
A design system is never done. Over time, your system should evolve to include new components, principles, and content. You should improve your processes over time. A good design system provides guidance on how this evolution should occur. Making changes to the system is one of the most complex challenges you will face, so here are some research questions to help you fully understand the challenge.
Wrapping It Up
In summary, performing user research for your design system is important because your organization has unique needs. Your organization employs people with different perspectives, skills, and experience. Tapping into this rich data can be challenging, but it will result in greater buy-in and engagement. With this article, you can arm yourself with some great questions to research at each step of the process. If you give it a shot, let us know how it goes in the comments.
