Understanding one’s audience is discussed with increasing importance for brands seeking to capture the possibilities of digital transformation.
Yet, predicting human behavior can be beguiling, even to scientists, and designing compelling user experiences can be somewhat likened to the experience of preparing a meal for a food critic: a deep understanding of the guest’s preferences, likes and dislikes — Are those green peas on my plate? No, thank you! — can have an important impact upon the ability to meet, and ultimately exceed, user perception of your brand and site.
In this post, we will be discussing our use case for conducting qualitative research as an important method for gathering insights and a nuanced understanding of digital users’ behaviors and expectations. Such research has been an important way our design teams establish context and insight to make decisions about thoughtful user touch points and introducing new functionalities. Likewise, we hope our experience can help other brands and start-ups apply similar research strategies to successfully create powerful, relevant digital experiences for their users.
Designing the Study
Bonnie Bohan, customer experience designer, and her partnering UX researcher, Adam Young, PhD, led the study, which primarily sought to attain insights to what users really expect and want out of Acquia’s developer products. It began as an open-ended research initiative, and while the team did not initially plan to specifically collect data on the website experience, once the study began, user feedback indicated this was a primary starting point for most users. This provided Bohan with an important clue for further enquiry very early on in the study: most users will go first to the website, which becomes the primary resource from which they learn about the products. “We interviewed a group of participants who were familiar with Acquia products and those of our competitors, as well as a second group who was not familiar with Acquia at all. When the second group searched our company on Google to learn about our products, the first place they arrived was our company website.”
The feedback the team received was compelling in understanding users’ views of the website and how it can be further developed. To delve into these findings further, Bohan spun off a separate research project, and teamed up with the digital marketing team to prioritize further optimizing the website. When she kicked off the project, Bohan decided a workshop was the best way to bring together members of the UX team with representatives from each of the facets of marketing, to contribute varying perspectives. Together, they mapped out a strategy for identifying the target users of the site and what the main goals of this research study would be.
A research study that involves multidisciplinary groups can help produce broad insights for the organization, and Bohan underscores the importance of ensuring all perspectives are considered. “When designing the study, we realized it was important to bring members from each team together because, based on one’s role and responsibilities, we all bring different views on what we consider to be the most important learning objectives. With this approach, we were able to put all ideas on the table and decide collectively on which goals would be the focus for this study,” said Bohan.
The workshop enabled each member to talk through the answers they wanted to ultimately retain — the “who, what and how” of the user experience:
- Who are we trying to attract to our website?
- What are they trying to find or do?
- How are they are finding this information?
They discussed in-depth: who are the different individuals who come to the website, and what was drawing them there? With multicolored post-it notes all over the wall, they voted on who they determined to be the most important website users, removing outliers who are not part of the target audience.
Narrowing this down, they were then able to vote on the top ideas for what was most important to users when visiting the site. Through this categorization, Bohan was ready to begin planning the user interviews and recruiting participants.
She describes a focused recruiting process to ensure the number of participants reflected the proportion of users who came to the website to ensure an unbiased sample. This included an equal number of developers to marketers, as well as individual contributors to managers, in order to gain a sense of how different responsibilities and hierarchical levels may play a role in the evaluation process. Bohan then structured the interviews by dividing each session between user interview and user test.
Bohan described some of the probing questions she asked as a deep dive into capturing a broad set of data. “The first part of the interview, I asked users questions about their company, its size, industry, and about their team structure, individual responsibilities, and typical process for evaluating a prospective software product. Additionally, we wanted to know how the decision-making process looked for each individual within their organization.”
Synthesizing the Data
Once the data was gathered, Bohan began to work out strategies for how to approach one of the most challenging aspects of qualitative research: if it is human nature to want to quantify information, then how could the team begin to parse through the notes, and connect data points when a certain number of respondents report one piece of feedback, and another sets reports a different experience?
Bohan advises with a small sample size, one must generally avoid quantifying such results, as this line of enquiry is best suited for larger scale studies.
In this study, to accomplish data synthesis, she invited observers from different departments to take notes while she was interviewing participants, which she noted helped greatly in organizing notes from the fifteen half-hour interviews of information collected. “Having good notes and direct user quotes was extremely helpful to our data collection process. Once we began the review process, after about the ninth interview, we began to see a few commonalities and themes emerge throughout our notes.”
Bohan then created a conceptual structure of categories from the research: design improvements; content improvements; user flows; and contact methods. For each one that was related, she captured the most constructive criticism and feedback to take and build from.
Some of the responses we received included feedback such as users finding a certain form too long and unlikely to submit it; users having difficulty reading ( x ) because the text is too small; and, as a larger theme, the request for video communication as a preferred media for receiving information.
Bohan described the process of spotlighting this detail because they could potentially be easily fixed, while also forming recommendations to address broader expectations that was observed in the data. The data was then presented to the cross-functional team in a presentation deck to review high-level findings, and a comprehensive research report was also prepared so others could read through and discuss among their teams. During the presentation, Bohan created short video samples, of 20 to 60 seconds, showing actual users expressing the obstacles they experienced on the site, which helps to create a more personal account of the data.
One of the most important findings our team discovered throughout this research is the usefulness of a qualitative study to capture a more nuanced view of users’ interests, motivations, and behaviors. Bohan was able to attain specific insights and use direct user quotes to support their conclusions and recommendations for areas our design team should designate as a priority for how to further develop certain functionality and usability of the site.