Research collection and analysis when everyone’s remote

Remote user research is an approach to conducting research where the researcher facilitates participants online using technology such as video conferencing, digital surveys, online usability testing software, and digital analysis tools. Researchers are now, more than ever, looking to leverage remote research approaches to ensure they provide meaningful insights to their organizations.

Although there are some limitations of remote research, there are also many benefits. Unlike traditional approaches where a researcher may facilitate research in-person, in a lab, or focus group, conducting research remotely can be significantly faster, more reliable, and scalable. It can reduce the time it takes to get to actionable insights.

Performing research remotely can also make it easier to recruit participants as you’re not constrained by location, in-person scheduling, or equipment. Similarly, remote research enables researchers to observe and collect feedback from participants in their own environment or using their own devices.

We’ve broken down remote research into two parts – the collection phase where you conduct user research digitally using various tools, and the analysis phase, which includes remote collaboration and analysis of the data you’ve collected.

Collection phase

While in-person research is excellent for building a human connection, remotely conducting your research collection has many benefits, including:

• Taking distance out of the equation. Remotely-conducted user studies allow you to connect with subjects anywhere in the world. Because you’re not restricted by location, you have access to a larger pool of participants. This is particularly useful if you have recruiting criteria for niche topics, enabling you to recruit a more diverse sample set to meet this need. Ultimately, this means that your research can be based on a more generalizable sample of people, not just the people that were able to come to the office.

James Vinh, President of the SDXD (San Diego Experience Design) community, indicated:

• Keeping costs down. Because participants use their own equipment in their own environments, remote research lowers the cost of structured interviews and usability testing. It also helps you reduce the amount of time you spend setting up and facilitating in-person tests in the lab.

• Engaging participants who are more at ease online. Face-to-face testing requires researchers to be in the room, and that can prove distracting or pressurizing to subjects and can influence their responses. Remote participants can be more focused and honest in their feedback compared to subjects performing tests in-person. Conducting research remotely can make people share more information, especially when the topic is sensitive. Researchers have historically favored in-person research because it enables them to quickly build rapport with participants and get a sense of non-verbal cues in a conversation. If you’re running remote research, make sure you spend the time creating this rapport virtually and figuring out other ways to gauge non-verbal perceptions, such as ensuring the video is on.

Caitlin McCurrie, Senior Design Researcher at MYOB, chats about how remote research can also accelerate turnaround:

Remotely conducting research can be a blessing to research teams that need a fast, cost-effective, and scalable way to get insights. But of course, every business is unique, and the way you drive your research initiatives could differ vastly from the rest. Luckily, there are a bunch of varied methodologies and technologies you can use to help develop an effective remote research practice in your organization.

Remote user interviews

Interviews are one of the most common remote research methods. Here are a few tips to consider before you get started:

• Interview using video calling software like Zoom or Google Hangouts. Most platforms offer screen-sharing, so you can follow how a participant uses a SaaS product, for example, and ask questions along the way. You could also share your screen and adapt workshop activities to run remotely.

• Make sure you hit record so you can transcribe your interview with a tool like Dovetail, which quickly beats scrubbing back and forth on playable files.

• Remember to store your recorded videos and transcriptions in your research repository to effortlessly find specific insights later on.

Remote usability testing

Remotely facilitating a usability test differs from doing it traditionally because you’re not in the same room as the participant. And regardless of whether the test is conducted in-person or remotely, it can either be moderated or unmoderated.

Moderated sessions

A facilitator is present, allowing interaction with the participant, so both can ask questions or talk through any issues instantly.

The same problems that exist in face-to-face studies can become amplified in remote usability tests. This could be that participants might feel awkward being watched over, and facilitators might struggle to find a balance between letting users know they are there to help and distracting them.

Unmoderated sessions

A participant tests alone, meaning that researchers may need to be more upfront with participants to use the think-aloud protocol and carefully put the questions together because they don’t have the opportunity to ask follow-ups. On the plus side, you’re more likely to get insights quicker as participants can take tests in their own time instead of fitting into a set schedule.

Tools like Lookback (for working products) or Maze (for designs) can take the headache out of remote usability testing by allowing you to share task details with your participants and record their experiences.

Remote diary studies

Remote diary studies are a great way of conducting longitudinal research, especially if you’re after data on behaviors or actions that happen sporadically, or if your research falls under a level of sensitivity that would not work well under direct observation.

Digital diaries can be set up in several tools. Cloud-based file sharing and live collaboration tools like Google Forms and Google Sheets are great to keep information up-to-date and organized. Then there are platforms like Indeemo and dscout that enable diary updates on the go to include videos, photos, and audio to help understand a participant’s context and experience.

Recruiting remote users for your studies

Remotely conducting research gives you access to a larger pool of participants, which could be advantageous when looking for a specific set of requirements. Here are a couple of tools that’ll help you find remote participants:

• User Interviews lets you review profiles and accept or reject participants who have qualified for your research needs. They also handle things like scheduling and incentive payments so you can focus on what’s important – the research.

• UserZoom sources participants through an independent engine, or lets you manage the sourcing through several channels such as QR codes and emails.

Caitlin McCurrie, Senior Design Researcher at MYOB, added the following tools:

• Askable gives you control over pre-screening participants of their extensive panel and run studies directly on their platform.

• If you are running unmoderated research, Prolific, is a great and economic platform to consider. Typically used for academic research, participants are pre-screened extensively, and a minimum wage is set to ensure ethical payment.

Analysis phase

Contrary to popular belief, you don’t need a massive war room or synthesis wall to analyze your research data. In fact, sometimes, it’s easier to analyze your research and produce actionable insights remotely. Some reasons include:

• Collaboration can be digital. If everyone has to be in the same physical location at the same time, you may never be able to get the full team together. Managing the collaborative analysis phase as a remote team means everyone can be involved, speeding up the process.

• Your research repository can be universally digitized. Using a digital-first approach makes it easier to build a research repository that can reach the entire organization regardless of timezones and locations. Doing so will significantly accelerate your organization’s journey to creating a living library of insights.

• You can build relationships directly. Often, research teams hear things second-hand because the in-person research has already taken place, and they only get to see the findings. Remote research allows you to use tech that lets your users speak for themselves, for example, with videos that drive action from your team.

• Improve the quality of your research. Despite how careful we can be as researchers in our analysis, many minds are better than one. By introducing many people into the analysis phase, each can bring their own frame of reference and interpretation of the data.

Digital analysis also reduces the barrier to entry for your colleagues to peer review your tag taxonomy during analysis, helps you build a shared language to describe the research insights, and makes it more accessible to get everyone involved.

Turn silos into teams

Lastly, when physical barriers are removed, departments that were once siloed can collaborate with greater ease online.

Dovetail’s Product Designer, Lucy Denton, reflected on a few methods she used in her previous role at Atlassian for analyzing and synthesizing research data as a team:

• Use digital affinity mapping to organize and cluster ideas and to ideate on UX strategy and vision to understand problems and solutions. There are various tools on the market that allow for the digitization of this process, such as Miro, an online collaborative whiteboard.

• Source opinions and advice from subject matter experts that aren’t always privy to the analysis stage. By bringing design, engineering, marketing, and sales together into one cross-functional research team, you’ll get better outcomes.

While remote research changes things, it’s not all bad news. Being remote has its upsides, and with more online collection and analysis tools than ever before, it’s never been a better time to be conducting your research remotely.

Written by Caitlin McCurrie, Ph.D., Lead Researcher, Atlassian and James Vinh, President, San Diego Experience Design Professionals.