Analyze User Research with AI in 2024

Analyze User Research with AI in 2024

This is part of a series called Navigating Ambiguity, designed to provide guidance on creating clarity from chaos when designing, inventing, and improving products or services. If you’re a product manager, strategist, user researcher, indie hacker, or basically anyone that builds, this is for you. Subscribe to get notified when the next article drops.

Hooray! You’ve conducted your user interviews and now you’re faced with the gigantic task of analyzing all of your conversations to extract themes and the nuggets of gold.

Before you get your coffee ready and prep yourself to sift through hours of interviews at 2x speed (we know you do it), consider using the latest tools to help you save time, de-bias, dig deeper, and double check.

AI is completely changing how we understand people. It’s why InsightLab believes that one day, machines will understand people better than people can understand people. The scale and depth of insight that AI enables is unprecedented.

New AI tools are emerging every day and it can feel a little overwhelming and difficult to find the signal through the noise. We’ve done the heavy lifting to help you get a clear picture of the market.

The Transcribers:

These are tools that will take audio / video recordings and convert them into transcripts. These tools are fairly common, but the latest tools allow you to add specific words or phrases to increase accuracy when used in specific domains. For example, if you’re interviewing doctors, you might want to add specific medical terminology to ensure it doesn’t mistake the word “neuroplasticity” with two words “neuro”, and “plastic”.

Otter.ai - Transcribes conversations with high accuracy but often struggles with heavy accents or background noise, making it less reliable for diverse research environments.

Fireflies.ai - Transcribes Conversations and is a little more affordable than Otter, but its user interface is not as intuitive, leading to a steeper learning curve.

InsightLab - Transcribes conversations with high accuracy, you can add custom vocabulary for domain specific terminology, and you can differentiate between up to 10 different participants - this is great for focus groups and keeping track of participants in large groups.

The Summarizers:

These are tools that help you quickly condense the amount of data you need to sift through by summarizing large transcripts.

Notta: Notta is great for note taking, but it’s mostly tailored towards summarizing meetings, which means it’s querying abilities are limited.

ChatGPT: This is ultimately the king of summarizing. However, the workflow really isn’t optimized for handling a lot of files at once, which unfortunately means that you’ll be uploading one transcript at a time to get AI summaries.

InsightLab: InsightLab can help you get a high level overview of all or even part of your conversations. Use the semantic querying tool and split your data by demographic to dig in. However, the team firmly believes that summaries are not insights. As a result, InsightLab is focused on empowering you and your team to get curious and extract insights from your data more effectively.

The Hallucinators:

Synthetic Users: Claims to enable you to conduct user research without the user. The theory is that by talking to a simulation of a human, you can unearth valuable insights without the cost and time investment required to speak to real humans. We know that true insights emerge from discovering the outliers, the edge cases, and the fringe as well as the average. Because the digital persona can only answer questions based on what it’s been trained on, which is ultimately an aggregated average response, the insights you get are extremely limited and unreliable. You also can’t yet ask any questions related to recent events, cultural shifts, or experiential prototypes. InsightLab on the other hand believes real insights require talking to real humans.

The Analyzers:

These are tools that help you dive deep and extract key themes from your data.

NVivo - Powerful qualitative analysis software, but very complex and difficult to use. The interface is quite clunky and feels quite outdated.

InsightLab - Powerful tools for qualitative analysis to help you extract themes, quotes, and more. The semantic similarity search allows you to get answers to questions quickly with references to the original data, and the software auto-generates audio/video clip references. Use the Kanban board to extract and organize themes. InsightLab also allows you to query any subset of your data, so you can split by demographic or any other tag you might choose.

The interviewers:

InsightLab - On top of allowing you to analyze data very effectively, InsightLab enables you to run AI powered interviews. You simply create your discussion guide using the study-builder and deploy just like a survey. The AI interviewer will ask every key question in the discussion guide, and then ask any relevant follow up questions, live at the time of study to dig deeper. In this way, each participant has a unique conversation. The results speak for themselves, clients have used InsightLab to run hundreds of interviews at scale. The interview link can be embedded into any app, and viewed on desktop/mobile. Participants can respond with text or audio.

We hope this guide helps you navigate the latest and greatest AI tools. We don’t believe AI is going to replace humans anytime soon, but leveraging this technology can help you get to where you need faster, and without compromising quality or depth of insight.