Every interaction with an existing or potential user is an opportunity to learn about their needs, challenges and how they perceive your product. If your company is using Intercom (or any text-based support tool for that matter) chances are that you already have access to thousands of customer interactions in the form of chat or email transcripts.
A birds-eye-view of what your customer conversations are about can give you invaluable insights into how well your product currently solves user needs, what it’s shortcomings, whether your positioning is effective and the opportunities to extend the value.
How to get a picture of user needs?
A common way to get a feel for user needs is to have an occasional browse through recent support conversations. The problem with this approach is that it is subject to several cognitive biases which can lead to invalid conclusions:
- Confirmation bias: you will naturally tend to favour messages that confirm your pre-existing assumptions and ignore those that challenge them.
- Selection bias: the messages you happen to read during your browse might not represent your user base and lead you to believe that something that applies to one user also applies to the rest even if that’s not the case.
- Recency illusion: humans tend to value events that happened recently more so you might attribute higher weight to recent messages which might not represent your user base.
A better approach that minimises the effect of the above biases is a regular analysis of customer feedback. In this post, I will show you how you can analyse your Intercom messages step-by-step.
STEP 1: Gather customer messages
The first step is to gather your Intercom messages in one place and get them ready for analysis.
Since Intercom does not offer conversation exporting you are left to manually copy & paste messages into a spreadsheet or use Intercom’s API.
To save everyone time and effort I built Prodsight which allows Intercom users to export conversations to CSV format in a few clicks.
Exporting with Prodsight:
1. Visit prodsight.com and tap Connect with Intercom
2. Under Conversation Export section, tap Create an export. It might take a few minutes to generate your export.
3. Once it’s ready, tap Download CSV
The export file is compatible with most spreadsheet processing software.
STEP 2: Filter your dataset
At this point you might have more data than you want to analyse. Ideally you want to end up with around 500 messages so the findings are representative. Some ways of refining the data set:
Filter out admin and bot messages
These types of messages are posted by Intercom’s bot or your support agent. You only want to be analysing messages posted by your users as that’s where the insight lies. This step will reduce your visible row count by at least 50%.
Set bounds to the analysis period
Check the row count. If it’s more than 500, then try experimenting with filtering the “message_created_at” column to only show messages posted in the last week, for example.
- Now you should have a neat list of about 500 user quotes.
STEP 3: Add analysis columns
Now create 3 additional columns for recording your interpretations of the data. I will explain the purpose of each column in the following steps.
STEP 4: Categorise feedback by type
Now you are ready to analyse. The first step is to categorise your messages by feedback type. Most user feedback falls into these broad categories but you might want to modify the list to suit your context:
- New feature request
- Usability issue
- User education issue
- Performance issue
- Generic positive
- Generic negative
- Go through each row and put an appropriate value in the field
- Once you’re done, filter out all rows that contain “Feedback Type” of “Junk”, “Other”, “Generic positive” and “Generic negative” as it is not worthwhile further categorising these rows by theme and code.
You should end up with something like this:
STEP 4: Categorise feedback by theme
Once you have finished categorising your feedback by type, you might want to tag it by theme. Themes will most likely align with your product components/features. For a music streaming service themes could consist of themes such as “Music Library”, “Artist pages”, “Player”, “Playlists”, “Profiles” and “Billing”.
If you have a small amount of feedback and/or a young product, I recommend picking 5–6 broad themes. If you have a large product and/or large amounts of feedback, you might want your theme set to be larger and more specific.
It’s also helpful to have a generic theme such as “Other” to catch things that don’t fit into any pre-defined theme.
- Go through each row and put an appropriate theme that relates to the piece of feedback
STEP 5: Categorise feedback by code
Now that you have done two passes of analysis (by type and by theme) you will have a pretty good idea of the types of things users were asking for. You are ready to code
“A code in qualitative inquiry is most often a word or short phrase that symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a portion of language-based or visual data.” (Saldana, 2008)
Simply put, in this context, a code is a short phrase that specifically describes the meaning of the message.
- Sort table by “Feedback Type” as it’s likely that issues of the same type will have similar codes
- Go through each row and enter an appropriate code name that relates to the piece of feedback
- Re-use previously created codes if a message relates to it
- It’s an iterative process so if you want to update your code on previously categorised messages, feel free to do so.
STEP 6: Summarise findings
You’ve done the hard part — congratulations! Now all is left is to summarise your findings and share with your team.
The most basic way to summarise your feedback is to count up the instances of each feedback type, theme and code.
- Create a new worksheet and name it “Summary”
- Find unique names of feedback types. Position your cursor in the top left corner, paste the below formula and replace ”data!C:C” with the position of the Feedback Type column:
=ArrayFormula(unique(transpose(split(concatenate( data!C:C &”,”),”,”))))
3. Now in the column on the right, count up the instances of each feedback type by pasting the below formula and again replacing ”data!C:C” with the position of the Feedback Type column:
=COUNTIF( data!C:C ,A2)
4. Then, copy the formula to cells below by grabing the corner and dragging it down. You will end up with something like this:
5. Copy the above process for Feedback Theme and Feedback Code columns and you should end up with full summary:
So here you have it, a summary of the 500 most recent Intercom messages. Now you will know what types of issues your users where having recently, which parts of your product or service need attention and what are the specific new features, improvements or bug fixes you should implement to increase the value of your offering.