AI or Authentic Insights April 11, 2024

AI or Authentic Insights

As a market researcher, one of my roles has been to train colleagues and clients in how to generate deep, useful insights. Anyone who has worked in research knows that getting beyond the data and identifying meaning is a critical skill and what sets insight apart from simple information. It involves curiosity, creativity, an understanding of context, and the ability to draw together knowledge and understanding from a range of sources.

The trick to doing this is to understand that consumer insights come from identifying the connection between two or more different pieces of information to explain why people behave the way they do. For example, the classic insight behind Nike's "Just Do It" is that people really want to get out and get fit (piece of information #1), but they also need some motivation to get started (piece of information #2).

To help people understand how to generate insights, I have often used a training exercise where people are given 2 random pieces of information and asked to come up with an explanation of how they relate to each other. It's a fun exercise that drives the point home and gets people thinking creatively to come up with an interesting story.

Which brings us to AI, or more specifically, the use of AI in market research. It's the latest hot topic. It's also something that we have been working on for a while at Edentify. We have developed a range of sophisticated APIs linking to various versions of AI and Large Language Models (LLMs) into our research platform so are deeply invested in this kind of technology. We use LLMs to help identify themes in survey replies, populate code frameworks, theme qualitative transcripts and validate respondent replies.

However, the more we have studied and implemented AI into our workflow, the clearer it has become that, while it is a helpful tool, it is not the whole answer. LLMs are great for understanding and summarising information quickly. They can allow you to get a feel for the main issues and sentiment coming through from large written data sources. They can save an enormous amount of time.

But what AI and LLMs cannot do is make creative connections that build insights, nor can they come close to integrating diverse knowledge and experience like a human.

And AI certainly cannot replace talking to real people: run far away from anyone trying to sell you "synthetic" research respondents.

Instead, the somewhat generic outputs that these AI tools provide are useful starting points. They allow us to streamline the repetitive and time-consuming aspects of the job and get to the real craft of thinking and analysing.

At Edentify we often talk about the power of research technology only being realised when it is combined with real people and real thought. It's the curiosity and creativity of researchers that takes research from information to insight: the ability to look at data, draw on experience, make connections, and think, "Ooh, that's interesting. I wonder why..."

We also wrote about this back in 2018, and while our technology has advanced a lot since then, our conclusions were similar: AI in Research

If you need help or advise on how best to use AI and LLMs then please contact us.

Note: An actual human being from Edentify called David wrote this article but chatGPT was used to come up with the title.

Our latest thinking

May 20th, 2024
NRL's Battle for Members

The Australian football market is one of the most cluttered in the world, with 4 codes – the big guns (NRL/rugby league, AFL) and the challengers (soccer/football, rugby union) all competing for fans.Even beyond this battle for fans is the challenge of building club membership. It's a...

Read more
September 13th, 2022
Why brand goodness is important

In a trend that has emerged over the past few years, more and more brands are jostling to position themselves not just as products and services, but as responsible citizens. At the same time, consumers have become more focused on the impact that the brands they purchase have on the world, whether...

Read more
August 05th, 2022
COVID-19 and the Great Resignation

The COVID-19 pandemic turned many aspects of our lives upside down. One of the biggest disruptions was to how we worked.Frontline workers bore the brunt putting in shift after shift in our busy hospitals, supermarkets and essential services.Those who work in hospitality, retail and tourism among...

Read more
May 25th, 2022
Is the Australian Dream still a reality?

One of the hottest issues of the 2022 election was home ownership. All sides of politics put forward new policies to address calls to make buying a house more achievable.This issue is nothing new: buying a home has been a hot topic on Edentify’s Café Chat for a while now. We asked...

Read more