How to get better customer insights using AI

🤖 This is a Science Says special covering the latest scientific research in AI 🎓 Use it to get better, evidence-based results when using AI in your products and marketing 📈

This insight is brought to you by… Modash

Sending gifts to influencers is a big part of influencer strategy for most B2C brands. It can drive sales, generate authentic content, and build awareness, all at once.

The team at Modash worked with pros like Ieva Rees (Gifting Team Lead, BURGA) and Maris Niilus (prev. Influencer Marketing Exec, Trendhim) to put together a full 101 guide to gifting.

They share the step-by-step process, including what response rates & post rates to expect, and more.

Want to sponsor Science Says? Here’s all you need to know.

🤖 Introducing: AI special series 

Here we are! I’ve been counting down the days after a nice 2-week break, because we are kicking off 2025 with AI month!

This and the next 3 insights will cover some of the latest, most useful research on how we should use AI as marketers. 

I found these studies extremely useful to understand the evidence of what truly works - without all the hype that’s around AI (and, frankly, plenty of BS). I hope you’ll love them as much as I do.

P.S.: This is an experiment, I’d love to understand how useful you find these insights (or not!). If you do, I’ll cover AI-related marketing insights more often. 

Just answer this email to let me know, or use the feedback form at the bottom!

📝 Intro

You’re working on the go-to-market of a new productivity tracking app.

You’ve done some initial research and have a general understanding of who your target audience is.

But now you need to dig deeper, so you can truly understand your prospective customers and maximize the chances of success of your new product.

For example, you want to understand:

• Which pain points (and messaging) resonate most

• Their typical customer journeys

• What other products do they often buy

In the past, this would require months of work.

New scientific research finds that you can now use AI to get these answers faster - and with even better results.

Use AI as a partner to get better market research insights, faster

Topics: AI*
For: B2C. Can be tested for B2B 
Research date: August 2024
Universities: University of Wisconsin-Madison

*New AI category: As more AI studies are getting published and we cover them in Science Says, we’re creating a new AI category on the Science Says Platform.

📈 Recommendation

When doing market research on your customers (e.g. to understand their tastes, how they buy, their pricing preferences), use generative AI (e.g. ChatGPT, Google Gemini) to:

• Help you design the research questions

• Gather existing customer information as well as external information (e.g. competitor messaging)

• Create AI-generated virtual clones of your customers

• Interview and survey your virtual customers with deeper follow-up questions (than what you otherwise usually could with real customers)

You will get better, more useful insights than if you do this alone - or let the AI do the research directly on its own.

🎓 Findings

• Using a combined human and AI approach to collect customer research gives better results than if the research is carried out by humans or AI alone. 

• Researchers took 2 previous market research studies done by a Fortune 500 FMCG company in 2019 and replicated them, but this time using ChatGPT-4. They found that:

  • When researching the celebration of Friendsgiving, AI-simulated respondents gave responses that were more in-depth (0.680 point increase in depth scores - on a 5 point scale), coherent, and informative (0.498 points higher in insight scores) than human ones

  • AI Identified key ideas, and grouped them into themes as well as human analysts did and found new themes that human analysts did not

• However, to achieve the highest quality of data, AI needed human supervision:

  • If data was missing depth, analysts added it by prompting the AI to ask more questions 

  • Data generated was more similar to each other than in typical human responses. Analysts improved that by training the AI with examples of real-life question-answer pairs and giving AI access to a research database from a previous partner study (known as “Few-shot learning” & “Retrieval-augmented generation”)

🧠 Why it works

• AI allows us to:

  • Ask deeper and endless questions that regular customers or survey participants would not have the patience, capability, or time to answer

  • Access AI-generated respondents (known as “synthetic respondents”) that might be difficult or impossible to locate, access, or reach (e.g. 200 CFOs of $1B companies)

  • Take into account more information (and do so at a much quicker pace) than we could do alone or even with a team of researchers.

  • Easily adjust if more information is added at any point, even if the bulk of the research was already completed.

• However, AI needs human guidance to get optimal results (e.g. to point out that data is missing depth or there is too little variation in the responses given) - making co-research is more effective than relying on AI alone.

📘 NEW: Pre-order the Science-based playbook of AI Best Practices

After selling 1,700+ Playbooks my team and I have nailed down the process of creating actionable resources that WORK. Well, this one is going to be EVEN BETTER (I have a very special co-author too!).

On January 29th we are releasing the:

Use it to learn how to implement and use AI in a way that will drive better results.

✋ Limitations

• Generative AI (e.g. ChatGPT) is still a new technology that makes mistakes, fails to understand prompts, misinterprets data, and returns inconsistent results, so any output needs to be carefully monitored and checked for quality and usability.

• Research only analyzed two generic contexts (Friendsgiving and refrigerated pet food). Both are food-specific, broad, and based on Western data, so the specific findings may not be accurate for niche contexts where there’s much less data available online (e.g. attitudes a small town’s residents may have about demolishing a local building to replace it with a mall). 

• Human oversight will likely continue to be needed but may become gradually less necessary as AI improves.

• Studies focused on the English language - there may be variations in accuracy in other languages, but this was not tested.

🏢 Companies using this

• Many or most marketers are already using generative AI models such as ChatGPT or Perplexity to help them in their market research. This is hard to measure, as this type of AI use tends to be unstructured and often hidden from employers.

• There are various tools and services that claim to use AI to help perform market research. For example:

  • Speak turns unstructured audio and video feedback into structured consumer insights through natural language processing

  • Pecan turns imported data sets into actionable predictions (e.g. customer retention rates, demand forecasting, ROI)

  • Quantilope helps to build surveys (or use templates) with a pre-programmed library of questions and methods

  • Evidenza* specializes in surveying AI copies of your customers to inform your marketing strategy

*Fun fact: while we’re not directly related, we share academic advisors (Stefano Puntoni, Co-Director of AI at Wharton) and customers (e.g. Mars, which my team and I help with our on-demand insights service).

⚡ Steps to implement

• Use prompts similar to the ones below (which are based on those used by the researchers) to instruct GenAI (e.g. ChatGPT) to generate an AI clone of your customers to interview. 

  • Give context to set the overall scenario and purpose of the task.

You are a respondent in an in-depth interview. You have been selected with a handful of others across the country to share your thoughts and opinions in this research discussion. I will be guiding you through an online discussion. 

• Give detailed persona information and expectations:

You have been chosen to be a part of this discussion because you previously mentioned you have a new puppy. Your name is Scott. You are a 32-year-old Caucasian Male living in NYC. You are the sole dog owner.

• Give defined role and expectations:

For the remainder of this discussion, we are going to be talking about being a new puppy owner. I would love to understand your opinions and thoughts so please answer all the questions using as much detail as possible.

• You can also instruct GenAI (e.g. ChatGPT) to help identify and simulate a larger sample of your customers.

  • Use AI to generate a potential sample:

We want to understand people’s attitudes and behaviors after getting a puppy. How they feel, what they do, and what thoughts and emotions they experience. What would be a good representative sample of people to talk to, what demographic qualities should we pay attention to?

• Use AI to construct a list of generated personas:

Now generate 50 personas of people who have the characteristics enumerated above. Not every persona needs to have every characteristic. Each persona should be sufficiently unique.

• Or use AI to assist you in moderating discussions (with both - real and synthetic respondents) by generating real-time follow-up questions if responses lack depth

Write a list of 10 qualitative research questions and 10 quantitative survey questions to understand potential customer needs and pain points, features they value most in [Product X's category], willingness to pay and price sensitivity, and how they perceive competitors in the market.

"Using the synthetic personas created earlier, simulate detailed answers to the following question: “'What would make you choose [Product X] over competitors?' Provide a unique response for each persona, reflecting their background and preferences." “For persona 5, please expand and provide more detail on the last part of the response about price being an important factor”

• When possible, train the AI with few-shot learning (i.e. providing some examples or real-life question-answer pairs) and RAG (i.e. importing example datasets from similar studies) to improve the output.

• Once data is collected, use AI to extract themes and generate summaries. Use the summaries as a guide for further human analysis of themes and patterns.

• Always review and analyze the AI's synthetic data to confirm it makes sense, is usable, and reflects actual consumer preferences. Carry out any research or survey modifications that are needed.

🔍 Study type

AI experiments

📖 Research

AI-Human Hybrids for Marketing Research: Leveraging LLMs as Collaborators. Journal of Marketing (August 2024)

🏫 Researchers

• Neeraj Arora, University of Wisconsin-Madison

• Ishita Chakraborty, University of Wisconsin-Madison

• Yohei Nishimura, University of Wisconsin-Madison

Remember: This is a new scientific discovery. In the future it will probably be better understood and could even be proven wrong (that’s how science works). It may also not be generalizable to your situation. If it’s a risky change, always test it on a small scale before rolling it out widely.

Here is how else Science Says can help your marketing:

• 📈 Join the Science Says Platform to unlock all 250+ insights, real-world case studies, and exclusive playbooks

• 📘 Boost your sales and profits with topic-specific Science-based Playbooks (e.g. Pricing, Ecommerce, SaaS)

• 🔬 Get on-demand evidence to make better decisions. My team of PhDs and I regularly help leading brands in FMCG (e.g. Mars), retail, and tech. Reach out here.

🎓 It took 3 of us 16.5 hours to accurately turn this 70 page research paper into this 3min insight.

If you enjoyed it, please share it with a friend. If you’re that friend, you can subscribe below for $0 👇