Big Data, AI, and Market Research: Keeping us Foxy

Written by


Published on

November 7, 2017



Time to read: 3 minutes

A contradiction lies at the heart of modern research: although predicting things remains extremely difficult, modern technology and big data has led to the frequent expectation that it should be easy to solve the most complex of problems.

What increasing amounts of data has done is allow us to better explain what has already happened. While nimble-minded researchers can use this information to guide their forward-looking analysis, we need to bear in mind that there is a great deal of difference between post-rationalisation and prediction.

That’s why the combination of big data, artificial intelligence, and market research is so powerful – it gives us access to the full spectrum of human voices and opinions, which in turn allows us to generate insights that are truly predictive (and not just cherry-picked).

This powerful combo makes us all better ‘foxes’ – but what do I mean by that?

The Fox and the Hedgehog

Nate Silver in The Signal and the Noise identified two types of researchers: foxes and hedgehogs. Foxes take into account all available information, and rationally assemble their arguments. This makes their predictions more powerful and, ultimately, more likely to come true.

Hedgehogs, on the other hand, have pre-existing opinions of what is right and wrong (often expressed in a way designed to gain attention). They follow their gut regardless of the information at their disposal.

In an ideal world, having more data at one’s disposal should make everyone more fox-like. But the actuality of the situation is that more data means it is easier to cherry-pick facts and statistics that fit pre-existing prejudice (ie like the hedgehogs Silver describes). It’s little wonder that experts have become less trusted when the information they use as proof (and the conclusions they draw) rarely reflects the reality of peoples’ lives.

Tapping into reality (and big data) with Signify

So how can we most fruitfully use Big Data to better tap into real lives? That’s the question we sat down to answer when we met with data science company Signify to talk about how we could create a product that integrated their powerful AI offering with our own in-depth experience in qualitative research.

A little context to this meeting:

Before the infamous 2016 American election, Nate Silver was rightly held up as a Delphic figure in the worlds of research and predictive analysis. His website, FiveThirtyEight , was a shining example of a predictive website that worked because it was run by a man who knew what he was talking about. He harvested/ harvests information – aggregating opinion polls and adding demographic data to them.

Up until Trump/Clinton, Silver made powerful conjectures by taking on the mantle of a fox – gathering, assessing and analysing information at his disposal. However, he was very wrong with his predictions of this election. At the time the polls closed he gave Clinton a 71.4% chance of victory.

So what went wrong? In short, Silver discounted the wealth of information available to him. Signify, in contrast to Silver, called all the swing states right. They used a combination of clever, bespoke scripts and publicly available search and social data to find out what millions of voters cared about. By understanding what mattered to them Signify were able to forecast how they would vote.

When we heard about Signify’s success in the American election, we just knew that a partnership had the potential to create a game-changing new product for our research clients…

Artificial Intelligence and Market Research – Making us all better foxes

Together Hook Research and Signify have created ‘Social Intelligence’ – a powerful new process that lets us shed light on: what an audience is saying online, why they are saying it and how it makes them feel.

Distinct from traditional social listening, Social Intelligence ensures that we, as researchers, remain fox-like – combining artificial intelligence and market research to unlock the immense power of the internet and give us access to the full range of human opinion and behavior.

You can learn more about Social Intelligence on our blog, or just drop us an email at We’d love to chat more about this exciting new process (and how it can help power up your next piece of research) over a coffee.


You should probably settle….

It’s Autumn which only means one thing – Married At First Sight (MAFS) UK on E4. Whilst undeniably entertaining, MAFS crystallises a point about dating and life made by Oliver Burkeman in his book Four Thousand Weeks. That point is that people should be more...

Spider-Man: The Ultimate Origin Story

In the vast universe of superheroes, one name stands out as an emblem of relatability, resilience, and redemptive power: Spider-Man. So, it makes sense that he's the cultural phenomenon we have chosen as this months Hook Top Trump! Since 1962, Peter Parker’s everyman...

A new era of Big Brother

When Big Brother launched last century it was totally new and authentic. Contestants had no idea that Big Brother would pull in huge audiences, and acted accordingly: some contestants were comfortingly humdrum feeding chickens and reading books; and some, like Nasty...

Hook’s Iconic Animated Male Leads for Kids

As promised, following last week's non- animated and animated iconic female leads we have the male leads. All of us at Hook Towers have collaboratively put these lists together in response to The Guardian's top 50 kids' shows to binge over the summer. There were so...

Creating Little Happy Learners with Sophie David

Little Happy Learners was founded by Sophie David, an ex primary school teacher and Early Years Foundation Stage specialist. Using her expertise in education, Sophie launched Little Happy Learners in 2019. She now has over 170k followers on her Instagram page, a...

Ninja Turtles: A Timeless Pop Culture Phenomenon

The world of pop culture is riddled with iconic characters, but few have stood the test of time quite like the Ninja Turtles.  And their blend of martial arts, mutant animals, heroism pizza eating and catchphrases.At its core, Teenage Mutant Ninja Turtles has the DNA...

Team Hook’s Favourite Books

September is always a big month for parents and kids alike, because it always means one thing: back to school. There are a lot of things that can be associated with the start of a new school year; for kids it’s the excitement (and sometimes nervousness) of new...

Free Resources to Support Learning at Home

To help welcome children in primary aged groups back to school, we’ve put together ten free resources. These can be used at home to support your child with different areas of learning. Area of learning: Managing stress and worries Resource: @TheMindfulnessTeacher Age:...

Hook’s Iconic Non-Animated Male Leads for Kids

As promised, following our recent post and in response to The Guardian's top 50 kids' shows to binge over the summer... we have the non-animated male leads. A lot of tough choices and some disagreements here at Hook Towers... especially over The Suite Life of Zack and...

More posts in this category

get a digest of our latest thoughts delivered to your inbox


Don't Miss Out!

Each month, Hook's experts create a roundup of hot takes and insights into the Kids and Media industries...for free!

This information will never be shared with third parties