Big Data, AI, and Market Research: Keeping us Foxy

Written by


Published on

November 7, 2017



Time to read: 3 minutes

A contradiction lies at the heart of modern research: although predicting things remains extremely difficult, modern technology and big data has led to the frequent expectation that it should be easy to solve the most complex of problems.

What increasing amounts of data has done is allow us to better explain what has already happened. While nimble-minded researchers can use this information to guide their forward-looking analysis, we need to bear in mind that there is a great deal of difference between post-rationalisation and prediction.

That’s why the combination of big data, artificial intelligence, and market research is so powerful – it gives us access to the full spectrum of human voices and opinions, which in turn allows us to generate insights that are truly predictive (and not just cherry-picked).

This powerful combo makes us all better ‘foxes’ – but what do I mean by that?

The Fox and the Hedgehog

Nate Silver in The Signal and the Noise identified two types of researchers: foxes and hedgehogs. Foxes take into account all available information, and rationally assemble their arguments. This makes their predictions more powerful and, ultimately, more likely to come true.

Hedgehogs, on the other hand, have pre-existing opinions of what is right and wrong (often expressed in a way designed to gain attention). They follow their gut regardless of the information at their disposal.

In an ideal world, having more data at one’s disposal should make everyone more fox-like. But the actuality of the situation is that more data means it is easier to cherry-pick facts and statistics that fit pre-existing prejudice (ie like the hedgehogs Silver describes). It’s little wonder that experts have become less trusted when the information they use as proof (and the conclusions they draw) rarely reflects the reality of peoples’ lives.

Tapping into reality (and big data) with Signify

So how can we most fruitfully use Big Data to better tap into real lives? That’s the question we sat down to answer when we met with data science company Signify to talk about how we could create a product that integrated their powerful AI offering with our own in-depth experience in qualitative research.

A little context to this meeting:

Before the infamous 2016 American election, Nate Silver was rightly held up as a Delphic figure in the worlds of research and predictive analysis. His website, FiveThirtyEight , was a shining example of a predictive website that worked because it was run by a man who knew what he was talking about. He harvested/ harvests information – aggregating opinion polls and adding demographic data to them.

Up until Trump/Clinton, Silver made powerful conjectures by taking on the mantle of a fox – gathering, assessing and analysing information at his disposal. However, he was very wrong with his predictions of this election. At the time the polls closed he gave Clinton a 71.4% chance of victory.

So what went wrong? In short, Silver discounted the wealth of information available to him. Signify, in contrast to Silver, called all the swing states right. They used a combination of clever, bespoke scripts and publicly available search and social data to find out what millions of voters cared about. By understanding what mattered to them Signify were able to forecast how they would vote.

When we heard about Signify’s success in the American election, we just knew that a partnership had the potential to create a game-changing new product for our research clients…

Artificial Intelligence and Market Research – Making us all better foxes

Together Hook Research and Signify have created ‘Social Intelligence’ – a powerful new process that lets us shed light on: what an audience is saying online, why they are saying it and how it makes them feel.

Distinct from traditional social listening, Social Intelligence ensures that we, as researchers, remain fox-like – combining artificial intelligence and market research to unlock the immense power of the internet and give us access to the full range of human opinion and behavior.

You can learn more about Social Intelligence on our blog, or just drop us an email at We’d love to chat more about this exciting new process (and how it can help power up your next piece of research) over a coffee.


Kung Fu Panda: The Ultimate Heros Journey

It’s been sixteen years since Kung Fu Panda initially graced our screens. Three films and three series have already been released, and with another film due out in March, it’s showing no signs of slowing down. Sixteen years of a franchise is impressive, especially...

Chicken Run: Dawn Of The Nugget

Three take-outs from Hook It’s been 23 years since Aardman released its first feature film Chicken Run. It remains the highest grossing stop-motion film of all time. The original Chicken Run mimicked The Great Escape, with a flock of chickens led by Ginger (Julia...

Trailblazing Trolls: Their Journey to 2023

DreamWorks Animations recently launched a third film in the Trolls franchise Trolls Band Together (2023)! Today’s star-studded Trolls have come a long way from their humble beginnings in 1950s Denmark. Which begs the question, how have Trolls managed to evolve and...

You should probably settle….

It’s Autumn which only means one thing – Married At First Sight (MAFS) UK on E4. Whilst undeniably entertaining, MAFS crystallises a point about dating and life made by Oliver Burkeman in his book Four Thousand Weeks. That point is that people should be more...

Spider-Man: The Ultimate Origin Story

In the vast universe of superheroes, one name stands out as an emblem of relatability, resilience, and redemptive power: Spider-Man. So, it makes sense that he's the cultural phenomenon we have chosen as this months Hook Top Trump! Since 1962, Peter Parker’s everyman...

A new era of Big Brother

When Big Brother launched last century it was totally new and authentic. Contestants had no idea that Big Brother would pull in huge audiences, and acted accordingly: some contestants were comfortingly humdrum feeding chickens and reading books; and some, like Nasty...

More posts in this category

get a digest of our latest thoughts delivered to your inbox

Don't Miss Out!
Each month, Hook's experts create a roundup of hot takes and insights into the Kids and Media industries...for free!
This information will never be shared with third parties