Apr 24, 2012

Head in the Cloud: How Big Data Misses the Point.

Meta Data

My argument here will be that "Big Data" leads us to overlook important realities. Although crunchy numbers are an important part of any information diet, Big Data approaches can lead to deficiencies in other essential sources of information. By "Big Data" I mean consumer experience and marketing insight approaches based on data mining and statistical algorithms.

Taken out of Context

When I grew up in Calgary, Alberta, Canada, a long time ago, it was pretty much common knowledge that Asian people couldn't drive. If a car turned without signaling, someone would surely remark, "probably a Chinese driver".

There was something about being Chinese and poor driving. Later, I lived in Singapore, and when I went to the swimming pool, I was surprised to find that Chinese people swam the way they drived. At the pools I was used to swimming at, in Australia and Canada and New York, people swam in lanes. The lanes were marked by ropes and people swam up one side, down the other. It's a good system. There are fast lanes, and slow lanes, and a lot of people can swim laps in a pool without bumping into one another.
In Singapore, the swimming pool looked to me like chaos. Everyone swimming in every direction. And this was adult swim time, all adults, mostly with speedos, caps and goggles, people were stretching on the deck. We were all there to exercise, not a splash in the bath. It was very difficult to swim from one end of the pool to another without bumping into anyone. The only lane rope in the pool was extended across the width of the pool at the end, marking off the end of the pool for old people doing water exercises. Helpfully, I explained the benefits of ropes and lanes to the pool staff, but to no effect. They didn't seem to feel the need.  It wasn't just the swimming pool. It was the sidewalks, the markets, the traffic. It all seemed so chaotic! Was this truly the stuff of an ancient civilization?

 And then, I had kind of revelation. As part of a research project I was involved in, involving interactive television and isolated Australian Desert Indigenous communities, I learned about differences between "high" and "low" context cultures. High context cultures assume everyone shares a high level of common understanding, so that much of communication can be left unsaid. Low context cultures assume little shared understanding, so communication must be self-contained and explicit.

 Asian cultures tend to be high context, Western Cultures tend to be low context. For example, in England, we might say "would you like some tea now?" but in Japan, it's more likely someone would simply lift the teapot meaningfully. In high context cultures, the meaning of a symbol or a gesture is more dependent on circumstances, relationships with other things. For example, while words in western cultures tend to have relatively fixed meanings or sounds, the meanings and sounds of characters in Asian cultures depend much more on the surrounding characters.

 This insight into differences in sensitivity to context in different cultures explained a few things for me. For example, Tai Chi. What was the point of a stork pose? When I looked a bit deeper, I could see differences in traditional Asian and Western sports in terms of High and Low context cultures. Tai Chi is about perfection in relation to the context of an ideal movement and form. Western sports are about absolute, context-independent objectives: Faster, Higher, Stronger.

We talk about Western sports in terms of records and statistics. Western sports lend themselves more readily to numbers. Numbers are the ultimate low-context communication. A three is a three is a three in any circumstance, at any time. When we can express things in terms of numbers, we have boiled away all context. Numbers are difficult to misunderstand. This is the great virtue of numbers.

 Large corporations tend to develop low context cultures. Multi-national organisations need to communicate in ways that will be understood consistently by different groups in different regions with different responsibilities and circumstances. Large corporations also need to be understood by financial markets, which understand only numbers. In large corporations, numbers talk, and they speak in algorithms. Large corporations have a natural tendency to reduce life to formulas. Corporate culture is a little autistic like that.
European Member of Parliament Francisco Sosa Wagner (non-aligned, Spain)
Humor is high context. To appreciate the funniest situations, you have to "be there". It often doesn't translate well. Like irony and love (…and coffee), it is an essential element of life. This is one explanation why the European Parliament has failed to excite the public. Translations boil out the humour and passion, leaving only the low context information. This is why there is so little humour and so many numbers in corporate communications.

And now I see how serious and silly I must have seemed in the Singapore Toa Payoh swimming pool. I was focussed on lap counts and times, while everyone around me was pursuing other aspects of swimming fitness less focussed on statistics, ways of swimming that allowed for navigating around fellow swimmers. They don't need lanes because being aware of what's going on around them is all part of the experience. It was quite uneducated of me to swim so aggressively. Just where in the pool did I imagine I was going in such a hurry?

 I'm riding on a busy expressway with another expat. He signals to turn, but no one opens seems to slow to let him in. Finally, out of frustration, he steers the car into a very tight opening. He's probably cursing the ignorant expats. Only a Westerner would put so much faith on such a simplistic and one-dimensional – low context – device as a turn indicator. Traffic – and much else – flows not so much because all drivers adhere to a common set of rules, as because every driver is responsive to the behaviour of the vehicles around them. An intention to turn is as likely to be signalled through a gradual drift into the next lane, as an explicit signal, but usually a combination of circumstances will lead other drivers to anticipate the move. Good taxi drivers are able to weave through by being very responsive to circumstances. The thing about low context approaches is they are so self-contained that they can make you oblivious to context.

Blinded by Science

We can think of Technology in terms of creating low-context solutions; i.e., solutions that do a job consistently and predictably, independent of external circumstances. For example, a hammer is designed to lead even an inexperienced carpenter to drive the nail efficiently; a map will show you the way regardless of you whether you have visited a place before; A mathematical formula will lead you to a logical conclusion regardless of how well you know the particular subject matter.

 But the thing about low context approaches is they are so self-contained, so independent of circumstances,  that they can make us oblivious to context. As Mark Twain said, to the man with a hammer, everything is a nail. Technology fosters a kind of blindness.
Nassim Nicholas Taleb illustrated this type of blindness in his book "Black Swan" with an example of a disciplined Wall Street "rocket scientist" actuary and a street wise trader predicting the outcome of a coin toss. The problem put before them is:
 "Assume that a coin is fair, i.e., has an equal probability of coming up heads or tails when flipped. I flip it ninety-nine times and get heads each time. What are the odds of my getting tails on the next throw? Actuary: "One-half, of course, since you are assuming 50 per cent odds for each and independence between draws" Street Trader: "1 percent. You are either full of crap or a pure sucker to buy that "50 puhcent" business. That coins got to be loaded. It can't be a fair game." (Taleb 2007 p. 124)
And where would we be without GPS? They padded the lampposts along Brick Lane in London, to prevent injury to people walking into them as they looked at their phones. I couldn't find any indication that this was a joke. And this is also funny.)  And there are regular stories of people driving their cars into rivers because their Satellite Navigation system didn't mark the ferry crossing.

 We think about maps as giving us the big picture, but it helps to remember that in actuality, maps are just little pictures. Big data is like a map; it gives a useful distilled view of the world, but it hides important realities. Big Data approaches make one way of looking at things much more available -- easier -- than others; and the particular lens that Big Data gives us is numbers, which are by nature distilled of context.

Further, Big Data is by nature mostly about looking at things that have already happened, or at best, tracking what is happening in real time. Taken too far, it can be like looking at your feet while you walk. It's easy to walk into a lampost. This is how Big Data misses the point. Over-confidence in data-based insights led Walmart to lose 1.85 bn dollars. In 2008 Walmart changed the customer experience of its stores based on customer survey data, reducing clutter and visible stock. This was wrong. The available data, in persuasive, corporate-friendly, numeric form, led Walmart to neglect the information it did not have: behavioural insight.

 The philosopher Martin Heidegger's essay "The Question Concerning Technology" is a detailed analysis of this tendency of technology to conceal at the same time as it reveals. By making certain paths very easy and available, other less available paths become hidden to us. When we travel at high speed down an expressway, we do not see the smaller footpaths, ways that may in fact lead to better places.

Your Brain on Data

Humans are by nature very susceptible to being misled by Big Data-type approaches. You may have heard the claim that we only use 10% of our brain. Not true. What is true, however, is that we are consciously aware of 5% of our brain's activity.
Think about the last time you drove home from work. You weren't paying attention to the steering wheel, or the turn indicator, or the brake pedal. You were probably thinking of something entirely other than driving. Dinner, perhaps. When you arrived home, you might not have been able to recall whether a particular traffic light was red, or which lane you drove across the bridge. And yet you were safe. This is because the 95% of our brain activity that is outside of our conscious awareness was busy keeping us safe.

 To use an example from Ian McGilchrist's book "The Master and His Emissary"  a bird uses the attentive 5% of its brain to locate and pick out a seed of grain from among gravel, while the unfocussed 95% of its brain stays alert for cats. 5% of brain activity deals with attentive, focussed, and methodical processes. This is the brain we use for symbol manipulation, including logic, math, and language. It deals with focus, goals, and conscious, ordered thought. The rest of the brain deals more holistically with sensory input. It is circumspect and largely silent. Its activities are not directly accessible to conscious thought.

 To use a cultural metaphor, the 5% is low context, the 95% is high context. Vision is a useful metaphor and a direct example of the relationship of these different functions of the brain. Foveal vision is the sharp central vision necessary in humans for reading, watching, driving, and any activity where visual detail is of primary importance. And yet the size of the foveal field is roughly the size of your thumbnail held at arm's length. The rest of vision is circumspect, providing context.

 But just because this (larger) part of our brain is circumspect, unfocussed, and largely silent, does not make it less important. Because, although we use the focussed part of our brain to solve problems, the circumspect part of our brain tells us what problems to solve. It's the circumspect part that switches our attention from seed to cat. Big Data approaches speak to the focussed, methodical 5% of our brains. But if we rely only on focussed, algorithmic, approaches, it's likely we will get taken by surprise ...and possibly eaten.

Big Data v. Big Picture

I'm arguing that there are low and high context ways of approaching the world, and that Big Data is a low context approach. Low context approaches are self-contained, focussed, algorithmic and explicit; High Context approaches are circumstantial, circumspect, contingent, and implicit. The table below compares the two approaches.
Low Context High Context
Big Data Big Picture
Rational Reasonable
Intellect Senses
Technical Practical
Corporate Entrepreneurial
Sound Methodology Credible Champion
Invention Innovation
New Patents New Behaviors
Evidence Belief
Corporate cultures tend to be low context, because cross-cultural, cross-disciplinary communication needs to be self-contained, and finally, reduced to numbers. Money is the ultimate low context technology. Low Context approaches are essential for detailed work, but can be oblivious to important realities. Big Data does not give us the Big Picture.

Really Big Data

The implication is that High Context approaches can identify opportunities that Big Data approaches would miss. So, what does a high context approach look like? How can we see the Big Picture? Firstly, we need a lot more data. Beyond the low-context quantifiable/intellectual inputs of Big Data, we need to draw upon all our senses to make sense of reality.
  • Sight
  • Sound
  • Taste
  • Touch
  • Smell
  • Balance
  • Morality
  • Timing
  • Humor
  • Beauty
  • Justice
  • ...
…and really, would you want to take a step  without consulting these senses?

 Your brain is the only processor capable of collecting, processing and synthesizing, this vast and diverse amount of information. Like other neural networks, it's a black box, which means that it's inner workings are opaque to us; but it is a highly efficient and reliable data mining engine, still unsurpassed in finding patterns in complex data.
So you want to get out of the office. In order to expose these senses to the complex environments where they can collect relevant information, you want to visit your customers in the wild, in the actual environments where your customers live and work. That's the main thing. Go to where your customers are.

That's 95% of the approach: go there and observe and absorb and let the intuitive brain do its work. But, of course, the focussed, analytical 5% of your brain will still need something to keep it occupied, and you will need a low-context way to explain your findings back at the office, so here are some explicit things you may want to look for in your customer's natural habitat.

Look for human traces. Scan the environment for improvisations, modifications, signs of people making use of objects in unexpected ways. Jane Fulton Suri of IDEO calls these Thoughtless Acts. English Sculptor Richard Wentworth has created a photo series of examples of these improvisations called Making Do and Getting By.  I like to imagine this is how the creators of Pinterest were inspired, by noticing that people collected photos – and pinned them on boards.

 Look for repeated behaviors, habits and rituals. This is what the inventors of NEST, a thermostat that learns and automates the regular adjustments you make to room temperature, noticed. See also Kevin Henry's essay comparing and analysing the approaches of Fulton Suri and Wentworth, Parallel Universes.

 Go into your customer's natural habitat and observe. Take notes and photos and videos so you can communicate back at the office. You will know you are done when you feel that you empathize with your customer.
em·pa·thy/ˈempəTHē/ noun:  The ability to understand and share the feelings of another. 
But don't be too rigorous! You must be relaxed and open. Stress releases the hormone cortisol into your bloodstream, which inhibits learning and creativity. Your brain will work best when you are enjoying yourself.

Concluding Low Context Takeaway

Here is the takeaway:
  1. Big Data can lead us to miss the big picture. Crunchy numbers are an essential part of a healthy information diet, but Big Data approaches have been shown to diminish the appetite for other essential sources of information.
  2. Go outside and Play! Get out of the office and into actual customer environments, and do it in a way you enjoy, so your focussed analytical methodology doesn't interfere with your brain's proper work.
  3. Money is the ultimate low-context technology. Don't do it for money. Look at the bigger picture:  Do it for love. You'll get better returns.

Table of Context

Here are the main books that have informed these thoughts:

Here is the presentation I used at the UX conference in Prague. The text is below. Thanks for inviting me.

And here's a wonderful RSA version of Iain McGilchrist's talk on the divided brain:

If your interests include theory and philosophy, please check out my other blog.


  1. Beautiful article Maurice.
    I was waiting to read this :)

  2. Bravo Maurice! I do miss our conversations and I hope we can work together again some time in the future.


don't hold back!