avatar
Why We Ignore the Obvious at our Peril


When Adrian Sanders asked the Murdochs, during their appearance before the House of Commons culture, media and sport select committee on Tuesday 19 July, if they were familiar with the term "willful blindness", their startled silence was revealing. The Liberal Democrat MP defined it for them by reading from a recent blog post I had written for the Huffington Post: "If there is knowledge that you could have had and should have had but chose not to have, you are still responsible."

It was, then and now, an idea that should alarm any executive. The legal concept of willful blindness originated in the 19th century. The judge in the case of Regina v Sleep (1861) ruled that an accused could not be convicted for possession of government property unless the jury found that either he knew the goods came from government stores or he had "willfully shut his eyes to the fact".

Nowadays, this law is most commonly applied in money-laundering and drug-trafficking cases, but the behaviour it describes is all around us – in banks, the Catholic Church, at BP, in Abu Ghraib Prison, in most industrial accidents. The narratives nearly always follow the same trajectory: years of abuse involving a large number of participants, plenty of warning signs, and, when the problem finally explodes, howls of pain – how could we have been so blind?

Cases of willful blindness aren't about hindsight. They feature contemporaneous infor­mation that was available but ignored. While it is tempting to pillory individual villains, the causes are more often systemic and cultural. There are many reasons – psychological, social and structural – why we don't see what we most need to notice. None of them provides an alibi or an excuse. What each does is shed light on how these organisational car crashes happen – and how they might be prevented.

Chief among culprits is power. When Richard Fuld was chief executive of Lehman Brothers, he perfected the seamless commute: a limo drove him to a helicopter, which flew him to Manhattan, where another limo whisked him to the bank's offices. Front and lift doors were timed so that Fuld could ascend to his office without encountering a single employee. Most leaders of organisations inhabit a bubble of power, of which Fuld's commute is a magnificent physical representation. They are isolated, or surrounded by those desperate to please. The powerful also communicate differently. Academic analysis of their language shows that, confronted by risky situations, the powerful think in more abstract terms, are more optimistic and more certain that they are right. They are both mentally and physically cut off from the reality most people would recognise. Power is a problem, not a perk, and it is exacerbated by money. Extremely high pay adds social isolation to the psychological solipsism of power. Moreover, because business decisions, for the most part, are framed as purely economic choices, the focus on money crowds out ethical considerations. If the numbers work, the decision works – doesn't it?

The use of money as the primary, often the only, measure of success imposes enormous pressure on ostensible independents, such as accountants, lawyers and consultants, to conform to what is expected of them by powerful corporations. Rupert Murdoch made much of the vast scale of News Corporation, in which the News of the World, he told MPs, represented less than 1 per cent of the whole. But any corporation might do well to ask whether it has become too complex to manage. Enron declared bankruptcy before it needed to because its balance sheet was so complicated that no one knew how much the company owned. Similarly, the banks had no accurate way of measuring systemic risk. Never mind too big to fail; we now have a large number of companies that may be too big to run.

Outsourcing renders oversight more difficult because it atomises processes until no one can see how they connect. It's like Isaac Asimov's I, Robot series, in which the law prohibits the killing of human beings; the robots resolve this by breaking their plans into so many steps that no single one is illegal.

After the Deepwater Horizon drilling rig explosion in the Gulf of Mexico in April 2010, Tony Hayward, the then chief executive of BP, was quick to point out that the rig was built by Hyundai in Korea to a design by a Texas firm, R&B Falcon, which was then bought by the Swiss operators, Transocean, who leased it to BP. Most of the victims were not BP employees. "This wasn't," Hayward said, "our accident." Even Apple, more rigorous than most in its scrutiny of companies at every stage of its supply chain, was blindsided by the suicides of at least 17 employees over the past five years at Foxconn factories in China. The sites may have been equipped with swimming pools, cinemas and canteens, but that spoke little of how the workforce was feeling.

Outsourcing has become so endemic in western economies that there are no areas in which it isn't considered, including war and policing. In the US and the UK, the number of security guards is now more than twice that of public police. Once subcontracting and outsourcing are this widespread, there is a high likelihood of a company becoming blind to how work gets done; the cynical will argue that that is what they are for.

F A Hayek wrote that "without a theory, the facts are silent" – but with a theory or ideology, inconvenient facts can become invisible. With disarming frankness, the Nobel Prize-winning economist Paul Krugman acknowledged in the New Yorker in 2010 that: "I think there's a pretty good case to be made that the stuff that I stressed in the models is a less important story than the things I left out because I couldn't model them."

The fervent belief in free markets of Alan Greenspan, chairman of the Federal Reserve from 1987 to 2006, similarly blinded him to repeated failures in unregulated derivatives trading. Between 1994 and 2008, billions of dollars were lost in derivatives bets that others (such as George Soros, Muriel Siebert and Frank Partnoy) interpreted at the time as early- warning signs that the market wasn't working, but Greenspan could not see what he would not see. Testifying before Congress in 2008, he was asked whether his ideology had misled him. He did not recant, conceding only that he had "found a flaw", which surprised him because, he said: "I had been going for 40 years or more with very considerable evidence that it was working exceptionally well."

Scientists can be just as myopic. In 1956, the Oxford-based epidemiologist Alice Stewart demonstrated, with startling data, that the chances of childhood cancer were vastly increased by X-raying pregnant mothers. At the time, these cancers were killing one child every week, yet it took 25 years before the practice was abandoned by the British and American medical establishments. Stewart's data flew in the face of current epidemiological theory – "threshold theory", which maintained that, while a large dose of anything could be dangerous, there was always a point, or threshold, beyond which it was bound to be safe. Her research indicated that there was no safe level of radiation for foetuses. Stewart was fiercely opposed by Britain's foremost epidemiologist of the time, Richard Doll, who was famed for identifying the link between smoking and cancer. Not until 1997 did he quietly retire the threshold theory with the most modest of mea culpas.

Big ideas can create tunnel vision, blinding the believer to disconfirming data. This cognitive dissonance is resolved in favour of the faith. Rupert Murdoch has always believed in the business value of political power and the importance of scale. Those beliefs blinded him to growing disgust with how our political elite behaved, and to popular discomfort with large foreign corporate takeovers. It is tempting to deride other people's theories and ideologies, but most people, governments and organisations have them. As Greenspan testified: "Ideology is the way people deal with reality. Everyone has one."

Whether it is the belief that military intervention saves lives, or big government is bad, or companies must go global to succeed, ideologies are what the psychologist Anthony Greenwald called "totalitarian egos". They work to preserve our main beliefs by erasing threatening ideas, suppressing contradictory evidence and rewriting history. Once enlisted, those totalitarian egos are strikingly submissive.

Ever since the Yale psychologist Stanley Milgram's 1961 experiments into obedience, we have known that, without reward for compliance or punishment for refusal, most people (roughly 65 per cent) will commit unethical acts when asked to do so by someone in authority. Repeated around the world ever since with unchanging outcomes, the experiments showed, Milgram wrote, "the capacity for man to abandon his humanity – indeed the inevitability that he does so – as he merges his unique personality into larger institutional structures . . . It would not be true to say he loses his moral sense. Instead it acquires a radically different focus. His moral concern now shifts to how well he lives up to the expectation the authority has of him."

Authority is a capacity whose dangers few chief executives recognise. Ambitious employees will work hard to deliver what's wanted, to infer what will make them successful in the eyes of the organisation they have joined. And their moral focus will change. In the 1950s at Swarthmore College, Pennsylvania, Milgram's teacher Solomon Asch had conducted experiments into conformity, which demonstrated that, in simple tests comparing three vertical lines, most people would rather give an obviously wrong answer that kept them in a group than a correct answer that would make them outsiders. Fifty years later, MRI scan versions of this experiment were able to demonstrate that, at the moment of conforming, less conscious thinking occurs in the brain.

Instead of groups benefiting from the collective wisdom of many, they produced reduced thoughtfulness from each. In successful and glamorous companies, the lure and rewards of conformity can become more extreme, even cult-like. Deborah Layton, one of the few survivors of Jim Jones's "Peoples Temple" in Guyana, has written eloquently about how her fellow members were eager to surrender critical judgement in exchange for social acceptance. That unquestioning habit persisted right up until their mass suicide in 1978.

Today, Layton sees cultish qualities in many corporations. If everyone is drinking the same Kool-Aid, you can be pretty sure no one will speak up when something is wrong. They are too eager for inclusion, too afraid of exclusion. For the most part, we would rather be wrong than alone. A 2003 New York University study into organisational silence found that 85 per cent of executives had had issues or concerns at work that they had never articulated. The chief reason was fear of becoming an outcast, rejected by boss and peers.

When I conducted a similar study at the University of Bath in 2010, the numbers were the same but the cause was different: here, silence was provoked mostly by a sense of futility – speaking up wouldn't change anything. That's how millions of pounds of payment protection mortgage insurance gets sold.

Though ambition, competitiveness and hierarchy may exacerbate these behaviours, high moral purpose won't protect against them; silence is implicated in many NHS and care-home scandals. The mistreatment of the elderly at the Royal Sussex County Hospital, and the violent abuse of adults with learning difficulties at Winterbourne View, weren't hidden – they took place on public wards where everyone could see. At the Bristol Royal Infirmary in the early 1990s, it was public knowledge that babies were dying in heart operations. For years, the medical establishment, notoriously conservative, said nothing; research shows medical students are more conformist after their training than when they begin, an effect that is called "the hidden curriculum".

In his memoir, Beyond Business, published in 2010, the former BP chairman John Browne wrote: "I wish someone had challenged me and been brave enough to say, 'We need to ask more disagreeable questions.'" Yet he did not seem to understand why they hadn't. When managers say that they want to hear the bad news, that they won't shoot the messenger, most employees do not believe them. I would argue that this is the biggest challenge facing any organisation today.

Discussing willful blindness with me, the cognitive psychologist Albert Bandura argued that "people are highly driven to do things that build self-worth; you can't transgress and think of yourself as bad. So people transform harmful practices into worthy ones, coming up with social justifications, distancing themselves with euphemisms and numbers, ignoring the long-term consequences of their actions." His examples included TV producers, gun manufacturers and climate-change deniers. But the greatest single cause of willful blindness may also be the most basic.

To build that sense of self-worth, we surround ourselves with people and information that confirm it. Overwhelmingly, we prefer people like ourselves – and there is a solid physiolo­gical reason why. The brain can't handle all the information it is presented with, so it prioritises. What gets a head start is information that is already familiar – and what is most familiar to us is us.

So, we feel most comfortable with people and ideas we already know. Just like Amazon's recommendation engine or eHarmony's online dating programmes, our brain searches for matches, because building on the known makes for highly efficient processing. At a trivial level, this preference shows up in consumer preference for products whose names share their initials: Carol likes Coke but Peter prefers Pepsi. More seriously, over time our neural networks, just like our opinions and ideologies, become deeper but also narrower.

That is as true for us, when we choose media we agree with, as it is for party leaders who give priority to editors who agree with them. Everyone is biased in favour of themselves; it may be one reason why, despite decades of diversity programmes, women and minorities have made so little progress inside corporations. Looking at the constituents of the News Corporation board, you see the bias played out in full: the directors are either Murdochs or friends and employees of the Murdochs. Strikingly absent are individuals sufficiently different and independent to provide any challenge. This isn't unusual. At the time that the HBOS board was collapsing, Dennis Stevenson remarked that it was "as one"; it did not seem to occur to him that this might have been the problem. News Corp isn't the first organisation to fall into these traps and it won't be the last. The central irony of willful blindness is that it makes us feel safe even as it puts us in danger.

As Colm O'Gorman, one of the first people to uncover abuse in the Catholic Church in Ireland, told me: "We make ourselves powerless when we pretend we don't know." But just because willful blindness is endemic does not make it irresistible. Roy Spence, a Texan advertising executive, refused to work with Enron even as the rest of the world beat a path to its door. How did he see what others missed? He thought a lifetime of seeing through the eyes of the powerless gave him different perspectives. "My sister had cystic fibrosis and I used to wheel her to school every morning," he told me. "I could see people pitying us, oblivious to the richness of our relationship. It made me ask, then as now: if they're missing so much about us, what I am missing about them?" That internal dialogue is what Hannah Arendt called thinking.

Margaret Heffernan is the author of "Willful Blindness" (Simon & Schuster, £12.99)

  

Leave a comment