Adam Curtis Feed

HYPERNORMALISATION

2016-10-11T09:56:00+00:00

I have a new film going up on iPlayer this Sunday - the 16th. Here’s a background to what the film is about. And a trail.

We live in a time of great uncertainty and confusion. Events keep happening that seem inexplicable and out of control. Donald Trump, Brexit, the War in Syria, the endless migrant crisis, random bomb attacks. And those who are supposed to be in power are paralysed - they have no idea what to do.

This film is the epic story of how we got to this strange place. It explains not only why these chaotic events are happening - but also why we, and our politicians, cannot understand them.

It shows that what has happened is that all of us in the West - not just the politicians and the journalists and the experts, but we ourselves - have retreated into a simplified, and often completely fake version of the world. But because it is all around us we accept it as normal.

HyperNormalisation

The film has been made specially for iplayer - and is a giant narrative spanning forty years, with an extraordinary cast of characters. They include the Assad dynasty, Donald Trump, Henry Kissinger, Patti Smith, the early performance artists in New York, President Putin, intelligent machines, Japanese gangsters, suicide bombers - and the extraordinary untold story of the rise, fall, rise again, and finally the assassination of Colonel Gaddafi.

All these stories are woven together to show how today’s fake and hollow world was created. Part of it was done by those in power - politicians, financiers and technological utopians. Rather than face up to the real complexities of the world, they retreated. And instead constructed a simpler version of the world in order to hang onto power

But it wasn’t just those in power. This strange world was built by all of us. We all went along with it because the simplicity was reassuring. And that included the left and the radicals who thought they were attacking the system. The film shows how they too retreated into this make-believe world - which is why their opposition today has no effect, and nothing ever changes.

But there is another world outside. And the film shows dramatically how it is beginning to pierce through into our simplified bubble. Forces that politicians tried to forget and bury forty years ago - that were then left to fester and mutate - but which are now turning on us with a vengeful fury.

Trailer for the 2016 Adam Curtis film - HyperNormalisation

TRAILER TRASH

2014-12-09T16:20:04+00:00

THE FILM'S LAUNCH DATE ON BBC iPLAYER IS NOW 25TH JANUARY 2015

Politicians used to have the confidence to tell us stories that made sense of the chaos of world events.

But now there are no big stories and politicians react randomly to every new crisis - leaving us bewildered and disorientated.

And journalism - that used to tell a grand, unfurling narrative - now also just relays disjointed and often wildly contradictory fragments of information.

Events come and go like waves of a fever. We - and the journalists - live in a state of continual delirium, constantly waiting for the next news event to loom out of the fog - and then disappear again, unexplained.

And the formats - in news and documentaries - have become so rigid and repetitive that the audiences never really look at them.

In the face of this people retreat from journalism and politics. They turn away into their own worlds, and the stories they and their friends tell each other.

I think this is wrong, sad, and bad for democracy - because it means the politicians become more and more unaccountable.

I have made a film that tries to respond to this in two ways.

It tells a big story about why the stories we are told today have stopped making sense.

But it is also an experiment in a new way of reporting the world. To do this I’ve used techniques that you wouldn’t normally associate with TV journalism. My aim is to make something more emotional and involving - so it reconnects and feels more real.

BBC iPlayer has given me the opportunity to do this - because it isn’t restrained by the rigid formats and schedules of network television. It's a place you can go to experiment and try out new ideas.

It is also liberating - both because things can be any length, and also because it allows the audience to watch the films in different ways.

The film is called Bitter Lake. It is a bit of an epic - it’s two hours twenty minutes long.

It tells a big historical narrative that interweaves America, Britain, Russia and Saudi Arabia. It shows how politicians in the west lost confidence - and began to simplify the stories they told. It explains why this happened - because they increasingly gave their power away to other forces, above all global finance.

But there is one other country at the centre of the film.

Afghanistan.

This is because Afghanistan is the place that has repeatedly confronted politicians, as their power declines, with the terrible truth - that they cannot understand what is going on any longer. Let alone control it.

The film shows in detail how all the foreigners who went to Afghanistan created an almost totally fictional version of the country in their minds.

They couldn’t see the complex reality that was in front of them - because the stories they had been told about the world had become so simplified that they lacked the perceptual apparatus to see reality any longer.

And this blindness led to a terrible disaster - support for a blatantly undemocratic government, wholesale financial corruption and thousands of needless deaths.

A horrific scandal that we, in our disconnected bubble here in Britain, seem hardly aware of. And even if we are - it is dismissed as being just too complex to understand.

But it is important to try and understand what happened. And the way to do that is to try and tell a new kind of story. One that doesn’t deny the complexity and reduce it to a meaningless fable of good battling evil - but instead really tries to makes sense of it.

I have got hold of the unedited rushes of almost everything the BBC has ever shot in Afghanistan. It is thousands of hours - some of it is very dull, but large parts of it are extraordinary. Shots that record amazing moments, but also others that are touching, funny and sometimes very odd.

These complicated, fragmentary and emotional images evoke the chaos of real experience. And out of them i have tried to build a different and more emotional way of depicting what really happened in Afghanistan.

A counterpoint to the thin, narrow and increasingly destructive stories told by those in power today.

HAPPIDROME - Part One

2014-10-22T14:53:20+00:00

In the battle for Kobane on the Syrian border everyone talks about the enemy - IS - and the frightening ideas that drive them. No-one talks about the Kurdish defenders and what inspires them.

But the moment you look into what the Kurds are fighting for - what you discover is absolutely fascinating. They have a vision of creating a completely new kind of society that is based on the ideas of a forgotten American revolutionary thinker.

He wanted to create a future world in which there would be no hierarchies, no systems that exercise power and control individuals. And the Kurds in Kobane are trying to build a model of that world.

It means that the battle we are watching night after night is not just between good and evil. It is also a struggle of an optimistic vision of the future against a dark conservative idea drawn from the past.

It is a struggle that may also have great relevance to us in the west. Because the revolutionary ideas that have inspired the Kurds also shine a powerful light on the system of power in Britain today. They argue that we in the west are controlled by a new kind of hierarchical power that we don’t fully see or understand.

There are two men at the heart of this story.

One is the American revolutionary thinker. He is called Murray Bookchin. Here is a picture of Bookchin looking revolutionary.

The other man is called Abdullah Ocalan. He is the leader of the Kurdish revolutionary group in Turkey - the PKK

Here he is in 1999 after he had been captured by Turkish security forces and was on his way to a jail on a tiny island in the Sea of Marmara where he would be the only prisoner.

In his solitude he would start to read the theories of Murray Bookchin and decide they were the template for a future world.

Both men began as hardline marxists.

Murray Bookchin was born in New York in 1921. In the 1930s he joined the American Communist Party. But after the second world war he began to question the whole theory that underpinned revolutionary marxism.

What changed everything for him was the experience of working in a factory. Bookchin had gone to work for General Motors - and he realized as he watched his fellow workers that Marx, Lenin and all the other theorists were wrong about the working class.

The Marxist theory said that once working men and women came together in factories the scales would fall from their eyes - and they would see clearly how they were being oppressed. They would also see how they could bond together to become a powerful force that would overthrow the capitalists.

Bookchin saw that the very opposite was happening. This was because the factory was organised as a hierarchy - a system of organisation and control that the workers lived with and experienced every second of the day. As they did so, that hierarchical system became firmly embedded in their minds - and made them more passive and more accepting of their oppression.

But Bookchin didn’t do what most disillusioned American Marxists in the 1950s did - either run away to academia, or become a cynical neo-conservative. Instead he remained an optimist and decided to completely rework revolutionary theory.

Here is Bookchin in 1983 talking about how his thinking became transformed - and how his factory experiences led him towards anarchism. It’s part of a fantastic film called Anarchism in America - as well as Bookchin it’s got a great bit with Jello Biafra, and it’s really worth watching if you can get hold of it.

Abdullah Ocalan was born in 1948 in south-eastern Turkey. In the early 70s he went off to Ankara and became a student - and like many students then he became fascinated by revolutionary Marxism.

But Ocalan also believed in Kurdish nationalism. His family were Kurds - and like millions of other people in the south east of Turkey they considered themselves part of an invisible nation that stretched across the border into Northern Syria, Iraq and parts of Iran.

The Kurds had always had a bad time. They were oppressed by the Ottoman empire. Then - at the end of the first world war they were promised a homeland, but the new Turkish state refused to give them any land. While the British went and created the new state of Iraq and sent aircraft to bomb the Kurds there into submission.

Winston Churchill - who was the Secretary of War at the time - said they should be gassed. He said:

“I am strongly in favour of using poisoned gas against uncivilised tribes.”

But the British government thought it wasn’t a good idea. The Kurds would have to wait for Saddam Hussein - who was also strongly in favour of using poisoned gas.

In 1978 Ocalan, along with a group of other student revolutionaries formed The Kurdistan Workers’ Party - the PKK. Its aim was to create an independent Kurdistan that would also be a marxist state. Then, in 1980 there was a military coup in Turkey - and Ocalan decided that the PKK party should use violence to achieve its aim.

He went off to the Bekaa valley in the Lebanon and set up a training camp to create his army of liberation. Here is some footage of them training - along with Ocalan explaining what they are trying to achieve.

It’s slightly ghostly - because over ten years years before, all the Palestinian marxists and the European student radicals had also set up training camps in the middle east. They were now either all dead or defeated - and Ocalan and his followers were like latecomers to a party that was almost over.

But that didn’t stop their growing popularity in Kurdish areas in Turkey. I’ve included some film from inside Turkey at that time - where weddings turn into celebrations of PKK attacks.

At the very time that Ocalan was training in the Bekaa - Murray Bookchin was in the US writing a grand theory of how to create a new revolutionary kind of world.

The problem - he had decided - were the hierarchies throughout society that controlled people. In many cases they had become so deeply embedded in peoples minds that they were almost invisible - people just accepted them as a natural part of life.

To begin with Bookchin turned to anarchism. But he didn’t stop there - he started mixing anarchist theories with all sorts of other radical ideas drawn from libertarian individualism, scientific theories of nature, Trotskyism and experimental psychology. One of the most important influences on him was a writer called Lewis Mumford who in the1960s argued that new and unseen hierarchies were emerging out of human beings relationship to computers.

The trouble was that all the anarchists and the marxists and the ecologists didn’t like their theories being mixed up together. Bookchin also had a habit of telling these other revolutionaries off for being limited in their ideas - writing pamphlets with titles like “Listen Marxists!”, and another which accused American anarchists of being “narcissistic lifestyle anarchists”

As a result Bookchin’s ideas remained out on the fringes.

Meanwhile Abdullah Ocalan was helping to create horror on a vast scale in Turkey. The war he had started had become a nightmare. By the mid 1990s nearly 40,000 people had been killed - the majority of them Kurdish civilians. The very people the PKK were fighting to liberate.

The Turkish security forces had responded with extraordinary brutality. They destroyed over 3000 villages - sometimes bombing and shelling them to destruction. At the same time they paid Kurdish villagers to set up local defence forces called “Village Guards” - to stop the PKK. If they didn’t agree their villages would be destroyed.

What resulted was a large part of South-eastern Turkey being transformed into a strange half-life zone. Millions of people lived in a state of constant fear and suspicion. The mood was captured by a film made by the writer Michael Ignatieff. Here are a couple of extracts.

It starts with him going to a PKK camp over the border in Northern Iraq where the fighters are still devoted revolutionary marxists. There is a really good section showing the “self-criticism” session for the women PKK fighters being held in a tent. He also takes part in target practice with the fighters.

But then Ignatieff goes into the heart of the Turkish-Kurd disputed area - to the town of Dyarbakir - and you get a real sense of the frightening world the war of national liberation had created.

Then - in 1999 - Abdullah Ocalan was captured by the Turkish security forces. He had been forced out of his base in Syria and had become a stateless person on the run. Everywhere he went - Russia, Italy, Greece - forced him on, until he ended up in Nairobi where the Turks kidnapped him when he was en route to the airport. Allegedly they were helped by the CIA.

Ocalan was then taken to Turkey where he was put on trial. There were massive protests by Kurds around the world. Here are some of the news reports of what happened. Plus some of the strange footage the Turks released showing Ocalan gagged and bound on the private jet taking him back to Turkey - and then on the boat out to the island prison.

Ocalan was taken to a maximum security prison on the island of Imrali. He was the only prisoner - surrounded, it is said, by a thousand military guards.

In his solitary confinement Ocalan began to read. And in 2002 he found a book by Murray Bookchin called The Ecology of Freedom - The Emergence and Dissolution of Hierarchy. Despite the title, Ocalan was inspired by Bookchin’s arguments - and it changed the way he saw the world.

It made him realise - Ocalan said - that all systems of power create 'submissive persons’, and that the only way to really create a true revolutionary world was to build one without any hierarchies. He turned his back on Marxism and nationalism and proposed instead a completely decentralised system of government - run by local committees.

It was modelled on Bookchin’s ideas - and Ocalan sent out instructions to all militants that they should read The Ecology of Freedom. He even sent a letter to Bookchin asking to meet him - but Bookchin was too ill. Two years later in 2006, when Bookchin died, the PKK saluted him from their mountain hideouts as - “one of the greatest social scientists of the twentieth century.”

Then came the Syrian civil war - and the Kurds in the north of Syria used the chaos to create a series of free enclaves - one of which is the city of Kobane. The Kurdish group who did this is allied with the PKK - and it seems that they have set up free self-governing communities in these areas that are inspired by Bookchin’s ideas.

It is a fascinating story - but in our cynical age such ideas seem unrealistic, pie-in-the-sky experiments.

But I have found a film in the archives that suggests that maybe these kinds of ideas may have more relevance to the modern world than we think.

It’s a film made in the mid-1960s called Towards Tomorrow - and it tries to envisage what utopias might emerge in the next century. It asks how society, in an age of the masses and the decline of old forms of authority, will be organised. It describes two possible and contrasting visions of the future.

One is put forward by the American thinker who inspired Murray Bookchin - the writer and techno-theorist Lewis Mumford, and is very similar to the very ideas that the Kurds are now experimenting with.

The other is a vision of a world managed by an elite group of technocrats who see people as passive beings who need to be constantly monitored and managed in order to keep them happy. The section of the film that outlines this possible future is very odd - because as you watch it you get an eerie sense of familiarity. That what is being described is the very world in which we all now live.

Here - for example is one of the exponents of this future world. He is called Herman Kahn and was one of the first think-tank futurologists. The other man with him is his assistant. Listening to them talking in 1966 and describing what they think might come in the future is quite strange.

The film then goes to another promoter of this utopian future - an experimental psychologist called B.F. Skinner. He outlines a new way of controlling and ordering people. It is no longer possible, he says, to tell people what to do. In an age of individualism and mass democracy people won’t accept that any longer. Instead you reward them for behaving in the ways you want them to.

You make them happy, and they feel that they are in control - because by doing something they get the reward.

Skinner had started doing this with pigeons - and the film shows how he trained them to peck at a particular button because if they did they got a pellet of food, whereas if they pecked another button they got nothing. For Skinner though, that was just the start - and the film shows how his ideas were also being applied to human beings.

Here is that section - along with Skinner explaining his ideas. The film records an experiment in a mental hospital in San Bernadino - California. The patients are given rewards in the form of plastic fake money if they do what the doctors consider the right social behaviour. They can then use that money at meal-times to buy their way onto a “nice” table - with tablecloth and flowers.

Those without the rewards have to eat - as one of the nurses puts it, “in less elegant conditions”.

What emerges in the hospital is a new, ordered hierarchy created by a system of reward - but one where the patients don’t feel controlled - instead they feel “empowered” because it was through their actions that they received the reward. Skinner makes clear in the film that he sees this as a model for how to run a future kind of society.

Watching these sections of the film does make you think that what is being described is spookily close to the system we live in today. And that maybe we have misunderstood what really has emerged to run society since the 1980s.

The accepted version is that the neo-liberal right and the free market triumphed. But maybe the truth is that what we have today is far closer to a system managed by a technocratic elite who have no real interest in politics - but rather in creating a system of rewards that both keeps us passive and happy - and also makes that elite a lot of money.

That in the mid 1980s the new networks of computers which allowed everyone to borrow money came together with lifestyle consumerism to create a system of social management very close to Skinner’s vision.

Just like in the mental hospital we are all given fake money in the form of credit - that we can then use to get rewards, which keep us happy and passive. Those same technologies that feed us the fake money can also be used to monitor us in extraordinary detail. And that information is then used used to nudge us gently towards the right rewards and the right behaviours - and in extremis we can be cut off from the rewards.

The only problem with that system is that the pigeons may be getting restless. That not only has the system not worked properly since the financial crash of 2008, but that the growing inequalities it creates are also becoming a bit too obvious. The elite is overdoing it and - passive or not - the masses are starting to notice.

Which makes the alternative - the vision put forward by Lewis Mumford in the film, and which inspired Murray Bookchin - and the Kurds, seem more interesting as an alternative.

Here is Mumford from the film. He starts by criticising the managed utopia - how it turns people into sleepwalkers. He has a great quote:

“You reward them. You make people do exactly what you want with some form of sugar-coated drug or candy which will make them think they are actually enjoying every moment of it.

This is the most dangerous of all systems of compulsion. That’s why I regard Skinner’s utopia as another name for Hell. And it would be a worse hell because we wouldn’t realise we were there.

We would imagine we were still in Heaven.”

Mumford then goes on to describe eloquently the alternative, a system of direct democracy where we would all awake and become genuinely empowered - able to take part properly in deciding our destiny.

It is a powerful and optimistic vision of a new kind of progressive politics. But it has one very serious problem.

It means we would have to spend a lot of time going to meetings

THE VEGETABLES OF TRUTH

2014-10-10T14:00:13+00:00

This is really just an excuse to show a wonderful film about vegetables.

But it is also about how modern science has radically changed in a way that hasn’t been fully understood.

How it has gone from promising extraordinary new worlds of the future - to become a powerfully conservative force that holds progress back and tends to keep people in their place.

And the odd role vegetables have played in showing how this has happened.

There are two - parallel - universes of science. One is the actual day-to-day work of scientists, patiently researching into all parts of the world and sometimes making amazing discoveries.

The other is the role science plays in the public imagination - the powerful effect it has in shaping how millions of ordinary people see the world.

Often the two worlds run together - with scientists from the first world giving us glimpses of their extraordinary discoveries. But what sometimes happens is that those discoveries - and what they promise - get mixed up with other social and political ideas. And then the science begins to change into something else.

This happened in a dramatic way in the second half of the twentieth century. Science did very well in the second world war and after the war ambitious scientists promised they could build a new kind of world.

But by the 1970s it became clear that there were unforeseen consequences. It started with chemical pollution - especially DDT killing wildlife. But it was nuclear power that really broke the faith in the optimistic view of science - with the disaster at Three Mile Island in the US in 1979.

What emerged instead was a powerful distrust of the idea that science and technocratic experts could make a better world. Here is a good example of that new mood. It’s an anti-nuclear rally held in New York after Three Mile Island.

Jane Fonda makes a celebrity appearance - and her interview articulates the mood very well. I also love the protest song at the end.

“Just give me the restless power of the wind

Give me the comforting glow of the wood fire

But please take all your atomic poison power away”

But if the scientists had been naive - so too was much of the counter-reaction.

The truth was that it might not have been the science itself that was at fault - but the way the science had been distorted and corrupted by the economic and political demands made on it.

Here is a section of a film I made about what went wrong with the building of the first big nuclear reactors. It shows how the companies building them - like General Electric - were under enormous economic pressure and political demands because of the cold war. And the technologists designed giant systems they knew were potentially unsafe.

Then came the Chernobyl disaster in 1986. All the distrust of big science that had been building up exploded out - and science became the problem. Not the solution any longer.

There was one man who articulated this new view of science very powerfully. He was a German political scientist called Ulrich Beck who wrote a book just before the Chernobyl explosion called Risk Society. In the wake of the disaster it captured the public imagination - and has been incredibly influential on social and political thinking in the west ever since.

The book was powerful because it laid out a new way of looking at the world. Beck said that what the scientists and technologists had been doing with these giant projects was not building a new and glorious future. Without realising it they had been doing the opposite - they had been creating enormous new dangers for the world.

Beck used the word risk. The scientists he said had been “manufacturing risks”.

In the past the big risks to human societies tended to be freak events of nature - like earthquakes and volcanoes and storms. But now the risks came from human ingenuity and ambition. Much of what had been created had potentially world-threatening side effects - like atomic fallout and ecological disasters.

The world had been turned upside down. It wasn’t nature that was the real threat to human existence any longer - it was now human science and technology that had the power to destroy nature and the whole of the planet. And it wasn’t going to stop - this was a new and growing danger.

It meant - Beck said - that the whole role of politics would inevitably change. In the past politicians’ main aim had been to create a more equal society. That was now in decline. In the new “risk society” their main focus should be to create safety.

Beck didn’t mince his words:

“Whereas the utopia of equality contains a wealth of substantial and positive goals of social change, the utopia of the risk society remains peculiarly negative and defensive. Basically, one is no longer concerned with attaining something ‘good’, but rather with preventing the worst.

The dream of the old society is that everyone wants and ought to have a share of the pie. The utopia of the risk society is that everyone should be spared from poisoning”

That was written in 1986 - and it is remarkably prescient. Because that short paragraph pretty much describes the present day mood in our society. A world where individuals are constantly calibrating risks in their lives, while politicians are expected to anticipate and avoid all future risks and dangers.

And everyone gives up on the idea of creating equality, which allows inequality to increase massively.

Beck’s book is extraordinary - because he came from the liberal left. Yet he is basically saying that in the face of these new potential risks we will have to move away from the political idea of progress and social reform - and instead hunker down in the brace position and try and anticipate all dangers that might be coming at us out of the darkness.

To be fair to Beck he is ambiguous in the book about the kind of pessimistic and anxious society that will arise from this new approach. But he says it is inevitable. And in a way it is a very honest depiction of what happened to the liberal mind set at the end of the 1980s - how it retreated into a gloomy pessimism where the only response to events is “oh dear.”

I think the truth probably is that it was the baby boomers losing their youth - and finding themselves unable to face the fact of their own mortality - they started to project their fears onto the rest of society. But somehow people like Beck transformed this into a grand pessimistic ideology.

I want to put up part of an extraordinary documentary made during the events of 1986 that dramatically shows just how different our attitudes to risk used to be. It is the record of the group of Soviet technologists who volunteered to go into the ruined reactor core at Chernobyl after the disaster.

It is extraordinary because they all knew they would die. Their protection against the radiation - as you see in the film - was minimal. It consisted of taping up their cuffs and trouser legs and not much else. But they went in because it was the only way to find out how to contain the disaster.

It is so moving because they are men from an older world. To them risk is irrelevant. They believe in something grander - bigger than their own lives. There is also the most fantastic remote controlled camera - it is mounted on a toy tank and its images are great.

The idea of the Risk Society gave modern science and technology a real kicking. Because they were the ones - it said - who were mainly creating the risks.

But it also allowed science to invent a new role for itself. Because a new breed of scientists came forward and said that they knew how to analyse the dangers - and anticipate the risks. They wouldn’t try and build dazzling new futures, instead they would keep the world safe by spotting the dangers before they arrived.

It was the beginning of a modern science which now permeates the modern world and whose full dimensions I don’t think we’ve fully recognised. It has become central to all sorts of areas - from medicine and public health, through climate change, finance and the welfare state - and even to the anticipation of terrorism and crime.

What the scientists and technologists do is look for patterns, associations and correlations in large amounts of data. It has permeated the public imagination mostly through the regular reports that find associations between diseases and human behaviour. Journalists love them - the one I particularly liked was the scientific report that said that snoring gave you cancer.

But this science does have powerful roots. It was just this kind of search for correlations that allowed scientists to prove that there was a link between smoking and cancer. It was a piece of scientific investigation that changed the world - and has saved millions of people from an early death.

It was work like that which has allowed modern science to rise up in a different form - and again become central to society - because now it was warning us of the dangers. Which is good.

But there is a weakness in this scientific approach that can allow it to be shaped and manipulated by wider social and political forces.

This is because if you look for correlations you often have no real idea of why something is happening - just that it is somehow associated. The warning phrase is “correlation does not mean causation.”

The scientists know this very well - and they constantly try and cross-check to see if the correlation is real or spurious by accounting for all sorts of variables. They look for hidden factors that might really be the reason for something happening and try and adjust the data for these. They have a good name for these hidden factors - “residual confounders.”

But the problem is that they are always trying to imagine what the hidden variables are - and the choice of what you do imagine and what you don’t is inevitably shaped by wider social and political views of the world.

Which bring us to the vegetables.

Seven months ago a scientific report came out that illustrates this danger very clearly.

It was from University College London and it said said that people who eat seven or more portions of vegetables every day - rather than the recommended five - live longer. And it wasn’t just a little bit longer, one of the authors of the report said that the effect was “staggering.”

The claims were powerful. It said that if you ate up to five portions a day it would reduce your risk of death by 29%, but if you ate seven or more portions the risk was reduced by 42%

As a result the report got a lot of publicity - with people arguing that the national guidelines ought to be changed. Here’s some of the TV and newspaper coverage.

But if you look into the report, two rather surprising things emerge.

First that the data is not really strong enough to support the confident conclusions of the researchers. It is, as one scientist not connected with the report told me, far more a leap of faith than a scientifically proven conclusion.

And secondly the scientists who did the research may have ignored another - and very different - conclusion that the data might point to. This is that if you want to live longer you should change society.

To do the report, the researchers had taken data from the Health Surveys of England. Every year a random group of people are asked questions about their lives - and one of the sections asks them how much fruit and vegetables they have eaten in the past twenty four hours.

The researchers took the answers from one of those surveys twelve years ago and compared the fruit and vegetable answers with who had died over the past twelve years. And that is basically it. The report was based on what sixty-five thousand people said they had eaten on one single day a long time ago.

I asked Professor Tom Sanders, who is Professor of Nutrition at Kings College London, about the research behind the report.

He was pretty scathing. The data was dubious he said - because there is no way of finding out if the respondents had lied. He had a good line - “Men lie about smoking. Women lie about vegetables”.

But far more importantly, he said, was the fact that they might have misinterpreted the data. That the reason that some people in the survey live longer may not have anything directly to do with eating more vegetables. It might be that eating more vegetables is the sort of thing people higher up the social scale do.

And people in higher social classes tend to live longer - because of all sorts of other factors, like access to better health-care throughout their life, less stress, living in nicer neighbourhoods with less pollution. All the kinds of things that you tend to get if you have more money and more freedom.

I pointed out to Professor Sanders that the researchers said in the report that they had “adjusted the data for social class”. But he was dismissive of this - saying that the data they used to do this was “incredibly weak” and at times non-existent.

And if you read the report, buried away is an admission that one of the most dramatic correlations - that people who eat canned rather than fresh fruit die much earlier - might be due to some other factor completely.

There is absolutely nothing wrong with eating healthily. And it is very sensible to eat fruit and vegetables regularly.

But something else is going on here. The scientists behind the report are playing on our anxieties and saying you must eat even more healthy food so as to avoid dying early. When in fact the data might be pointing to the very opposite.

That the way to avoid dying early is to reform and restructure society so poor people have more access not just to better food - but to all the kinds of opportunities that richer people do. These are a range of social factors from health care and housing and education to social isolation, stress, unemployment, and higher-risk occupations. These are the sorts of things that also affect how long people live.

To lump it all onto vegetables is unfair.

Science and scientists do all kinds of wonderful things. But when they venture into the social and political world they tend to get bent the way the ideological wind is blowing.

Once it was to support politicians trying to expand their power by remaking society. Now - in an age of individualism - it is to keep us in our place by promoting the idea that we should just focus on ourselves and our bodies. And not think about the wider problems of growing inequality and the unfairnesses that brings.

The scientists are saying - just go and eat another banana or cabbage and you’ll be alright. They are loading everything onto the isolated individual.

Such reports - and there are many - keep us locked inside the anxieties of the “risk society”. While the truth is that you might have a better chance of living longer if you banded together and used that collective power to change society. It would also be a lot more fun than laboriously counting vegetables.

As an antidote - here is a beautiful film about vegetables. It’s a documentary made in 1972 about a leek-growing contest in Newcastle. It is very camp - with lots of men discussing the length and diameter of their leeks.

It is also all about statistics and numbers - because it is the measurements that will decide the winner. But in this case it’s not about the fear of death. It’s all about pride and glory in the vegetables - among men who lead the unhealthiest of lives. Constantly smoking and drinking as they talk about their beloved vegetables.

NOW THEN

2014-07-25T14:00:33+00:00

If you are an American politician today, as well as an entourage you also have a new, modern addition. You have what's called a "digital tracker". They follow you everywhere with a high-definition video camera, and they are employed by the people who want to destroy your political career.

It's called "opposition research" and the aim is to constantly record everything you say and do. The files are sent back every night to large anonymous offices in Washington where dozens of researchers systematically compare everything you said today with what you said in the past.

They are looking for contradictions. And if they find one - they feed it, and the video evidence, to the media.

On one hand it's old politics - digging up the dirt on your opponent. But it is also part of something new - and much bigger than just politics. Throughout the western world new systems have risen up whose job is to constantly record and monitor the present - and then compare that to the recorded past. The aim is to discover patterns, coincidences and correlations, and from that find ways of stopping change. Keeping things the same.

We can't properly see what is happening because these systems are operating in very different areas - from consumerism, to the management of your own body, to predicting future crimes, and even trying to stabilise the global financial system - as well as in politics.

But taken together the cumulative effect is that of a giant refrigerator that freezes us, and those who govern us, into a state of immobility, perpetually repeating the past and terrified of change and the future.

To bring this system into focus I want to tell the history of its rise, and its strange roots - the bastard love-child of snooping and high-level mathematical theory.

It begins with the grubby figure of the early 1960s in Britain - the Private Detective. Up till then private detectives mostly did divorce work. They would burst into hotel rooms to find a married person engaged in adulterous activity. Often these were prearranged situations, set up to supply the necessary evidence to get round Britain's tough divorce laws.

Then two things happened. The divorce laws were reformed - which meant the bottom fell out of the market. But at the same time home movie cameras became cheap and available. Private detectives began to spend their time hiding round corners and behind bushes - recording what their suspects got up to.

Here are two clips I've put together. The first is one of the old-school private detectives going to a hotel room in Brighton to "surprise" the occupants. Followed by a wonderful item from 1973 where one of the new breed shows how he can film people without them noticing. Or so he says. From the evidence you'd doubt it.

He mostly works for the insurance companies - following people and filming them to see if they are faking an injury they are claiming for. I love the 8mm cameras he uses.

The item also includes an interview with a man who is opposed to this snooping. The interviewer says surely they are just trying to find the truth - that a film can't lie. The man's response is great:

"A film can lie very easily - the insurance company or the investigator can edit the film. Supposing someone has a bad limp that only occurs on wet days, or it's a nervous spasm that comes on some days rather than others. The film is shown in court - and shows only the good days when there's no limp"

It's wonderfully silly - but he has a point. Bit like documentary films.

Then - in the early 1970s - the private detectives found they could buy another kind of technology really cheaply.

Bugging equipment.

A new business grew up - often based in tiny rooms above electronic hardware shops in central London. An odd collection of electrical engineers and refugees from the music industry spent their days soldering together miniature transmitters and microphones - and selling them to the private investigators.

Here's a really good film made about this new world in 1973. It not only reports on what is happening - but also catches the essence of what was coming. Most of the film is just set in one room where there are three hidden bugs as well as the normal camera and microphone recording the presenter who is called Linda Blandford. But she doesn't know where they are.

The film evokes the strange repetitive nature of an enclosed world where everything is recorded and played back. Way ahead of its time. It's a smart bit of reporting.

But then - at the end of the 1970s - people began to get worried. It began with revelations that the security agencies were eavesdropping not just on enemy spies but on their own people. Trades unions, radical journalists, politicians had all had their phones bugged.

It quickly spread to a wider concern about all the snooping and bugging that was going on, not just by the state but by private investigators, and by journalists. It was the start of the concern that Britain was becoming a "surveillance society".

Here is a bit report from that time about the growing fears. By now the private detective had become a man in a phone box blowing a harmonic whistle into the mouthpiece.

Journalists also started to get keen on all this new technology. It allowed them to snoop and listen to people in new ways. Here is great section from a fly-on-the wall documentary made about the News of the World in 1981.

There's a wonderful assistant editor who is convinced that Special Branch is bugging his phone. While reporter David Potts is testing his bugging equipment that's going to be used by Tina the junior reporter to expose a child sex ring in North London.

What then happens to Mr Potts' scoop is very funny. And it shows how difficult it was back then to bug someone. It's obvious that what they needed to find was an easier way of snooping on peoples' lives.

In 1987 the growing paranoia finally burst out. The trigger was a BBC television series called The Secret Society made by an investigative journalist called Duncan Campbell.

In 6 half-hour films Campbell pulled what had been happening all together - and drew a frightening picture that still haunts the imagination of the liberal left.

Not only were the security services and the police secretly watching and listening to you - but dark elements of the "security state" had a corrupt relationship with the private security world. The films showed how investigators could easily buy confidential information on anyone.

And at the same time other secret bureaucracies were building giant listening networks - and keeping them hidden from politicians. One of the episodes was about the plans to launch a spy satellite called Zircon. Campbell revealed that the project had been kept hidden from the very politicians who were supposed to oversee it.

The government and the head of GCHQ panicked and put enormous pressure on the BBC, who caved in and said they wouldn't transmit the episode. It was an enormous scandal - and it seemed to prove dramatically everything that Campbell was saying about the secret state who watched you - but didn't want you to know things.

Here are some extracts from the series. In one bit Campbell reveals how, as well as the state, the private sector have developed huge computer databases full of information about millions of ordinary people. In a great sequence he goes to a market in Knaresborough in Yorkshire and asks people if they'd like to see what these databases know about them.

Their reactions of horror to what they are shown are so innocent. It's like a lost world.

I've also included a brief bit from the Zircon film - so you can see what all the fuss was about. It didn’t remain banned for long - and has been shown since.

Looking back you can see how programmes like the Secret Society were part of the growing distrust of those who governed us. They seemed to prove that there were hidden, unaccountable and corrupt forces at the heart of the British state.

And the paranoia about surveillance carried on growing.

But at the very time as this happened - a new system of watching and monitoring people rose up. It would do pretty much what the spies and the private detectives had been trying to do - but much much more. It would record not just all our actions - but also be able to understand what was going on inside our heads - our wishes, our desires and our dislikes.

It was called the internet.

The problem was that the only way for the systems on the internet to work would be with our willing collusion. But rather than reject it - we all embraced it. And it flourished.

The key to why this happened lies in an odd experiment carried out in a computer laboratory in California in 1966.

A computer scientist called Joseph Weizenbaum was researching Artificial Intelligence. The idea was that computers could be taught to think - and become like human beings. Here is a picture of Mr Weizenbaum.

There were lots of enthusiasts in the Artificial Intelligence world at that time. They dreamt about creating a new kind of techno-human hybrid world - where computers could interact with human beings and respond to their needs and desires.

Weizenbaum though was sceptical about this. And in 1966 he built an intelligent computer system that he called ELIZA. It was, he said, a computer psychotherapist who could listen to your feelings and respond - just as a therapist did.

But what he did was model ELIZA on a real psychotherapist called Carl Rogers who was famous for simply repeating back the the patient what they had just said. And that is what ELIZA did. You sat in front of a screen and typed in what you were feeling or thinking - and the programme simply repeated what you had written back to you - often in the form of a question.

Weizenbaum's aim was to parody the whole idea of AI - by showing the simplification of interaction that was necessary for a machine to "think". But when he started to let people use ELIZA he discovered something very strange that he had not predicted at all.

Here is a bit from a documentary where Weizenbaum describes what happened.

Weizenbaum found his secretary was not unusual. He was stunned - he wrote - to discover that his students and others all became completely engrossed in the programme. They knew exactly how it worked - that really they were just talking to themselves. But they would sit there for hours telling the machine all about their lives and their inner feelings - sometimes revealing incredibly personal details.

His response was to get very gloomy about the whole idea of machines and people. Weizenbaum wrote a book in the 1970s that said that the only way you were going to get a world of thinking machines was not by making computers become like humans. Instead you would have to do the opposite - somehow persuade humans to simplify themselves, and become more like machines.

But others argued that, in the age of the self, what Weizenbaum had invented was a new kind of mirror for people to explore their inner world. A space where individuals could liberate themselves and explore their feelings without the patronising elitism and fallibility of traditional authority figures.

When a journalist asked a computer engineer what he thought about having therapy from a machine. He said in a way it was better because -

"after all, the computer doesn't burn out, look down on you, or try to have sex with you"

ELIZA became very popular and lots of researchers at MIT had it on their computers. One night a lecturer called Mr Bobrow left ELIZA running. The next morning the vice president of a sales firm who was working with MIT sat down at the computer. He thought he could use it to contact the lecturer at home - and he started to type into it.

In reality he was talking to Eliza - but he didn't realise it.

This is the conversation that followed.

But, of course, ELIZA didn't ring him. The Vice President sat there fuming - and then decided to ring the lecturer himself. And this is the response he got:

Vice President - “Why are you being so snotty to me?”

Mr Bobrow - “What do you mean I am being snotty to you?”

Out of ELIZA and lots of other programmes like it came an idea. That computers could monitor what human beings did and said - and then analyse that data intelligently. If they did this they could respond by predicting what that human being should then do, or what they might want.

They key to making it work was a system called Boolean Logic.

It had been invented back in 1847 by a mathematician called George Boole. One day he'd been walking across a field near Doncaster when he had what he described as a "mystical experience". Boole said that he felt he had been "called on to express the workings of the human mind in symbolic or mathematical form".

Boole's idea was that everything that went on in the human mind could be reduced to a series of yes or no decisions that could be written out on paper using symbols.

His idea was pretty much ignored for over a hundred years - except by Lewis Carroll who as well as writing Alice in Wonderland, wrote a book called Symbolic Logic - that laid out and developed Boole's ideas.

But when computers were invented people immediately realised that Boole's idea could be used to allow the computers to "think" in a reasoned way. Computers were digital - they were either 0 or 1 - and that was the same as "yes" and "no". So Boolean Logic became central to the way computers work today. They are full of endless decision trees saying "if this happened then this, and not this".

Here is a picture of George Boole taken in 1864. It was just before he died and it is one of the earliest portrait photos - he'd stopped off at the new London School of Photography at 174 Regent Street.

In the early 1990s researchers became convinced they could get computers to predict what people might want.

It started in 1992 with a small unit set up in the University of Minnesota. They called themselves GroupLens. Their idea was that if you could collect information on what people liked and then compare the data, you would find patterns - and from that you could make predictions.

They called it "Collaborative Filtering" - and the logic was beautifully Boolean. As one researcher put it -

"If Jack loves A and B and Jill loves A, B, and C then Jack is more likely to love C."

They began by comparing the news articles that people recommended in online newsgroups-

"GroupLens monitored user ratings of news articles. After a user had rated several items GroupLens was able to make recommendations about other articles the user might like. The results were astounding. Users read articles that we recommended highly three to four times as often as those we didn't"

Then, in 1994, a young professor at MIT did the same with music. She was called Pattie Maes - and she designed a system called RINGO. She set up a website where people listed songs and bands they liked. One user described how it worked

"What Ringo did was give you 20 or so music titles by name, then asked one by one whether you liked it, didn't like it, or knew it at all. That initialized the system with a small DNA of your likes and dislikes. Thereafter, when you asked for a recommendation, the program matched your DNA with that of all the others in the system. If some of the matches were not successful - saying so would perfect your string of bits. Next time would be even better"

Again it worked amazingly well. And Maes started to do the same with movies. Then the University of Minnesota group had a brainwave. If these systems could tell you what articles and songs you would like - why couldn't they tell you what products you would like as well?

So in 1997 they set up a company called Net Perceptions. And one of their first clients was Amazon.

But one of Amazon's young software engineers called Greg Linden soon realised that there was a problem with these systems. You had to spend all your time finding out what people said they liked. And as the systems became bigger and bigger - this was proving incredibly cumbersome.

Plus - people were fickle and they changed their mind a lot. Or - in computer engineer speak - they were "dynamic".

Linden saw what the solution was. You give up finding out what people said they liked and instead you just look at what they've done in the past. You assembled all the data from people's history - all the stuff they've looked at and bought in the past - and then compared that with other peoples' past.

Out of that came patterns and correlations that the human brain could not possibly see - but from those correlations you could tell what individuals would want in the future.

Linden was part of what was called The Personalization Group in Amazon. He said:

"the joke in the group was that if the system were working perfectly, Amazon should just show you one book - which is the next book you are going to buy."

And it worked - sales soared, and Jeff Bezos who runs Amazon allegedly crawled up to Linden on his hands and knees saying "I am not worthy".

What Amazon and many other companies began to do in the late 1990s was build up a giant world of the past on their computer servers. A historical universe that is constantly mined to find new ways of giving back to you today what you liked yesterday - with variations.

Interestingly, one of the first people to criticise these kind of “recommender systems” for their unintended effect on society was Patti Maes who had invented RINGO. She said that the inevitable effect is to narrow and simplify your experience - leading people to get stuck in a static, ever-narrowing version of themselves.

Stuck in the endless you-loop. Just like with ELIZA

But like so much of the modern digital world - these new systems are very abstract. And there is little to see that happens apart from endless fingers on keyboards. So it's difficult to bring these effects into any kind of real focus.

Last year - in a live show I did with Massive Attack - we tried to evoke this new world. We used a song from the 1980s called "Bela Lugosi's dead" - which I love because it has a very powerful feel of repetition. The audience were surrounded by 11 twenty-five foot high screens.

I'm not sure how successfully we did it - but what I was trying to show is how your past is continually being replayed back to you - like a modern ghost. And it means we stand still unable to move forwards. Like a story that's got stuck.

I’ve put a short bit of it together from some camera-phone videos shot by the audience in New York. It’s a bit rough - as is the sound - but you’ll get a sense of it.

For all the online companies that use these systems, the fact that they tend to inhibit change is an unintended consequence.

But there are other - more powerful systems that grew up in the 1990s whose explicit aim is exactly that. To prevent the world from changing, and hold it stable.

And they operate in exactly the same way - by constantly monitoring the world and then searching their vast databases for patterns and correlations.

ALADDIN is the name of an incredibly powerful computer network that is based in a tiny town called East Wenatchee - it's in the middle of nowhere in Washington State in North America.

Aladdin guides the investment of over $11 trillion of assets around the world.

This makes it incredibly powerful. Aladdin is owned by a company called Blackrock that is the biggest investor in the world. It manages as much money as all the hedge-funds and the private equity firms in the world put together. And its computer watches over 7% of all the investments in the world.

This is unprecedented - it's a kind of power never seen before. But Blackrock is not run by a greedy, rapacious financier - the traditional figure of recent journalism. Blackrock is run by the very opposite - a very cautious man called Mr Fink

Here he is. He's called Larry Fink

Back in 1986 Mr Fink was working his way up the First National Bank of Boston when an unpredicted fall in interest rates caused a disaster for the bank. He swore that it would never happen again - and for 20 years he built Aladdin.

It has within its memory a vast history of the past 50 years - not just financial - but all kinds of events. What it does is constantly take things that happen in the present day and compares them to events in the past. Out of the millions and millions of correlations - Aladdin then spots possible disasters - possible futures - and moves the investments to avoid that future happening.

I can't over-emphasise how powerful Blackrock's system is in shaping the world - it's more powerful in some respects than traditional politics.

And it raises really important questions. Because its aim is to not change the world - but to keep it stable. Preventing any development thats too risky. And when you are moving $11 trillion around to do that -it is a really important new force.

But it's boring. And there is no story. Just patterns.

Here is some video of Aladdin. A few weeks ago I was filming in Idaho - and decided to go and have a look at the buildings that house Aladdin. I had asked Blackrock if I could have a look inside. Surprisingly the guy in charge of their PR said yes. But a little while later he left the company in what seemed to be a reorganisation.

But it didn't really matter - because you know what it will look like. Row upon row of servers roaring away, and surrounded by giant batteries that will rescue the system if the power supply goes.

Here's the shot from the car driving past the computer sheds that house Aladdin. A 37 seconds tracking shot, and you can see how dull it is.

It is the modern world of power - and it's incredibly boring. Nothing to film, run by a cautious man who is in no way a wolf of Wall Street. It's how power works today. It hides in plain sight - through sheer boringness and dullness.

No wonder we find it difficult to tell stories about it.

There are also a growing number of systems that use data from the past to predict whether individuals are going to commit crimes in the future.

On the surface it's laudable. But it's also rather weird - and in some cases can be false and dangerous.

In every case the systems monitor individuals' behaviour and then sees if that shares similar characteristics with groups of other people stored on the databases who have behaved dangerously in the past.

There is software being used by the Department of Work and Pensions that detects fraudsters by analysing the voices of people who ring its call centres. If you ask the wrong kind of questions - or even ask the right kind of questions in the wrong way - it puts you in the dangerous group.

The government also has what they call a Social Exclusion unit which has an Action Plan. It's aim is to use data to predict when things might go wrong in poor families - even before birth. In one scheme the unborn child of a pregnant mother might be categorised as potentially being a future criminal.

This is based on things like the mother's age, her poor educational achievements, her drug use and her own family history. If the system decides that the unborn child is a potentially dangerous criminal the response is not exactly Philip K Dick - a nurse is sent round to give advice on parenting.

But the oddest is STATIC-99. It's a way of predicting whether sex offenders are likely to commit crimes again after they have been released. In America this is being used to decide whether to keep them in jail even after they have served their full sentence.

STATIC-99 works by scoring individuals on criteria such as age, number of sex-crimes and sex of the victim. These are then fed into a database that shows recidivism rates of groups of sex-offenders in the past with similar characteristics. The judge is then told how likely it is - in percentage terms - that the offender will do it again.

The problem is that it is not true. What the judge is really being told is the likely percentage of people in the group who will re-offend. There is no way the system can predict what an individual will do. A recent very critical report of such systems said that the margin of error for individuals could be as great as between 5% and 95%

In other words completely useless. Yet people are being kept in prison on the basis that such a system predicts they might do something bad in the future.

Opposition Research - the constant recording of everything a politician says and does fits into the same pattern.

But it's a system of wonk-driven surveillance that goes even further - because it has the unforeseen consequence of forcing politicians to behave like machines. It leads them to constantly repeat what they said yesterday, and unable to make imaginative or creative leaps

Every night the digital tracker sends back what that politician said or did today. The first aim is to find something outrageous in that day's video that can be given to the media.

Here is one classic example. It is Jon Bruning who compared welfare recipients to racoons. His speech up to that point is actually quite a funny right-wing attack on what he sees as the absurdity of environmental protection. But then he went too far.

And he was shamed. And he lost the election.

But the other - bigger - task of the opposition researchers is to spend hours comparing what the politician said today with their recorded past that is stored in the computers. They look for contradictions and if they find one they release the videos to the media and again the politician is shamed.

So the politicians become frozen and immobile - because they have to have a blameless history. Which again seems laudable. But it means they can't change their mind. They can't adapt to the world as it changes.

Although if ALADDIN has its way that won't matter

George Boole - who helped start all this with his Boolean Logic had an extraordinary family.

One of them, his son-in-law was called Charles Howard Hinton. He too was a mathematician and he became famous at the end of the nineteenth century when he wrote a book called The Fourth Dimension.

It said that time was an illusion. That everything that has happened and that will happen already exists in a four-dimensional space. Human beings, Hinton said, don't realise this because they don't have the ability to see this four-dimensional world.

Our idea of time - Hinton said - is just a line that goes across this four-dimensional space like a cross section. But we can't see it.

The cumulative effect of all today's systems that store up data from the past is to create something rather like Hinton's world. Everything that has already happened is increasingly stored on the giant servers in places like East Wenatchee.

It never goes away. And this past bears down on the present - continually being replayed to try and avoid anything that is dangerous and unpredictable.

What is missing is the other half of Hinton's world. The future - with all its dangers, but also all it's possibilities.

But George Boole had another daughter called Ethel. She had an amazing life - which showed that there is another way. Because Ethel believed in the future.

Here she is in a Boole family photo - taken after her father died. Ethel is to her mother’s right. (Incidentally the rest of the Boole family that you see in this photo also had amazing lives - but that’s another story)

When Ethel was 15 she read a book about the Italian revolutionary Mazzini. It inspired her - and she wore clothes like him, dressing in black in mourning for the state of the world.

In 1889 she met a Polish revolutionary called Wilfred Michail Voynich. He had escaped from Siberia and had arrived penniless in London. They fell in love and married, and Ethel went off to Russia to smuggle in illegal revolutionary publications.

Then she met the master-spy Sydney Reilly. He is one of the most extraordinary figures in the odd world of espionage. He'd been born in the Ukraine, but turned against his family and faked his own suicide to escape.

After all kinds of adventures, including rescuing three British intelligence agents from the swamps of the Amazon jungles, Reilly went to London where he spent his time gambling - and he and Ethel began a passionate affair. They eloped to Italy where Reilly bared his soul to Ethel - telling her the extraordinary story of his life.

Then Reilly deserted her - and went off to Russia where he worked as a secret agent for the British. Ian Fleming is said to have used Reilly as the model for James Bond.

Ethel was heartbroken - and she wrote a novel called The Gadfly which, although she never admitted it, her biographer says is obviously based on the early adventures of Sydney Reilly.

It's the most amazing book. It's an over the top melodrama set in Italy about the hero, Arthur's battle against the church and the corrupt state - and his treacherous family. At the same time it is about his passionate love for an english girl - Gemma. It ends with Arthur being slowly tortured and then condemned to be shot.

Its message though is a revolutionary one. Arthur is sacrificed so that humankind can be redeemed and open the way to a realisation of the future possibilities for the world - once the old oppressive forces have been overthrown.

Here is Ethel with a wonderful revolutionary look in her eyes

The Gadfly was published in 1897 in New York - under Ethel's married name, E.L. Voynich. No British publisher would touch it because of its "outrageous and horrible character". But then it was published in Russia and became an astonishing success. One writer describes how all the young Bolsheviks read it and "it virtually became the bible of the revolution".

By the 1960s it was estimated that 250 million Russian teenagers had read the Gadfly in translation. And polls showed that Arthur was consistently the favourite hero of Soviet youth. And in 1955 a film version was made - with a soundtrack by Shostakovich - which won an award at the Cannes film festival.

In 1920 Ethel went back to her husband Wilfred Voynich. He had moved to New York and had become one of the world's greatest expert and dealers in rare books.

His most famous purchase was a mysterious manuscript written in code that has come to be known as The Voynich Manuscript. No one has ever been able to break the code - it seems to have many scientific references, and herbal and astronomical illustrations.

Voynich believed that it was written by the philosopher Roger Bacon - and then came into the possession of the legendary John Dee who was a mathematician at the court of Queen Elizabeth.

After Voynich died, Ethel kept the manuscript in a safe deposit box in New York for thirty years - and then sold it in 1960. And it ended up in Yale University. One of the great experts in cryptography wrote:

"The Voynich manuscript lies quietly inside its slipcase in the blackness of Yale's vaults, possibly a time-bomb in the history of science, awaiting the man who can interpret what is still the most mysterious manuscript in the world."

Ethel Boole died in 1960 at the age of 96. Still believing in the power of revolution to change the world. Here is one of the most beautiful sections of Shostakovich's music for the Gadfly - cut to images of the strange Boolean world that we live in today.

SUSPICIOUS MINDS

2014-04-02T13:06:17+00:00

Nobody trusts anyone in authority today.

It is one of the main features of our age. Wherever you look there are lying politicians, crooked bankers, corrupt police officers, cheating journalists and double-dealing media barons, sinister children's entertainers, rotten and greedy energy companies and out-of-control security services.

And what makes the suspicion worse is that practically no-one ever gets prosecuted for the scandals. Certainly nobody at the top.

There has always been Us vs Them in modern Britain - but this pervasive mood of suspicion and distrust is different.

In the past it divided along political lines. The Left was for Us and the conservative Right was firmly for Them. But now the politics have disappeared - because no politicians are trusted. It doesn't matter whether they are left or right, all politicians are despised. They will never do anything for the ordinary person - only for themselves and their other corrupt friends in power.

In some ways this is disempowering because it means there is no-one who is both powerful and trustworthy enough to challenge the corruption. But it is also a moment of great opportunity - because the present mood of distrust with authority is very powerful and it could be harnessed to create a new populist movement. This is what someone like Russell Brand has sensed - and is trying to do.

I want to go back and look at the roots of this tearing down of politics and of authority in modern Britain. To do this I am going to tell the story of three rather odd men who in their very different ways helped begin it over thirty years ago.

They are only a small part of a much wider history. But what makes them interesting is that their peculiar stories shine a powerful light on the hidden roots of today's mood of distrust. How really it was taken up with equal enthusiasm by both the political left and the right.

And what makes them even more interesting is that all three are also rather untrustworthy themselves.

The first one is a man called Stephen Knight. He is pretty much forgotten today, but back in the late 1970s he was a journalist and a best-selling author.

Knight is important in this story because his books exposed what he said were hidden conspiracies at the heart of the British establishment. But there is also a surprising, and rather sad, way to get a sense of him as a person.

In 1980 he answered an advertisement in the London Evening Standard. It had been put there by the BBC TV science department and it asked for people who suffered from epilepsy if they would take part in a documentary about the disease.

Knight had suffered epileptic fits for over three years. They were getting worse - and he wanted to know why - so he agreed to take part. Here is a section from the early part of the programme where the BBC filmed him talking to a psychiatrist who specialised in epilepsy.

Knight had been born in 1951 in Essex, left school at sixteen and started working as a salesman in the London Electricity Board offices in Chigwell. But he was determined to be a reporter and at 18 got a job on the local paper - called The Ilford Pictorial.

Knight was very much of his time. A suburban boy, confident and starting to question the paternalism that still wrapped its patronising arms round so much of society in Britain in the early 1970s.

And much of that questioning came from the suburbs - like Ilford. Here is a documentary made in 1969 about how a group of squatters have descended on Ilford and taken over a row of houses in protest against them being kept empty.

It is a big story - and Stephen Knight would have been there as a cub reporter. What's interesting is how the squatters deliberately refuse to be part of old left-wing class politics. Their battle is with the hypocritical council who pretend to care for people but are keeping the houses empty.

It starts with the squatters gathering material to make barriers. They then go to an Indian restaurant where the leader of the squatters holds court. He's called Ron Bailey and he's the star of the film. Ron has arranged for a large demonstration the next day to support the squatting.

It's a great bit of film-making that sharply captures a new mood. It's also very funny. There is a great moment when a post-office engineer turns up and says that the main house the squatters have occupied is not a house but is technically a government office. This causes some confusion - but Ron is is not fazed, his reaction is brilliant.

Stephen Knight did well and moved to the Hornchurch Echo. Then one night he watched a film on the BBC that transfixed him. It was an investigation into the famous Ripper murders in London's east end in 1888.

The programme was presented by two fictional detectives - who appeared in the series Z-Cars - and was full of dramatised reconstructions. But at the end it showed a strange filmed monologue by a real person called Joseph Gorman. He claimed to be the son of the famous painter Walter Sickert (although whether Gorman was really who he said he was would come to haunt Stephen Knight later).

Gorman said that the Ripper was not the mad serial killer that everyone thought - but that the five women had been killed on the orders of those in power in Britain at the time because they knew a terrible secret about the Royal Family.

Here is Gorman - preceded by a bit of the two fictional detectives, so you can get an idea of the oddness of the programme itself.

Gorman said that Queen Victoria's grandson had had an affair with a lowly artist's model, and they had a child. When the Queen and the Prime Minister Lord Salisbury found out about this they decided to cover it up. But a prostitute called Mary Kelly, who knew the secret, tried to blackmail the royal family.

So a group at the top of British society decided to have Kelly killed - along with a group of other prostitutes so as to make it look like the work of a maniac. But, Gorman said, the child survived.

Stephen Knight went to see Joseph Gorman who lived in a shabby flat in North London. He sat for hours listening to Gorman's story. He was fascinated - but was also very suspicious. Knight then went and did lots of research, and he kept on discovering unexplained coincidences that convinced him.

In 1975 he published a book that exposed the conspiracy. It was called Jack the Ripper: The Final Solution. It was an immediate best seller. It grabbed the public imagination not just because of the story he told, but because Knight also used the idea of a high-level conspiracy to bring into view a hidden force he said was at the heart of British society.

"Who played the Administration and who the Opposition at any given moment was of minor importance when Britain's entire political system seemed threatened.

What had to be made secure was the sacred Tory-Liberal merry-go-round - the Establishment - which, according to Sickert, had always to rank more importantly in Salisbury's list or priorities than any number of individuals. Hence the impending measures against the women."

What Knight was saying was that the idea there was a left and right battling it out in politics was just a sham. It was a disguise that hid the real network of power and control in Britain - a hidden group at the top of the establishment that looked after itself.

It's important to realise that the idea of "the Establishment" in Britain had only been put forward twenty years before - in 1955 - by a journalist called Henry Fairlie. He had described it as a linked matrix of powerful groups that you couldn't really see in Britain - but you knew were there.

Knight was using the Ripper conspiracy to transform that notion into a powerful cultural myth. It said that the real battle was not in traditional politics, but was against this entrenched group whose informal network stretched into all areas of authority in Britain - be they politicians, senior policemen, or the medical profession.

They pretended to care about you - but really they'd kill you if you threatened their power. Like they had in 1888.

At the same time as Stephen Knight was writing his Ripper book, something else happened in Ilford.

The showroom of the London Electricity Board was robbed by five masked gunmen. It was a dramatic raid because the gunmen shot a policeman, then ran over another - and were chased through east London in a car they hi-jacked at gunpoint.

In January 1975 four men were put on trial for the robbery. One of them is the second man in this story. He was called George Davis.

Here is a report of the raid.

Two of the accused were acquitted. The jury couldn't decide on another. Only Davis was convicted and sent to prison for twenty years.

Davis was not a sympathetic character. He was a petty criminal from Bow in east London who was also a mini-cab driver. But he insisted he was innocent. He said that the police had faked his statement by inserting words he had never said.

But no-one believed him

Except for his wife Rose, and his best friend called Peter Chappell who said he had seen Davis driving his mini cab at the time of the robbery. But no-one believed him either. So Chappell started a bizarre campaign. He drove a van into the front windows of various national newspapers - including the Daily Telegraph. He also smashed it through the gates of Buckingham Palace.

And just to make his point he put his van on the cross-channel ferry, went across to Paris and smashed the windows of the British Embassy.

Then a journalist on the Observer called Robert Chesshyre got interested. But he had to convince his editor. Chesshyre later described the meeting - it's a moment that captures the shift that was beginning in the mindset of the liberal middle classes.

"First I had to convince the Observer's editor, David Astor. He had moulded the Observer as the intelligent liberal flagship of the postwar press, but his interests were, on the whole, affairs of state, national policy, topics of concern to the great and the good.

Would he buy this tale of a hoodlum and his campaigning wife and friend? Astor had two main questions. Did I genuinely think Davis had been wrongfully convicted? And, if I did, was it of consequence, set against the lofty events with which the paper then mainly concerned itself?

Sucking a mint, Astor pondered. It wasn't his sort of story he said finally, but he could see that times were changing and that if a man had received 20 years for a crime he hadn't committed, we shouldn't sleep easy until he'd been freed."

Chesshyre then went off to investigate - and what he discovered was rather odd. He described it in a long article that the Observer splashed on their front page:

"There was no forensic link between Davis and the robbery. The police relied on identification evidence and Davis' supposed statement, which he claimed contained words made up and inserted by a police officer.

Davis was picked out at an ID parade by three policemen, all of whom had been travelling in the same car. At a second set of parades held three months after the crime, 34 out of 39 witnesses failed to pick out Davis. And three made wrong identifications.

Two 'mistakes' were made after Davis swapped his shirt with another man, who was then identified - an odd event at the very least."

Criminals had always known that the police would fit them up if they needed a conviction. But the Observer's reporting, and Chappell's dramatic campaign crystallised a growing feeling among the liberal classes - that maybe those in authority weren't as good as they thought. That maybe the police lied - and sent innocent people to prison.

Chappell carried on with his stunts. He dug up the cricket pitch at Headingley the night before a crucial test match. And there were big marches full of the different groups that were emerging because of the disenchantment with traditional politics - punks, radical lawyers, revolutionaries, east-end squatters and artists, and lots and lots of ordinary people who were shocked by the miscarriage of justice.

Rose Davis called these groups "soap avoiders". She said - "they are very intellectual, but most of them never wash".

But they did her - and her husband - proud. A radical playwright even wrote an agit-prop play about George Davis which was put on at the Half Moon Theatre in the east end. It caused a great fuss and the BBC's arts programme Arena reported on the first night - and interviewed some of the audience about their reactions.

Here's the item. I've included the introduction because it tells you a great deal about the campaign and its supporters - both the dress of the presenter, and her great phrase:

"this really is theatre at the barricades"

Under this enormous pressure the Home Office brought in an outside police force to investigate. What they discovered was so shocking that the Home Office refused to release the report to anyone for 34 years - even to Davis' lawyers.

The Home Secretary, Roy Jenkins, ordered that Davis be released from prison - and Davis travelled back to London on a train full of reporters and film crews. It was a hero's return.

Here is Davis on the train - and arriving at Waterloo station.

George Davis' release was just the beginning of a wave of revelations about how the police had ignored the rules, altered and hidden evidence, and faked statements and confessions. Scandals that would include the Guilford Four and the Birmingham Six - and that brought not just the police into question, but revealed how some of those at the very top of the judicial system had known for years that the convictions were unsafe.

It was a bit like what Stephen Knight was saying. That there was an interlinked group at the top of British society who said they worked only for the public good - but really looked after themselves, even if it meant ordinary people were left to rot in jail.

But the experience changed George Davis. Here is an interview with his wife, Rose. She describes how he became a celebrity in this new revolt against those in charge in Britain - but also how she began not to trust him any longer.

But it wasn't just the disaffected left and liberals who were challenging authority. There was an assault simultaneously coming from parts of the conservative right. One of the leading figures in this was Rupert Murdoch who was also convinced that many of those in charge in Britain were hypocrites.

The man who allowed Murdoch to demonstrate this hypocrisy in a dramatic way is the third man in this story. He was called Lord Lambton. But what the events really revealed about power in Britain was far odder and more unsettling than Rupert Murdoch realised.

Lord Lambton was a minister of defence in the conservative government in the 1970s. His family was one of the highest in the land - his ancestor "Radical Jack" Lambton had been behind the Great Reform Act of 1832 which had removed corruption and allowed ordinary people to freely vote for the politicians they wanted.

Lord Lambton was obsessed by his title. When his father died in the 1970 he had wanted to remain as an MP - but that meant he had to give up being an Earl. But he insisted on still being called Lord Lambton. The speaker of the house said that he couldn't, but Lambton insisted he could - and there were endless investigations which couldn't come to any firm conclusion.

In his spare time Lambton went to spend afternoons with a prostitute in a flat in Maida Vale. She was called Norma Levy, and she had quite an odd collection of clients. They included Billy Butlin and John Paul Getty - who used to get her to lie naked in a coffin while he stood in his underpants just watching her.

Lambton was a lot easier - as well as straight sex he just liked smoking cannabis and chatting away to her. But Norma had a devious husband called Colin who decided to make some money. With the help of a reporter form the News of the World, he hid a camera in the wardrobe - and took pictures of Lambton in bed with Norma.

The News of the World also hid a microphone in the nose of Norma's teddy bear.

Initially Rupert Murdoch was hesitant - he was worried about the impact such an out and out attack on the government would have. But then he published the story - although he didn't print the photograph, allegedly because of pressure from the government.

Murdoch justified the story like this:

The revelations caused a sensation. And the News of the World produced a great character called Mariella Novotny who was interviewed by the BBC about there was a hidden network of call girls that used a model agency as a front - and had secret lists of those high up in the British establishment.

Novotny has great make up - and I love her observation:

"I've noticed, and found, the higher a man is work-wise, the more blase and pompous he is"

Lambton disappeared - but a few days later he surfaced and invited the BBC's most famous interviewer, Robin Day to come up to the Lambton's stately home in Northumberland.

What then followed was one of the most extraordinary interviews. You would never get anything like it today. Lambton was completely frank and open about what he did. But he does this completely from the point of view of him and his class - and in the process he reveals a brutal self-centred arrogance - especially in how he describes Norma Levy.

Lambton's interview confirmed Rupert Murdoch's view that those who ran the country were a patronising and hypocritical elite.

But the truth about the elite was much stranger and more unsettling. Really those who ran Britain were sleep-walking through a dream world. And many of them had lost touch with reality. And the next stage of the Lambton scandal showed this clearly.

Because Lambton was a defence minister, MI5 were worried there had been a security breach. So a senior MI5 officer called Charles Elwell turned up to interview him. Elwell wrote down the conversation - and Lambton began with an unexpected defence:

"He (Lambton) had never taken any of his papers out of the office. Indeed, he had no need to do so since he had so little work to do. He rather implied that the futility of the job was one of the reasons that he had got up to mischief (idle hands etc)."

I love - "Idle hands etc"

But then, suddenly - almost as if he was embarrassed by what he had just said and wanted to cover it up - Lambton gave Elwell another reason:

"He said that he had throughout his life made use of prostitutes from time to time, but that his behaviour since July 1972 was out of character and had been caused by his obsession over his failure to win his battle to use his title.

This had become an obsession with him to the extent that he was no longer able to read - and he had been a great reader - and had sought to forget his obsession in frantic activity. He had become an enthusiastic and vigorous gardener. Another example of this frenzied activity was his debauchery."

But the MI5 agent sitting opposite Lambton also didn't have anything to do.

Although MI5 told their political masters that Britain was full of Soviet agents - in reality they couldn't find any. Ever since 1971 when over a hundred Russian embassy officials had been deported, the security agency had turned up nothing.

Here is part of a wonderful film that MI5 made to warn of the danger that they were convinced was there. It was shown to anyone who had any access to secret material. The dialogue is brilliant - I'm sure it was written by an MI5 agent.

But the truth was that wherever they looked, the security agents found no hidden danger.

This had an odd effect inside MI5 and Charles Elwell's reaction was typical. He just couldn't believe it - so he convinced himself that mild, liberal organisations like the National Council for Civil Liberties and even the housing charity Shelter were really Soviet-run organisations. (Idle hands etc.)

Elwell's bosses in MI5 began to realise that this was madness. Or as the Daily Telegraph wrote later - "he was over-inclined to see subversion where none existed". Elwell was passed over for promotion.

Which, of course, proved that the MI5 bosses were really Soviet agents themselves

In the late 70s Elwell and a number of other MI5 agents convinced themselves that those who ran the agency were really under Soviet control - as were senior politicians. To get a sense of the crazed mood within the security establishment at the time here is the ringleader of this mad group - Peter Wright - talking about their plot to bring down the then Prime Minister, Harold Wilson

By 1980 those in authority were being attacked from all sides. From the left - like Stephen Knight, from the right - like Rupert Murdoch, as well as from within - by the paranoid suspicions of the security services.

If one looks back now it is possible to see it as a kind of revolution in which both left and right were collaborating to overthrow an old, decayed patrician culture. And as it grew in the 1980s and 90s it would give rise to something that went beyond politics - to a general culture of suspicion and distrust of everyone in authority.

But the roots of that revolution were a little fragile.

Eighteen months after George Davis was released from jail as an innocent man - he was arrested again for taking part in an armed bank raid in North London. This time there was no doubt - Davis was found red-handed sitting in the getaway van, with weapons next to him on the front seat.

It was a bit of a shock to his supporters - and above all to his wife Rose. She gave a very powerful interview in which she expressed the deep sense of betrayal she now felt.

Meanwhile Stephen Knight decided he would prove that the British police, and many other groups in the present-day establishment, were secretly run by a hidden network of Freemasons. And he started writing a book that would, he said, expose this

But he too had a bit of a setback. The man he had relied on for his Jack the Ripper theory, Joseph Gorman, went to the Sunday Times and said that he'd made the whole thing up. Knight said that Gorman was lying. He really was Walter Sickert's son, he was just pissed off because he didn't like some of the things in the book.

It did seriously damage his theory.

Knight carried on though, and wrote his book about the police and the freemasons. Again it was a best-seller - because he had tapped into the growing mood of distrust of the police that George Davis had done so much to bring about.

It was very much a product of its time. The central theory about the extent of masonic penetration was very stretched - and at times straight bonkers. But the public were becoming increasingly aware that the police might well be seriously corrupt - and that other groups who might stop this seemed to do nothing. And there were lots of freemasons in the police.

It felt like the book made sense of the new distrust and confusion.

Here is Knight promoting his book on Breakfast TV. It's a great set-up, because the other guest is Dame Edna Everage lying in a large bed in the studio. Barry Humphries - who plays Everage - is very sharp, and you can see him getting interested in the theory.

At the end of the item Edna Everage tries to talk about it -

"It's spooky isn't it, the Freemasons? It intrigues a little bit. Because we live, don't we, in a world of disclosure. Now - you live in a modern world where even a mega-star like myself is filmed in bed at breakfast ......"

The presenter abruptly cuts her off. But it shows why Knight's book was capturing the public imagination.

Then Knight found another fan. He was the comic book writer Alan Moore. He read Knight's Ripper book - and saw something epic in it.

Moore wrote a book called From Hell. It tells the story that Knight put forward - but he turns it into something far grander. The Royal Physician, William Gull - who Knight said did the killings - becomes the centre not just of the vicious conspiracy, but a mad figure seeking to impose narrow order on a fluid world that he can only partially understand.

Even time is fluid. Events from other periods keep breaking through to Gull. The very way we perceive time and tell stories is part of the limitations imposed on us by a repressive society.

What Alan Moore was saying in From Hell was that we see the world in a narrow and limited way that is imposed on us by those in power. But that you can use the imagination to break free from that stranglehold - and make the world anything you want.

It was a very attractive message to a generation who now increasingly distrusted authority - and felt disempowered.

And Stephen Knight's own personal story in a strange way also carries the same message.

In 1980 he had agreed to take part in a BBC documentary about epilepsy. The producer of the film was keen to show a scan of an epileptic brain and Stephen Knight volunteered. He had been hit on the head by a cricket bat when he was a child - and the doctors believed that there was some dead tissue in his brain that was causing the fits.

What the scan revealed was much worse. There was a large shape the size of an egg in his brain. A biopsy was done - and it showed Knight had a cerebral tumour. He was immediately operated on.

Here is the part of the film where he has the scan. It is followed by Knight describing very eloquently what happens when he has a fit. He describes how something inside his brain that he cannot control takes him away from the safety of everyday perceptions to somewhere else.

It's a frightening experience that Knight describes - where "It", as he calls it, leads him to see his own death.

The programme shows how such visions are caused by giant electrical storms that sweep through the neural connections of the human brain.

Some people - hard AI believers - see this as proving that human brains are just complex electrical machines. But that is limited - it is like analysing a film by looking only at the projector.

What Knight is describing is close to what Alan Moore writes about in "From Hell". That the human mind has extraordinary powers of imagination when it is free of the limitations of normal perception.

The present day system of power - that has replaced the old patronising authority - is a new kind of limitation. It treats human beings themselves as very simple machines. Instead of telling them what to do, as the old power used to, the new system increasingly uses computers to read data about what human beings want or feel. And then fulfils those needs.

What Stephen Knight and Alan Moore are pointing towards is something different. How the human imagination has the power to conceive of worlds that have never existed before. And if that imagination can be integrated into a new kind of politics - then those worlds could be brought into existence.

It's unquantifiable and untrustworthy. But it's full of potential.

Stephen Knight died from his brain tumour in 1985 at the age of 33.

Rose Davis divorced George. He then married the daughter of a police inspector.

Lord Lambton went off to live in a villa in Italy - where he gardened frenziedly.

ONE'S PRIVATE LIFE

2014-01-19T15:33:27+00:00

Lots of people are gripped by the Francois Hollande affair - and they are gripped on two levels.

One is a fascination not just with the details - but with the fact that such a plain man can have such a glamorous love life.

The other is how strange it is that the French don't seemed to be gripped in the way we are. They seem to be weird and old-fashioned in their belief that there is a distinction between what goes on in a person's private life - and the details of their public life.

Underneath this is also a sense of nostalgia - a yearning for a time not so long ago when we in Britain were like that too. A time before horrible people like Rupert Murdoch came along and turned everything - even the most intimate details of a person's life - into cruel, public melodrama.

I'm not sure that's true.

I have stumbled upon a fascinating film made in 1967 about divorce - where members of the British upper and upper-middle classes talk in the most frank, gripping and sometimes incredibly moving way about the most intimate details of their private lives.

The film was nominally made because parliament was about to vote on changes to the divorce laws. But really it is an extraordinary factual drama where eight people who are at the heart of London high bourgeois life in the mid-1960s open out their personal lives to the camera.

It's a world that is very similar to the one surrounding Francois Hollande and Julie Gayet in Paris today. A world of famous novelists, public-relations men, beautiful models, world-weary aristocrats and the son of a famous explorer.

What makes the film so gripping is the structure. There are two stories - each one pivoted around one character. One is a famous novelist, the other is a beautiful "cover girl" and model.

Around each one are three other characters - the four making up the players in one divorce. They all talk about each other - sometimes very cruelly. And, in the second group of four, they describe what they are going through in incredibly sad ways that make you want to cry.

Together they build up the picture of the inner life of a powerful cultural elite - a world being turned inside out and shown to us in great emotional detail. And as you watch it you realise that something like this would never happen today on TV.

Despite all the talk about today's society being more open and transparent than the past - the reality is that those in power today are far more guarded and hidden than they were back then.

Four of the eight characters are the emotional heart of the film - and I have added some details of what happened to them later on over the following forty years. You may or may not want to know this before you watch the film.

ELIZABETH JANE HOWARD

Elizabeth Jane Howard was a novelist. She had married Peter Scott in 1942 - and they divorced in 1951. Then in 1962 she met Kingsley Amis, had dinner at his home with him and his wife. The wife went to bed and they stayed up talking - and started an affair. They married in 1965.

By 1972 she had got fed up with Kingsley Amis' drinking - and left him in 1980. She then had lots of psychotherapy because she was suffering from depression. But in 1990 that lifted - and she wrote The Cazalet Chronicles.

In 1995 she went on Desert Island Discs. A male fan wrote to her saying he wanted to be her friend - and he then charmed his way into her life and they had an affair. She lent him money.

Her daughter became suspicious and she discovered the man was a conman and a liar who had researched Howard's life to charm his way in. She was devastated - but you get the impression from Howard's memoirs that at the age of 72 she had very good sex with the con-man. A lot better than many of her other dalliances. She died at the beginning of this year.

When Kingsley died in 1995. He refused to see Elizabeth Jane Howard even when he was dying. He told an interviewer just before he died -

"Do I see her? No. It was bad enough being married to her."

He was the kind of man that said that kind of thing.

SANDRA PAUL

Sandra Paul was a beautiful model - and at the age of 18 she married Robin Douglas Home. He was the playboy nephew of the former Prime Minister, Alec Douglas Home. They had one son - called Sholto - whose godfather was Frank Sinatra.

But things didn't go well. He kept on going out to parties without her. And then, apparently, she caught him having sex with the Marchioness of Londonderry in the back of a car.

Sandra and Robin Douglas Home went through a brutal divorce - which they talk about in the film in great emotional detail. Sandra describes bluntly what she had to do to free herself from the marriage:

"I had to be ruthless in order to be free"

The film-makers use her quote as the title of the film.

Then she met an ambitious public-relations executive called David Wynne Morgan - and they married. But their relationship only lasted 4 years. After that she married an advertising executive called Nigel. But that only lasted 2 years.

Finally she met a young barrister and aspiring politician called Michael Howard in 1975. They talked about Scott Fitzgerald - and the next day he sent her a copy of Tender is the Night. And she moved in with him.

They've been married ever since. She has written some novels - including one called "Ex-Wives". Here she is with her husband on the day he became leader of the Conservative party.

ROBIN DOUGLAS HOME

In the film Robin Douglas Home describes in raw emotional detail how his marriage to Sandra Paul went wrong. It's very moving.

A year after the film was made he committed suicide - after being spurned by Princess Margaret. He did it by taking a cocktail of alcohol and sodium amytal.

He left a tape-recorded suicide note that said:

"There comes a time when one comes to the conclusion that continuing to live is pointless"

One does.

ROMAINE WYNNE MORGAN

Romaine is the first wife of David Wynne Morgan - who left her for Sandra Paul.

They had met and married when she was his secretary. At the time of the film she is living alone in a London suburb with their two children.

Romaine is the star of the film. She is incredibly moving. As opposed to everyone else in the film who you have mixed feelings about - you just really like her. I would love to know what happened to her.

(One interesting fact to know about David Wynne Morgan. Despite having the typical career arc of a successful London PR man, in early 2000 he decided to work for the accused Lockerbie Bomber - the Libyan Abdul Baset al-Megrahi. He believed Megrahi was innocent)

And here is the film:

WHAT THE FLUCK!

2013-12-05T15:37:21+00:00

Every month or so there is a new scandal - mass snooping by the NSA, allegations of price-fixing by giant energy companies, major banks corruptly rigging interest rates, giant modern bureaucracies like Serco and G4S ripping off the taxpayer, children's entertainers from the past charged with sexual abuse.

But these stories never seem to add up to a bigger picture. They are isolated events . And our reaction is always the same - shock and horror, and then it all subsides and we are ready to be shocked and horrified when the next scandal comes along.

It's like a ritualised dance - or the surprised kitty.

There is a lurking sense that there is a kind of seedy corruption underlying a lot of public life today. But while journalism does a very good job of describing that corruption, it is failing to bring it into a bigger focus. To explain what it is all about.

But sometimes you find an oblique angle that offers a bit more perspective.

Tamara Mellon is best known for creating the Jimmy Choo brand - and empire. She started it back in 1996, and by 2000 it had become an incredible success. It was an entrepreneurial story of our time.

But then Tamara Mellon wanted to expand - especially in America - and so she got involved with the system of Private Equity. A company called Phoenix Equity Partners poured in millions of dollars for a majority stake in Jimmy Choo.

They promised a wonderful vision of the future - but Tamara Mellon found herself trapped, she says, in a corrupted system that ripped the heart out of her company. Private Equity wasn't the noble force for good it pretended to be. And it ended when, what she calls, the ruthless financial sociopaths she had let in forced her out.

Tamara Mellon got angry and wrote an autobiography. It was full of lots of celebrity friends and catastrophic drug-taking - but it was also a full on blistering attack on the system of Private Equity.

Here she is being interviewed about it on Newsnight.I suspect the interviewer wanted to get as soon as possible to talking about shoes - but Tamara is going to say what she wants about the corrupt financial world that destroyed her.

But there is more to Tamara Mellon than just that. She's in the public eye because she's telling one kind of story - about Private Equity. But actually her own life story opens all sorts of other, unexpected doors that in a strange way help pull today's random scandals and corruptions into focus.

In particular one of those doors leads you back over a hundred years to a time in America that was rather like our own. There was a realisation back then that the power of money and vast corporate wealth was overwhelming politics and corrupting public life. But journalism was struggling to make sense of the full dimensions of it - and grab the public's imagination.

Then a small group of journalists took an imaginative leap that enabled them to, not only explain, but harness the scandalous events in such a way that created a powerful reaction among the public. A reaction that led to genuine social change.

The very thing we might be waiting for now.

Tamara Mellon had a fantastic father. He was called Tommy Yeardye (she was born Tamara Yeardye). Back in the 1950s Tommy Yeardye was a central figure in the louche nightclub-showbiz scene in London. He was 6ft 4" but also very handsome. One newspaper described him as having "fists like bricks and eyes like emeralds."

He wanted to be an actor - but spent his time being a stunt double for Rock Hudson and Victor Mature. The turning point for Tommy came when his back was used as a stand-in for Victor Mature's back in a love scene with Diana Dors. She decided she liked his front and they began a passionate affair. The Daily Mail described it like this:

"A splendid male, he satisfied Diana's sexual appetite and did his best to meet her constant need for attention and reassurance."

PA photos

Diana Dors was Britain's "sex symbol". Her real name was Diana Fluck - but her mother said she should change it because there was always the chance that her name would be up in lights outside a cinema - and one of the letters might fall off.

She was a good actress and one of her films, called Yield To the Night is really powerful. But most of the time in the 1950s she played roles that were pantomime visions of sex. One film critic wrote that in an age where sexuality was naughty, repressed and fit to burst - "Diana Dors was a joke about sex".

Here is a montage of Diana Dors at that time.

Tommy Yeardye began his affair with Diana Dors in 1957. What then resulted was an extraordinary drama that was played out in the popular press and gripped the nation. But it happened at a time when popular journalism was coming under new pressures - and the drama would end with an event that transformed British journalism.

An event that also set popular journalism on a course that would end with the phone-tapping scandals of today.

Diana Dors was married to a failed actor called Dennis Hamilton. One of her biographers described him as "an out and out louse, a thug, gigolo and serial philanderer". Hamilton was also paranoid about Diana Dors and he kept her under secret surveillance. He installed a two-way mirror in their flat and hid small recording devices to listen to her conversations.

From one of these tapes Hamilton discovered the affair with Yeardye - and he proceeded to smash up the flat. This culminated in a dramatic scene where Yeardye burst in to rescue a hysterical Diana Dors from Hamilton who was pointing a loaded shotgun at her.

This was reported in the press - who also described how Yeardye drove Diana Dors to safety in a green cadillac owned by a bubblegum tycoon called John Hoey. Yeardye was the hero - "I'm no marriage breaker" he said "I am a good samaritan, I have done only what any man worth his salt would do."

The progress of their relationship - and the disintegrating marriage to the paranoid husband - was charted in the press in the late 1950s. Apparently the person behind much of this was Yeardye himself - and he was, in a way, ahead of his time. With his connivance journalists constructed a roller-coaster story of celebrity chaos and drama.

He even arranged a seance so Diana Dors could try and contact her dead mother.

But Yeardye didn't last. By the end of the 50s Diana Dors had thrown him out - claiming publicly that he had been trying to steal thousand of pounds of her money. But then an event happened in Fleet Street that was to take Diana Dors further down this road of celebrity sexual drama.

The News of the World was in trouble - it's circulation was falling. Part of the problem was television, but also its tradition of titillating court reports - randy vicars caught with their trousers down - was feeling tired and out of date. So early in 1960 Sir William Emsley Carr, the alcoholic proprietor of the News of the World appointed a new editor called Stafford Somerfield.

On his first day as editor, Somerfield called his staff together and - as he described it - "pushed the boat out".

"What the hell are we going to do about the circulation? It's going down the drain. We want a series of articles that will make their hair curl."

In a brilliant book about the British Press, the writer Roy Greenslade describes what Somerfield introduced - "two new forms of provocative content: kiss-and-tell memoirs and saucy investigations"

And right away he found the perfect combination of these in Diana Dors.

Somerfield persuaded her to tell the intimate secrets of her life in a series of articles for the News of the World. He had been fascinated by the Yeardye - Hamilton guns and sex drama and was convinced there was far more to be mined from her life. To get the story he paid Diana Dors £35,000 which was an extraordinary amount for that time.

But he got what he wanted. He sat Dors down with a journalist who recorded everything - and then, as Dors later plaintively complained, took "all the mucky bits" and wrote the story of a scandalous, violent and seedy life.

In the articles Dors described how Hamilton and her had sex parties, how Hamilton used the two way mirror to watch couples having sex - taped them and then played the tape back to the entire household over breakfast the next day. She also described the violence in their marriage, and Hamilton's financial scams.

It was a complete humiliation for Diana Dors, and it shocked the nation. The Archbishop of Canterbury described her as "a wanton hussey". And Tommy Yeardye then joined in - offering other newspapers his stories too.

It worked brilliantly though - the circulation of the News of the World soared. But Greenslade argues that by bringing this provocative new content into journalism, Somerfield had also introduced a new "nastiness" into the popular press.

Journalists have always been cynical and "hard-boiled" in their view of the world - but Greenslade says that underneath the froth of silly headlines there was now in the News of the World.

"an underlying nastiness, and a willingness to traffic in human misery"

And he wasn't the only one to think this. In 1969 Rupert Murdoch bought the News of the World. By now Stafford Somerfield had made the paper an enormous success and Murdoch kept him on. But a year later he sacked him. Murdoch later explained why:

"I sacked the best editor of the News of the World. He was too nasty even for me."

The BBC managed to film inside the News of the World just after Murdoch took over. Here he is at an editorial conference with Stafford Somerfield.

They were about to publish the sex revelations of Christine Keeler. It led to even more public outrage - and Murdoch is interviewed defending their publication. I've also included a rather wonderful interview with Somerfield filmed just after Murdoch sacked him. He has a great last line.

But sacking Somerfield didn't get rid of the virus he had brought into tabloid journalism. Nearly 20 years later - in 1988 - one of the great tabloid pioneers, Hugh Cudlipp summed up how that nastiness had spread and possessed newspapers.

Cudlipp was no pompous moralist - he was a hard-boiled newsman who understood how tabloids worked. But now they had mutated into something narrower - giving way, he said, to an

"intrusive journalism for the prurient where nothing, however personal, is any longer secret or sacred and the basic human right to privacy has been banished in the interest of public profit."

I think it is a very interesting question why the tabloids became so nasty.

In the wake of the phone-hacking scandal - it's possible to look back and see how an obsession with exposing hidden lives - especially the sexual aspects - grew and grew from 1960 onwards. It happened during a period of growing openness about sex in society as a whole, but rather than reflecting that openness it manifested itself instead as a weird, vicious prurience.

It may be that we will look back and see it as the reaction of an older generation - both newspapermen and their readers - who found that when the lid was finally taken off talking about sex they didn't know how to deal properly with it. Instead they created a strange and pervy world that finally ran out of control as it became more and more desperate to pry into peoples lives.

Meanwhile in the 1970s Diana Dors became a stalwart of British TV's "Light Entertainment". It was a strange world that mixed old music hall sauciness with this new odd perviness.

Here is Diana Dors on a chat show discussing sexual "signals" with Desmond Morris - who had written a book called "Manwatching". Plus a rather dubious song she sings on a show with Petula Clark - both of them dressed up as little girls.

And as "entertainment" it was also very strange. Here is another song from the Petula Clark show. It is about famous people who were born under the sign of Scorpio.

Petula Clark walks along a row of giant boards - turning each round to reveal another giant portrait of a Scorpio that she then sings to. The list of people she serenades goes beyond weird - and the last one takes you directly to the sinister heart of this odd world.

Heavy entertainment.

Tommy Yeardye got over Diana Dors and went on to marry a beautiful model called Ann Davis. He set up a nightclub in London where patrons could draw nude models as they eat dinner.

He said it was an attempt "to bring art into the average man's life". But it didn't work.

Then he had a lucky break. He went into partnership with Vidal Sassoon - to market his haircare products. Yeardye helped turn Vidal Sassoon into a global brand, and became a multi-millionaire.

In 1967 Tamara Yeardye was born. When she was young the family moved to Beverly Hills - then she was sent back to a posh school in Britain where she met many children of the rich and famous. Like her father she spent a lot of time in nightclubs - but it was in the early days of rave in the late 1980s, wearing DMs and cycling shorts in a famous club called Shoom.

In the early 1990s she drifted into the fashion world, worked at Vogue, became heavily dependent on cocaine and ended up in rehab. But a year later she started Jimmy Choo and her career began.

Then she fell in love. She met Matthew Mellon - a funny good-looking American who was incredibly rich because he was one of the heirs to the Mellon fortune. He too had been in rehab - for overdosing on crack. He claimed that the character of Julian, the drug addicted rich boy, in Less Than Zero was based on him.

Tamara Yeardye described him:

"Matthew Mellon was utterly beautiful and utterly goofy, which was a very endearing combination. He was also damaged goods, wounded and struggling, and that, I think, is where we made the real connection. My mistake was in assuming that, because I'd overcome my addictions, he could too."

Getty/John Stoddart

She became Tamara Mellon. Here is some video of Tamara and Matthew in 2003 working together to build the Jimmy Choo brand at the Oscars. Unfortunately the invasion of Iraq had caused the Oscar red carpet to be cancelled - but Tamara and Matthew keep going - Matthew Mellon shows his own line of shoes, called Harrys.

Neither he nor Tamara look very happy.

Soon after, the marriage began to fall apart. According to Tamara, Matthew Mellon took lots more cocaine and became increasingly paranoid. And as part of that paranoia, by 2004, he became increasingly suspicious about his wife.

What Matthew Mellon then did leads you to the very heart of the giant secret industry that had grown up in Britain to spy on peoples' private lives.

It was a world of private investigators and corrupt policemen that had originally been created to satisfy the ever-growing demands of tabloid journalists for scandalous details about peoples' private lives. A demand that Stafford Somerfield and the News of the World had done so much to kick-start back in 1960.

But what Matthew Mellon's case shows is that this might just be the tip of a much bigger iceberg. A further scandal yet to emerge. That the secret intrusion into peoples private lives, and the surveillance of their behaviour goes far wider than previously thought.

Matthew Mellon got in touch with a company in the City of London called Active Investigation Services - AIS. It was run by a man called Jeremy Young who said he was an ex-detective from Scotland Yard. In reality Young was still a serving Met officer who was leading a double life. He managed to do this by constantly going sick - claiming stress and anxiety, and back pain.

Over 5 years Young took 1,640 days off on sick leave. There are 1.826 days in 5 years.

Matthew Mellon's paranoia was now out of control. He was convinced that his wife was hiding millions of pounds from him in offshore accounts - and he asked AIS to find the hidden money. AIS - who offered a special service called "Hackers R Us", agreed.

They tried to send Tamara Mellon emails that when she opened them would have injected a Trojan virus into her computer. This would them to read everything on the computer.

But at the very same time the police found out that AIS was bugging phones. So the police themselves started to secretly watch and bug the private investigators. It became a gigantic effort - codenamed Operation Barbatus - that lasted 3 years and involved ten police forces and the FBI.

The police raided AIS and seized 60 computers. The detective leading the operation said that what they uncovered was a "national network of corruption" where hundreds of blue chip companies and individuals were using AIS and their network to illegally bug, spy on and hack into individuals' computers.

But then something strange happened - despite all this alleged illegal activity, none of AIS's clients were charged. Except for one - Matthew Mellon. The police burst into his flat and arrested him for authorizing the hacking of his wife's computer.

It was a great trial because Tamara Mellon came up with a brilliant defence for her husband. Quite simply she said that he was too stupid to know what the private investigators were up to. She stood up in court and told everyone that he couldn't even read a comic, let alone a book. His QC helped by producing a psychologist who said that Matthew Mellon's inability to concentrate put him in the bottom 11% of the population.

So he got off.

Ever since Mellon's failed prosecution in 2007 there have been persistent reports that the corrupt Active Investigations network was itself part of something even bigger. The Serious Organised Crime Agency is alleged to have a report with the imaginative title:

"THE ROGUE ELEMENT OF THE PRIVATE INVESTIGATION INDUSTRY AND OTHERS UNLAWFULLY TRADING IN PERSONAL DATA"

It is supposed to reveal a wave of hacking and illegal "blagging" of information over the past few years that goes far beyond the simple intrusion into celebrities lives. Every now and then you get glimpses of this - like with the Tamara Mellon case. But despite calls from MPs and others, SOCA is refusing to publish the report.

If you talk to private investigators who know this world they tell you that not only do we not have the faintest idea of how widespread it has become - but that we haven't conceptually grasped the full dimensions of what is happening.

That at the same time as the police pursue the dodgy private investigators, like AIS, who are bugging and hacking their way into thousands of peoples' lives, the very same police - along with the security services, GCHQ and the NSA - are doing exactly the same to millions of other people. The only difference is that it's legal - because the Regulation of Investigatory Powers Act of 2000, and other laws, allow them to do it.

And while that is happening - all of us are happily allowing giant internet companies to scan all the intimate personal data in our emails and everything we send across social media networks.

It means that we are right to be paranoid. Just like Philip K Dick was.

But there is a paradox here. Because many of those who are shocked by the extraordinary extent of the secret surveillance - radical journalists, cyber-revolutionaries, internet libertarians - are also argueing for total transparency of information.

They not only accept that there is now no privacy personally online - but they believe that the way to bust open the corrupt elite structures of power in society is to release all secret information.

In the past, they say, the old patrician power structure maintained its power by restricting access to that information. Now the technology exists to overcome the gatekeepers and make everything public - as Chelsea Manning and Edward Snowden have shown.

This transparency paradox is part of a much wider present-day confusion. Over the past few years we have been presented with scandals that seem to be evidence of powerful forces that are busy undermining both individual freedoms and the political system that is supposed to protect those freedoms.

These range from the NSA and GCHQ, to global banks, private equity, giant international energy corporations, and parts of the media-industrial complex - like News International (and probably lots of other newspapers as well).

But the scandals do not join up to make a bigger picture. And our reactions are sometimes confused and contradictory - as in the case of transparency and surveillance.

It is as if the scandals are part of a giant jigsaw puzzle - and what we are waiting for is someone to come along and click those pieces together to give a clear, big picture of what is happening.

A hundred years ago, at a time very like ours, a small group of journalists did just that. And the person who takes you back to that time is Tamara Mellon's ex-husband - Matthew Mellon

At the end of the 19th century, Matthew Mellon's great great uncle - Andrew Mellon - was one of the most powerful and richest men in the world. He was part of a small group of bankers and industrialists who not only dominated America - but were using the power of money to undermine, corrupt and control politicians, judges and the whole system of democracy.

They were known as Robber Barons. Men who used new technologies - like the railroads, and global systems of finance - to make themselves wealthier than anyone had ever been in history.Like John D. Rockefeller - who had ruthlessly created the giant Standard Oil of America. Rockefeller now controlled almost all the oil industry in America along with the railroads that transported the oil

And Henry Clay Frick who built the giant US Steel corporation. He was known as The Most Hated Man in America. His special trick was to hire workers from one immigrant group to smash uppity workers from another.

One year he hired Hungarians to break a strike by Italian workers. But less than two years later he had to hire Rumanians to get rid of the Hungarians.

Frick had a great line about this - "The immigrant, however illiterate or ignorant he may be, always learns too soon."

Jay Gould - the railroad king. He was even blunter than Frick - "I can hire one half of the working class to kill the other half" - he said

And the most powerful of all was the banker JP Morgan. He arranged deal after deal that allowed the Robber Barons to build giant industrial monopolies. They were called "trusts". Here is a famous cartoon of Morgan sitting on his throne holding the reins of the economic power of America in his hands

At the same time American society was rocked by scandal after scandal along with terrible stories of the effect of growing inequalities. Politicians were bribed, policemen arrested and beat up innocent men and women, adulterated food was sold, and terrorists threw bombs. While the gap between rich and poor grew wider and wider.

But none knew what to do about it. The genteel middle classes who believed in reform were baffled and confused.

They knew that all these scandals were somehow a part of the enormous changes that were happening to American society. But they also knew that the new technologies and giant industries were bringing amazing benefits and transforming their world and the way they related to each other. Nobody seemed to be able to understand the true dimensions of what was happening.

The person who did a great deal to bring it all into focus was a novelist, Frank Norris. He wrote a book in 1901 called the Octopus about the tragic fate of small farmers out on the distant prairies.

He was recording the anger of these pioneers who had originally gone to live on the land to realise a dream of individual freedom.

But now they found themselves trapped. Their dream was being destroyed by the power of the Pacific and Southern Railroad which was systematically entrapping and exploiting them.

It's a brilliant and dramatic portrayal of a corrupted society. Norris shows how the railroads got the land for almost nothing in the first place - then forced the farmers to buy it from them at inflated prices. At the same time they charged the farmers more and more to transport their harvests to the ports - until they faced ruin and had to mortgage their land to the financiers behind the railroads.

But there was nothing the farmers could do to fight against this.

Norris describes how the tentacles of the railroad wormed their way into every part of the democratic system - bribing and corrupting judges, members of independent commissions, local and state politicians, and newspaper owners. Every institution that was supposed to help the farmers fight to preserve their freedom had fallen under the corrupt control of the railroad.

It's a wonderful book. Other journalists and writers had described this corruption before - but Norris gave it an incredible clarity and emotional force. It became a sensation because it cut through the confusion and gave a simple, clear picture of what was happening to America.

And it wasn't just the railroads trying to make profits. Beyond them was a new financial system that had realised that by making the farmers their servants in this way they could create a mass-industrialised system of agriculture that could both feed the world (which was good) and also make a few individuals so rich and powerful that they destroyed democracy (which was not good).

Here is part of a television history of America made in the 1970s that explains this extraordinary shift. It's done in an old-fashioned magisterial way - but it's good.

It's also got good stuff about the extraordinary world of the Robber Barons that was built out of this corruption.

One of the people who had encouraged Frank Norris to write the Octopus was the editor of a small, but growing, magazine called McClures that was targeted at the new urban middle classes. He was called Sam McClure and he was fascinated by the amazing public reaction to The Octopus.

He decided to try and push his journalists to do the same kind of thing - but factually, and on a grander scale. McClure wanted to create a new kind of journalism that would grab the imagination and the conscience of the middle classes - and not let go. His aim was to show how the concentration of economic power in America had completely overwhelmed and corrupted politics and the law.

He wanted to use journalism to change the way people saw the world. And through that change the world for the better.

It was an extraordinary ambition. And McClure did it by producing a famous edition of his magazine in January 1903 that shocked America.

In that issue McClure published three very dramatic stories. The three journalists who wrote them became superstars as a result - and their journalism was given a new name - "muckraking".

The first article was by a woman called Ida Tarbell. It was about the illegal methods and hidden corruption behind the rise to power of the richest man in America - John D. Rockefeller.

For two years she researched every detail of how he had created his gigantic oil empire. When she found that documents had been destroyed or curiously taken out of public files - she carried on, convinced that copies of the missing reports or investigations into Rockefeller's activities would "turn up" somewhere.

Tarbell's revelations were a national sensation. McClure told her - "You have become the most famous woman in America - that I am getting sort of afraid of you"

In a gripping narrative Tarbell described how Rockefeller's agents would swoop down on a region like Pennsylvania and use all kinds of ruthless and illegal tactics to take over small businesses and destroy the enterprising entrepreners who ran them.

Like with the farmers, Rockefeller controlled the railroads that carried the oil - but Tarbell showed that he also used bribery, fraud, criminal underselling and intimidation to destroy anyone or anything that prevented him creating his giant monopoly.

Here is some footage of Rockefeller - the bit at the end when he speaks is very weird and spooky.

The second article was called The Shame of Minneapolis. In it the journalist Lincoln Steffens blew open the whole political system that governed the City of Minneapolis. Almost everyone in public office was totally corrupt including the City Boss - Mayor Ames, his henchmen and the police captains who took cuts from businesses across the city. And they used a network of criminals to get the money.

Steffens told it like a graphic novel.

Here is Mayor Ames - and McClures is pretty blunt about how bad he is.

And it was all backed up with close up "facsimiles" of the ledgers that every day recorded the money taken from "the suckers"

The third story was a big surprise. It told how the trade unions had been corrupted - manipulating and deceiving their members. It revealed in shocking detail how striking union-members were attacking and sometimes killing workers who refused to go on strike.It was surprising because it took the side of miners who refused to strike.

It was a series of very moving first-hand testimony from miners and their families about the terrible complexities individuals discovered at the heart of bitter disputes. Strikes that seemed on the surface to be a simple battle between labour and capital.

It was - McClure said - about "dramas of human suffering, human loyalty and human fear - the feuds in the coal fields, the bitterness between union and non-union men, the uncompromising hatred opening wounds that only death can heal".

At the end of the issue Sam McClure wrote an extraordinary editorial. It explained what all three articles meant when taken together

It was simple and very powerful. McClure said that the journalism showed that everyone had been corrupted - the best lawyers in the country who are hired to advise corporations how they can get around the law, judges who use tiny "errors" to let people go free who in any common-sense judgement were totally corrupt, politicians who were in thrall to the money power, and police forces who had become organised networks of crime.

But more than that - McClure said - all the other institutions that were supposed to stand up against such a wave of corrupt power had been overwhelmed. From the church to the colleges - the institutions of learning.

"They do not understand"

And he finished:

What McClure meant was that it went beyond corruption. The real problem were the old institutions that could not understand or deal with the new powers that had emerged in society.

He was calling for the ordinary people to realise they had more power and more understanding of the problems than any of the old institutions. And that they should come forward and force politicians to take control and deal with the problems in a new way.

And they did.

The new journalism that McClure began spread like wildfire - and politicians took notice. They were led by the new President, Theodore Roosevelt, who decided to use the law to break the monopolies - or what he called "The Octopus" that was strangling democracy.
Here is the veteran BBC journalist Alistair Cooke describing what Roosevelt did.

Out of this came new laws and new kinds of bureaucracies designed to deal with the money power. And it bred new kinds of public servants who believed they could take on giant monopolies because, for the first time, they understood what was really happening.

To give you a sense of these kinds of people here are parts of two good documentaries made in the 1960s and early 1970s in the Appalachian mountains of Kentucky and West Virginia.

One is about is about a lawyer fighting for the rights of the coal miners in an area that is totally controlled by the mining corporation. He is called Harry Caudill.

Caudill cares deeply about the plight of hundreds of thousands of miners and their families who have been pushed aside by mechanisation in the mines. He takes the camera round the remote valleys and into the houses of the miners - driven by a moral certainty that he must help them confront the power of the mine-owners.

He is a figure from another age. I love the way he explains the inequalities of power to a woman whose house keeps being hit by chunks of rock. The rocks are blown out by the dynamiting going on all around her. She stands there patiently listening to his progressive vision.

I looked up what happened to Caudill. He spent the rest of his life fighting the corporate power in the Appalachian mountains. But by 1990 he was suffering from severe Parkinson's disease - and he committed suicide by shooting himself in the head.

The other is about the story of the killing of Jock Yablonsky. He had won the election to be head of the United Mine Workers, but immediately after his victory he, his wife and his daughter, were all shot and killed.

Everyone is convinced that the corrupt old leadership of the union did the killing - and the film sets out to investigate who did it and why. It is a great portrayal of a hard, corrupt world of union corruption and violence in West Virginia. And the interviews with the crony of the corrupt union boss is both sinister and funny.

In both these films there are echoes of that distant time of the original muckrakers - but also of the powerful new mentality that their muckraking journalism did so much to bring into existence.

Of course our time is completely different from the age of the muckrakers. There isn't the terrible poverty - nor the violent strikes where workers were gunned down by private armies.

But in other ways there are similarities. New technologies and giant financial systems are transforming society. They bring with them great benefits and exciting new ways of living - but at the same time there have been massive increases in inequality.

In Britain - the top 1 percent of people now pay 30 percent of all income tax. Thirty years ago the top 1 percent paid only 11 percent - and that was at a time when the taxes on the rich were much higher. At the same time the average wage has been static for ten years. All new increases in wealth, from productivity, go only to the rich.

The politicians seem to be helpless. The economic crisis of 2008 has revealed scandal after scandal in the financial system but there has been no real reform. When HSBC was revealed to have been laundering money for Mexican drug cartels no-one was prosecuted because doing so "might create instabilities in the system".

And scandals come and go like a series of blows that we experience as disconnected events - each one evoking shock and horror. And nothing happens.

A famous American Historian called Richard Hostadter wrote a study of the muckrakers. he said that before McClure's famous issue there was:

"a diffuse malaise - and it was the muckraking that brought that diffuse malaise of the public into focus"

I think there is an equally diffuse malaise today - waiting for a new kind of journalism to bring it into focus. Like with McClure's it won't be just a catalogue of shocking facts - it will be an imaginative leap that pulls all the scandals together and shows how they are part of some new system of power that we don't fully comprehend.

That maybe - like the farmers on the prairies - we have all become the components of a new kind of machinery of social organisation.

Computers, "financial engineering" and credit, social media, algorithms that predict what you want, NSA surveillance, giant new holding corporations called Master Limited Partnerships - all of these surround us and wrap us into a complicated modern web. Some of it is wonderful, other parts of it are threatening - while even more parts are just incomprehensible.

And behind it is the new money power - giant institutions, and individuals that can bend politicians to their will. The repeal of the Glass Steagall act in 1999 - which arguably did a great deal to create the financial corruption of our age - is just one example.

While the old institutions that grew up over the past hundred years to protect us now find themselves unable to comprehend or cope with the new systems of power. Politicians, regulatory institutions, intelligence agencies, the mainstream press, the police, the BBC, the colleges of academia- all of them, as McClure said in 1903:

"They do not understand"

And cut off from the real power struggles - these old institutions are starting to prey on each other. Leaving us both confused and undefended.

One newspaper editor writing about the loss of the independence of the farmers a hundred years ago summed up the new system:

"The farmers farm the land, and the businessmen farm the farmers."

Maybe today we are being farmed by the new system of power. But we can't see quite how it is happening - and we need a new journalism to explain what is really going on.

Meanwhile here is Diana Dors singing a beautiful, but sad, song in her garden at night. She has summoned Russell Harty and some others to celebrate the opening of her new swimming pool. Part of her extraordinary later life as the consummate Light Entertainer.

THE BABY AND THE BAATH WATER

2013-09-02T11:55:36+00:00

Much of the debate about whether to intervene in Syria or not is taking place in a strange ahistorical vacuum. As with so much debate about humanitarian intervention the underlying world view is of a simplified story of bad dictators and good, well intentioned westerners who must somehow intervene to stop him.

But the truth is that America has a very complicated relationship with Syria which stretches back over sixty years.

Back in the 1950s America set out to intervene in Syria, liberate the people from a corrupt elite, and bring about a new democracy. They did this with the best of intentions, but it led to disaster. And out of that disaster the Assad regime rose to power.

America's actions were by no means the only factor that led to the violence and horror. But their unforeseen consequences played an important role in shaping a feverish paranoia in Syria in the late 1950s - which helped Assad, and his Baath Party, come to power.

A while ago I wrote the story of America's strange relationship with Syria and the dark and bloody twists and turns that resulted - from 1947 onwards. I thought it would be good to link to it again because so much of what happened is relevant to today's debate.

You can find the original post on Syria here.

BUGGER

2013-08-08T15:31:10+00:00

The recent revelations by the whistleblower Edward Snowden were fascinating. But they - and all the reactions to them - had one enormous assumption at their heart.

That the spies know what they are doing.

It is a belief that has been central to much of the journalism about spying and spies over the past fifty years. That the anonymous figures in the intelligence world have a dark omniscience. That they know what's going on in ways that we don't.

It doesn't matter whether you hate the spies and believe they are corroding democracy, or if you think they are the noble guardians of the state. In both cases the assumption is that the secret agents know more than we do.

But the strange fact is that often when you look into the history of spies what you discover is something very different.

It is not the story of men and women who have a better and deeper understanding of the world than we do. In fact in many cases it is the story of weirdos who have created a completely mad version of the world that they then impose on the rest of us.

I want to tell some stories about MI5 - and the very strange people who worked there. They are often funny, sometimes rather sad - but always very odd.

The stories also show how elites in Britain have used the aura of secret knowledge as a way of maintaining their power. But as their power waned the "secrets" became weirder and weirder.

They were helped in this by another group who also felt their power was waning - journalists. And together the journalists and spies concocted a strange, dark world of treachery and deceit which bore very little relationship to what was really going on. And still doesn't.

PROLOGUE - SALISBURY PLAIN 1991

In January 1991, as the Gulf War began, MI5 became convinced they had discovered a secret Iraqi terror organisation based in Britain.

They had found a list of thirty three Iraqis who were studying for PhDs in London. The list had been sent by the Iraq embassy in London to the Bank of England to ask the Bank not to freeze the grants the students lived on. The Bank sent the list to MI5 and the agents quickly realised that actually they were looking at something far worse - a nationwide Iraqi military terror cell.

The reason they knew this was because the person who sent the list was the deputy military attache at the embassy.

Immediately the police were told to swoop on the 33 "students" - and they were taken to a disused military camp at Rollestone in the middle of Salisbury plain and interned as prisoners of war. They were surrounded by two levels of high security razor wire and guarded by a hundred heavily armed soldiers.

It was the first time anyone had been held like this in Britain since the Second World War.

In fact the letter showed nothing of the kind. The Iraqi military attache was also in charge of administering student grants for Iraqis studying in Britain.

Some of them did get funding from the Iraqi military - for studying things like the structure of polymers. But, as a British professor pointed out, if that same interpretation were applied to British science students, over half of them would be immediately re-classified as terrorists.

Here is part of a programme made later that year about the absurdity of what happened. It shows how neither the detainees or their lawyers were even allowed to know what the evidence was that had led to them being imprisoned.

The man who defends MI5 with such fervor will turn up later in this story - playing a very odd role. he is called Nigel West - but his real name is Rupert Allason.

I've added on the news reports of the same Iraqis suddenly being released from the heavily fortified camp. But now everyone is referring to them as "students".

An inquiry was held later that year into the scandal. It asked MI5 to produce its evidence. Other than the letter, the secret agents came up with nothing.

They had imagined the whole thing. But they justified it by saying

"It was best to err on the side of caution".

NEARLY A HUNDRED YEARS EARLIER

THE DAILY MAIL CIRCULATION DEPARTMENT 1906

William Le Queux was a popular novelist in the early part of the twentieth century. He was half French, half British and he wrote books with wonderful titles like Strange Tales of a Nihilist.

Le Queux had started off as a journalist on the Daily Mail - but then had travelled around Europe getting to know lots of famous and infamous people. But as he did so he became convinced that many of the European countries, but most of all Germany, envied Britain and wanted to get their hands on the wealth of the Empire.

The trouble was that the British people didn't realise this. So Le Queux set out to warn them - above all by telling them that the Germans were sending spies to Britain to prepare for an invasion.

But the ruling classes in Britain laughed at Le Queux. They said it was just fiction - which it was. Plus he wasn't really British and he hadn't been to a proper school, he was far too vulgar and insistent in his patriotism. In short he was a bore.

So Le Queux did what anyone in their right mind would do in such a situation. He turned to the Daily Mail.

He wrote a gripping account of a future German invasion of Britain and took it to Lord Northcliffe who ran the Mail. It was called "The Invasion of 1910" and it described how the Germans landed in East Anglia and marched on London.

Northcliffe loved it - but the Mail's circulation department said that many of the towns on Le Queux's invasion route didn't have many actual or potential Daily Mail readers in them.

So Lord Northcliffe changed the route of the invasion to make sure that all the towns that were sacked and pillaged had lots of Daily Mail readers. Here is the map of the invasion as agreed with the circulation department.

The serialisation was an enormous success. The prime minister got up in the House of Commons and said Le Queux was "a pernicious scaremonger" and that the story was "calculated to alarm the more ignorant public opinion at home."

Result.

Then things started getting out of control. Thousands of Daily Mail readers sent Le Queux letters telling him that they had spotted people acting suspiciously - which meant they must be German spies.

The letters were mirror images of what Le Queux had written in his books. But rather than making him suspicious, Le Queux decided that this proved that what he had written as fiction must actually be true. There was a gigantic German spy ring in Britain.

Thousands of Daily Mail readers couldn't be wrong.

The man whose job it was to uncover spies in Britain was very excited by all this. He was called Colonel Edmonds. He had a tiny budget and two assistants - and noone on the General Staff bothered with him.

But now Col. Edmonds saw his chance. He teamed up with Le Queux and together they bombarded the Committee for Imperial Defence with the evidence from the Daily Mail readers. Edmonds said that the government should set up a "secret service bureau" to combat the threat.

The head of the Committee - Lord Haldane - said this was ridiculous. But even he couldn't stand against the wave of spy fever that was sweeping the country. He gave in - and MI5 was set up - created in large part by the dreams of a socially excluded novelist, and the paranoid imaginings of the readers of the Daily Mail.

But the problem for MI5 was that the spy network didn't exist. The Germans did have some agents in Britain - but nothing like the 5000 that Le Queux had described.

When war against Germany was declared in 1914 - MI5 immediately rounded up 21 alleged German spies and proudly announced they had broken the network. But a brilliant piece of historical research by the historian Nicholas Hiley has shown that this wasn't true.

Hiley doesn't mince his words. Here are his conclusions (Kell and Holt Wilson were the director and deputy directors):

"One of the most famous successes of the British Security Service was its great spy round-up of August 1914. The event is still celebrated by MI5, but a careful study of the recently-opened records show it to be a complete fabrication - MI5 created and perpetuated this remarkable lie.

The great spy round-up of August 1914 never took place - as it was a complete fabrication designed to protect MO5(G) from the interference of politicians or bureaucrats.

The claim made next day that all but one had been arrested was false, and its constant repetition by Kell and Holt-Wilson was a lie."

In other words - MI5 had followed the shining example of William Le Queux and made it all up.

But that didn't matter - because it made a great story, and journalists loved it. Even in 1997 the BBC made a breathless documentary - using the recently released files - about how in 1914 MI5 had brilliantly rounded up the Kaiser's spy network on the eve of the first world war.

Aside from perpetuating a fiction, the film has two great moments - one is an interview with the grandson of the deputy head of MI5 who has an immortal line about his grandfather - "of course he was very private about MI5 - so the family knew nothing".

And the end the programme has some wonderful stills of the party MI5 held to celebrate the end of the war - it's on their rooftop. Their faces are great.

After the first world war MI5 declined in importance. But with the growing fears of communism in the 1920s and 30s a new threat emerged - not just communist agents from abroad, but British communists who might betray their own country.

In many cases they came from the same upper classes as those running the secret services. And a strange dance began - of toffs suspecting toffs.

But even then MI5 couldn't get it right.

Take the case of Cecil Day Lewis - who was Daniel Day Lewis' father. Back in the 1930s he was a teacher at Cheltenham College - one of the great Victorian public schools.

But, despite his job, Cecil was convinced that he was really a revolutionary. And in 1933 he decided to foment revolutionary action in Britain - by writing a poem. It was an epic he called "The Magnetic Mountain". He said his aim was to create

"A violently revolutionary poem with abundant images (for example) of a barren, cancerous land led by 'getters not begetters', demanding 'It is now or never, the hour of the knife/ The break with the past, the major operation."

Here is Cecil Day-Lewis looking both poetic and radical - alongside some of the poem - (you can see where Daniel Day Lewis gets it all from).

But Day-Lewis was disappointed by the lack of reaction. He admitted that the poem "did not create the slightest ripple of outrage amongst the guardians of Cheltenham."

Even though the communist magazine - the Partisan Review - had said that it was "perhaps the most important revolutionary poem as yet written by an Englishman".

And then MI5 noticed Cecil Day-Lewis. Not because of the poem - but because he had sent £5 as a donation to the headquarters of the Communist Party in London. So MI5 decided to put Day-Lewis under intense surveillance.

The historian James Smith has written a wonderful book about how MI5 spent a lot of time covertly watching many upper class British writers between 1930 and 1960. It is a great book because what it records is a strange and confused dance of manners among different parts of the British elite.

Smith describes how MI5 got the local police to spend weeks watching Day-Lewis' house and intercepting his post. But they found nothing suspicious. Their report said that:

"Day-Lewis seldom wears a hat, and is not altogether of smart appearance in dress. He is a good singer. He has moved into his cottage after having considerable structural improvements done there."

MI5 were completely incompetent. They didn't discover the poem that Day-Lewis hoped would help to bring about a communist uprising in Britain.

And not only did they miss the poem - they didn't even realise he was a poet. All in all MI5 found nothing dangerous or revolutionary about Cecil Day-Lewis. It was humiliating.

But they might have been right. James Smith describes how a few years later in 1940 Cecil Day Lewis was getting his mistress Rosamund Lehmann to pull strings in the British establishment so he could avoid getting called up to go and fight the fascists.

But in 1940 MI5 had its greatest success. It not only found a real German spy network in Britain - but managed to persuade many of the German agents to switch sides.

It was called the Double-Cross system - and it is celebrated in histories of MI5 as a brilliant use of espionage. The German agents carried on spying for their masters in Berlin - sending back detailed reports. But the information was all fake, designed to mislead and confuse the Nazis.

But something else happened to all the intelligence agencies during the war - MI6 as well as MI5. As they grew massively in size they became riddled with factions and infighting. And because all this happened behind a wall of secrecy, there was little to stop things becoming vicious and poisonous.

The journalist Phillip Knightley has written a really good history of spies - called The Second Oldest Profession. In it he quotes an agent describing what happened during the war years:

"The whole organisation was riddled with nepotism - dim, dreary people of utter unmemorability; sub-men who were doubled up with other sub-men to create an illusion of strength and only doubled the weakness; others made memorable only by poisonous, corrupt malevolence or crass, mulish stupidity; the whole run by a chain of command remarkable for its feebleness. The entire service was decrepit and incompetent."

At the end of the war the new Labour government knew that something had to be done to sort out MI5. So they went and found Percy Sillitoe - who was running a sweet shop in Eastbourne

Sillitoe had retired after being Chief Constable of Glasgow - where he had become famous as the only policeman brave enough to take on the "Razor Gangs" in the eastern part of the city.

The gangs had names like The Bingo Boys and The Baltic Fleet - and they terrorised Glasgow as they fought each other with hatchets, swords, open razors - and razor blades stitched into the brims of their hats.

You can get a sense of Sillitoe from this short film where he shows the BBC a new kind of armoured car he has invented to stop criminals holding up vans carrying cash. He invented the security van.

I very much like how he says he is "concerned for the little man".

I've also added an odd bit from a BBC film about graphology where the expert - a "psycho-graphologist" - analyses Percy Sillitoe's signature, and compares it to J Edgar Hoover's signature. Hoover was Sillitoe's American counterpart.

The government asked Sillitoe to come and sort out the chaos in MI5 - and he agreed. But he quickly found that it was a very odd place - all the insiders hated him, and they ridiculed him by speaking in Latin (which he didn't understand) in front of him. Plus they deliberately gave him the wrong papers when he went to see the Prime Minister.

Sillitoe came back and told his wife - "I sometimes think I am working in a madhouse." But he realised that he was dealing with very much the same situation that he had found in the slums of Glasgow - different factions locked together in a strange, poisonous bubble.

Here is a section of a very good film, made much later, about the successors to the razor gangs of Glasgow - the gangs that Sillitoe had tried to suppress in the 1930s. And you can see the similarity to the world of the spies - as one of the gang members puts it, "it's two ends of the same street at war with each other".

I also love the pigeon-fancier who shows off the most high-security pigeon loft you have ever seen. He then reveals that he doesn't breed the pigeons for racing. Their job is to go and kidnap the pigeons from the other gangs.

But before Sillitoe could do anything, it all went terribly wrong. Suddenly traitor after traitor was revealed in the very heart of the British establishment. It wasn't just pretentious radical poets who were a threat - it was spies, diplomats and nuclear scientists within the system itself who had been giving away secrets to the Russians.

There was a high-flying diplomat called Donald McLean, a nuclear scientist at the heart of Britain's atomic bomb project called Klaus Fuchs, plus two of MI6's top agents - Guy Burgess and Kim Philby.

One of MI5's main jobs was to find traitors - but the awful truth was that it had failed to spot any of them.

Percy Sillitoe was booted out. But things got even worse. In 1964 MI5 were told that one of their own men had been a spy for the Russians. He was called Sir Anthony Blunt - and not only had he been high-up in MI5 - but he had gone on to work in Buckingham Palace looking after the Queen's art collection. And even worse than that he was the Queen Mother's cousin.

MI5 interrogated Sir Anthony and he calmly said that it was all true - he had been a traitor. MI5 was so embarrassed that they kept it all quiet, gave Blunt immunity from prosecution, and he carried on working at Buckingham Palace.

The Daily Mail later said that the Royal Family had known all along anyway. That as far back as 1948 Sir Alan Lascelles - the most senior aide to the Royal family - had whispered "that's our Russian spy" to someone else as they passed Blunt in the palace.

But that could have been a misinterpretation. Blunt had shocked the Queen Mother by telling her that he was an atheist - and she had immediately assumed that meant he must be a communist.

Clever Queen Mother - wrong but right.

Blunt had also become a bit of a TV star. Starting in the early 60s the BBC went to him regularly to take the viewers on a tour of the treasures of Buckingham Palace - a sort of early Fiona Bruce.

Here is part of one programme from 1962 - two years before he was exposed as a traitor. Followed by a bit of another programme from 1972 - when a self-confessed KGB agent takes the viewers round Buckingham palace. Spot the difference.

And at the end there is footage from 1979 - when Blunt was exposed as a traitor. It's from some rushes I found in the library. The press chasing Sir Anthony are straight out of a British movie. And I love the interviewer's obsession that it was Blunt's "homosexual leanings" that made him betray his country.

Blunt, though, doesn't bat an eyelid. It's as though he is still talking about some painting.

Then, in 1971, MI5 got another big shock to the system. Most of their opponents - Russian secret agents in Britain - were kicked out, leaving MI5 with little to do. The irony was that it happened as a result of one of their few successes.

In August 1971 an ordinary London policeman arrested a man who was driving drunkenly down Tottenham Court Road. He turned out to be Oleg Lyalin who was a KGB agent. Lyalin spent a lot of his time buying socks in the West Midlands - pretending to be a member of the Soviet Trade delegation. But really he was spying.

Lyalin panicked and offered to tell MI5 the names of all the Russian spies in Britain. In return he wanted to stay and live in Britain with his mistress. MI5 agreed - and the Home Secretary expelled 105 other members of the trade delegation, because Lyalin said they were spies.

Here are the reports - plus a "News Special" which is an early example of the way TV journalism would report the hidden world of spying. It's got an anonymous British "research scientist" called "Jim Walker" who got caught up in all this - and has some great MI5 surveillance footage of Jim and his controller Viktor leaving information at a "dead letter drop."

Plus a very good telephone non-interview with the British Ambassador in Moscow.

But the problem for MI5 is that the expulsions pretty much destroyed the KGB presence in Britain.

The historian Stephen Dorril who has written a series of brilliant detailed histories of the intelligence agencies says that a later KGB defector called Oleg Gordievsky admitted that "the London residency never recovered from the expulsions".

Dorril also says that the British government and its civil servants were well aware of this, and they became deeply suspicious of claims from MI5 and its K Branch - whose job was to monitor foreign agents - that there was still a big Soviet threat in Britain:

"Senior civil servants dealing with the intelligence community were therefore aware that K Branch claims about the penetration of British political life and the threat to security from Soviet bloc operations were generally exaggerated."

The brutal fact was that by the early 1970s MI5 not only had very little to do - but also its political masters were beginning to question whether it might be seriously incompetent.

Edward Heath - who had been Prime Minister when all this was happening - later got up in the House of Commons and said bluntly what he had discovered about MI5 officers:

"They talked the most ridiculous nonsense, and their whole philosophy was ridiculous nonsense.

If some of them were on the tube and saw someone reading the Daily Mirror they would say - 'Get after him, that man is dangerous, we must find out where he bought it.' "

But those in charge in Britain also realised that there was nothing they could do to question or control the spies. The next prime minister in the 1970s - Harold Wilson - wrote a very serious book called The Governance of Britain full of long serious chapters.

But when he got to chapter nine - about

THE PRIME MINISTER AND NATIONAL SECURITY

This is what it looked like.

There are two paragraphs explaining that the prime minister has ultimate responsibility for the security agencies. And it ends with two more that simply say this:

"The prime minister is occasionally questioned on matters arising out of his responsibility. His answers may be regarded as uniformly uninformative.

There is no further information that can usefully or properly be added before bringing this Chapter to an end."

In response to these kind of doubts and attacks MI5 turned inwards.

The problem for the MI5 men - stuck in their secret bubble - was that they just couldn't believe that their failure was due to them being useless at their job. Not only had they failed to find any of the traitors, but operation after operation had ended in failure. And they convinced themselves that this meant there had to have been another traitor lurking somewhere in their building - the MI5 HQ in Mayfair.

They began a mad search for enemies inside the organisation itself - seeking to find more hidden traitors who could be used to explain why MI5 kept failing to do its job properly.

It was the search for "Fifth Man" - to go with the other four already exposed, Burgess, McLean, Philby and Blunt

A small group of MI5 men went to their boss and said they wanted to investigate all the past failures looking for evidence of treachery. Their boss was called Sir Roger Hollis - and he said no. His argument was that operations often went wrong because of simple human failure, and to re-examine them on the basis that failure was evidence of treachery would tear the agency apart.

Imagine what it would feel like he said to know you are being watched because a past operation you were involved with had gone wrong. "It's like the Gestapo" he said.

So the small group of MI5 agents decided he must be the traitor.

Here is a picture of Roger Hollis.

The small group in MI5 now became convinced that their organisation was not just penetrated by the Russians, it was actually run by a Soviet agent. They knew they had to get the truth out somehow even if it meant breaking the law. So they found a friendly journalist called Chapman Pincher and told him the hidden truth.

Here is Chapman Pincher being interviewed on the Wogan programme about what then happened. Up to this point Pincher had been the Defence correspondent on the Daily Express. He was successful for getting "scoops" from "inside sources" - although the historian EP Thompson said that really Chapman Pincher was:

"A kind of official urinal in which ministers and intelligence and defence chiefs could stand patiently leaking."

What the dissident MI5 agents now told Pincher was like super high-grade piss. Or, as he puts it in the Wogan interview, "it was like walking into an Aladdin's Cave". But what Pincher wrote was going to open the floodgates to a new kind of conspiracy journalism that still holds sway over large parts of the media imagination.

Have a look at him and decide yourself - high grade toilet or investigative journalist? Or maybe often they are the same thing?

I've also included Pincher being interviewed on the TV news reports as the scandal unfolded. Everyone tries to get in on the act. The BBC presenter quotes Kim Philby as saying that Hollis wasn't very good at his job. But the presenter says that this is "ambiguous" - and might be proof that Hollis really was a Soviet agent.

The leading MI5 dissident who was leaking the information to Pincher was called Peter Wright. He was one of the most senior members of MI5 but he was also somewhat paranoid.

To get a sense of Peter Wright and how he saw the world I have put together some bits of him being interviewed in the 1980s about another of his conspiracy theories. This was that the Prime Minister - Harold Wilson - had also been a Soviet agent.

In Wright's mind much of the British establishment had been directly or indirectly taken over by the Soviet Union. He had no hard evidence for this - but he was driven by an underlying mind-set that was going to spread throughout much of the intelligence agencies - and journalism - over the next twenty years.

This said that if you imagined the other side was doing something devilish and deceptive - then they probably were. It meant that in the dark world of intelligence, imagination was more powerful than obvious facts. Because if you couldn't find the evidence it proved how clever the enemy had been at covering their tracks.

It was a fevered romantic view of the world that would both entrance the readers of newspapers - but would also lead the intelligence agencies into the disaster of the weapons of mass destruction in Iraq in 2003

Here is the grandaddy of that conviction - Peter Wright. The person called Angleton he refers to was an even odder American equivalent of Wright who was high up in the CIA - and who also was convinced Wilson was a Soviet agent.

The tone of Wright's plaintive child like statement about Angleton - "he believed it - he did" tells you a great deal about the emotions driving these strange men in their spy-bubbles.

But as in all organisations - egos started to come into play. Other MI5 agents started leaking other names to other journalists. Pincher's main rival was a writer called Nigel West.

Nigel upped the stakes. He began to publish books and articles alleging that all sorts of other people had been traitors. Here he is on Nationwide in 1981 in full flow. He says that a man called Leo Long was a traitor, and then goes on to suggest that others - including even the former Governor of Uganda, Sir Andrew Cohen - might be traitors.

It's worth looking closely at what Nigel West says about Sir Andrew Cohen - because it shows how weird this paranoid outpouring from the secret world was becoming. When he was an undergraduate at Cambridge in the 1930s Cohen had been a member of an intellectual society called The Apostles. So had two of the spies - Burgess and Blunt.

The interviewer asks Nigel how he knows Sir Andrew might be a traitor. Nigel says:

"I haven't named him (Sir Andrew) up to now because it's not known whether he was a Soviet agent. But I think it's worth saying that anybody, if you are talking about the Apostles, many of them were Soviet agents. And he would undoubtedly have been questioned since he rose to a very senior position in the Department of Overseas Development"

That's it. But Nigel does have a fabulous haircut.

In the early to mid 80s more and more names poured out - all accused of being KGB agents in the heart of the British establishment.

One newspaper grouped them under headings

"CONFESSED" - "PARTIALLY CONFESSED" - "UNRESOLVED"

There was one great apology

"Our list of MI5 spy suspects included Cedric Belfrage who MI5 officers said had made a partial confession and we said was dead.

We are glad to make it clear he is alive, never made any confession and maintains he should not have been on the MI5 list at all."

And Mrs Thatcher also got involved. Because it seemed to prove to her the thing she had believed all along - that the British establishment were weak, spineless and easily corruptible. She happily admitted in Parliament that Anthony Blunt had been a traitor. And here she is in 1986 merrily joining in with the latest accusation - that Lord Rothschild had been the 5th Man.

It later turned out that he wasn't.

It became farce. The journalists who had started the mole-hunt went to war. Nigel West wrote a whole book announcing that he had discovered that the 5th man wasn't really Hollis, but was actually Hollis' deputy. He was a man called Graham Mitchell who in his spare time was a grand master in correspondence chess.

Apparently the dissidents in MI5 were convinced that the letters he sent his chess-friends were his way of contacting his Soviet controllers. The moves he typed out were actually secret codes that disguised his treachery.

Here is one of Graham Mitchell's games that he played in 1950. You are looking at a complicated code, whether it was secret messages to the Russians has never been proved.

Another writer then found a letter in an old government file that had been written by Roger Hollis in the 1940s saying that the Russians shouldn't be trusted. Some journalists said that this proved he wasn't a traitor. But others said that Hollis had put the letter there deliberately so it could be found and throw MI5 off the scent.

Here are the TV reports - both of the Graham Mitchell "revelation", and the Hollis letter. The leader of the pack - Chapman Pincher - still insists Hollis is the 5th man. Nigel West says he is innocent.

But Nigel now has a very good late 80s haircut.

Then another writer called W. J. West wrote a book saying that the 5th man was Hollis after all.

W J West turns out to have been an ex-hippie whose early years were memorably captured in a semi-autobiographical novel by another ex-hippie - called "Ten Men". She describes a road trip across America as she desperately but unsuccessfully tries to shag him.

Here's his book about the Fifth Man.

But then - in the midst of all these weirdos - a dissenting voice emerged.

James Rusbridger had been a spy back in the 50s and 60s - and he now wrote a book called The Intelligence Game arguing that all this was rubbish - and that all the journalists had been conned by a crazy gang of right-wing nutters in MI5.

Rusbridger said that the newspapers and TV were being used to promote the obsessive belief of MI5 officers that their failure to do anything worthwhile for a quarter of a century was the consequence of there being a Russian spy in MI5.

They couldn't face the fact that they were completely useless and incompetent.

At last a voice of sanity.

But unfortunately James Rusbridger was then found dead in his garden shed - apparently the victim of an auto-erotic game that had gone wrong. He was naked apart from a rubber coat and a gas mask - and his feet and legs were attached to the wall by a complicated system of pulleys.

Of course it might have been a fiendishly clever assassination.

Or just another spy-world weirdo.

But this crazed witch hunt didn't harm MI5 at all. Quite the opposite - because together the spies and the journalists created an image in the public imagination of a dark world full of hidden treachery. The spy world became a fascinating other universe that was full of layer upon layer of deception, where the men who inhabited it spent their time trying to penetrate through the circles of falsehood to the inner sanctum of truth.

It was an image that was powerfully helped by John Le Carre's novels - and his anti-hero George Smiley. Le Carre's novels were a clever piece of PR - because they appeared to be more gritty and realistic than the glamourised James Bond image.

But it was just another layer of deception - because Smiley and his search for a hidden mole expressed powerfully the paranoid and unfounded fantasies of the dissident MI5 agents.

But it was a world that was all made-up. Le Carre - who had himself been a spy - admitted this, and described what the true reality of the spy world was:

"For a while you wondered whether the fools were pretending to be fools as some kind of deception, or whether there was a real efficient service somewhere else.

Later in my fiction, I invented one.

But alas the reality was the mediocrity. Ex-colonial policemen mingling with failed academics, failed lawyers, failed missionaries and failed debutantes gave our canteen the amorphous quality of an Old School outing on the Orient express. Everyone seemed to smell of failure."

But this new image couldn't conceal MI5's incompetence for long.

Because at the very same time that everyone was talking excitedly about completely invented moles, MI5 missed the real moles at the heart of the intelligence services - even though they were completely obvious, and almost screaming to be noticed.

Michael Bettaney worked in counter-espionage in MI5. He had been recruited when he was at Oxford university - where he had been an admirer of Adolf Hitler and had a habit of singing the Nazi Party anthem in local pubs.

Here is Bettaney back then.

MI5 did a thorough check on him - called positive vetting - and decided he was fine. Perfect MI5 material. Bettaney was then sent off to Northern Ireland to fight terrorism where he was wounded by a car bomb. He then had a horrible experience. Hidden in a cupboard he had to watch in silence as one of his informants was shot through the kneecaps by other terrorists.

Here is Bettaney later - after he had been working for MI5.

Bettaney came back to London a changed man. He decided that MI5 was both corrupt and incompetent. He started drinking heavily and told his colleagues loudly that he was no longer a fascist - but he had become a communist.

So MI5 decided to promote him. He was positively vetted again - found to be perfect MI5 material, and sent to the Russian desk.

Bettaney became more and more unstable. In October 1982 he was convicted of being drunk and disorderly. The next week he was convicted for fare-dodging. Finally MI5 did begin to notice - and two separate inquiries were set up to look into Bettaney's behaviour. But each was unaware of the other's existence.

Neither of them noticed that he had been stealing a huge amount of MI5 top secret documents and stashing them at his home. Bettaney was only caught when he took some of the best of these secrets and tried to stuff them into the letter box of the Second Secretary of the Russian Embassy - Mr Gouk.

This is a picture of Mr Gouk.

Mr Gouk was so confused by this that, instead of passing them on to the KGB, he went round to MI5 and gave them back, and told them where they had come from. MI5 arrested Bettaney and he was put on trial.

The man who was in charge of the vetting of government employees - like Michael Bettaney - was then allowed to vet the members of the jury at Bettaney's trial. Luckily this time he got it right - and Bettaney was sent to prison on the Isle of Sheppey for 23 years.

Here are some of the reports. Including Nigel West turning up yet again on Breakfast Time. Even Nigel is shocked by how MI5 didn't spot Bettaney. And he's having a bad hair day.

The terrible truth that began to dawn in the 1980s was that MI5 - whose job it was to catch spies that threatened Britain - had never by its own devices caught a spy in its entire history.

The case that really shocked Mrs Thatcher was the traitor Geoffrey Prime. In the 1970s he had worked at the top secret listening centre GCHQ and had been selling all it's secrets to the Russians.

And yet again it wasn't MI5 who uncovered his treachery - it was the local police in Cheltenham.

In 1982 a policeman came to his house enquiring about his car - a rather distinct two-tone brown and white Mk IV Cortina - a which had been seen in the vicinity of an assault on a young girl.

Prime told the policeman that he had been at home all day. But that evening he and his wife Rhona went for a drive to the top of Cleeve Hill. As they sat in the twilight Prime told Rhona that he was the man the police were looking for. And not only that, he was also a Russian spy.

Here is part of a very powerful interview Rhona Prime gave to the BBC where she describes that day - and what she then did.

Prime was a paedophile - and had used spying techniques to monitor the activities of thousands of young girls around Cheltenham. He had created a vast set of index cards which showed when the girls were most likely to be alone at home. He then went round to their houses in his two tone Cortina and sexually assaulted them.

Despite this Prime had been positively vetted six times.

Even the Russians got worried about his paedophile activities and seemed to want to dump him. In 1980 Prime had gone to Vienna to meet the KGB. Instead of meeting him secretly as they normally did, the Russians took him openly to the best restaurants where they knew Western intelligence agents would recognise them as KGB agents.

But even then noone noticed them - or Prime.

Prime's wife Rhona wrestled with her conscience - and in the end went to the police and told them everything about Prime. He was sent to jail for 35 years for spying and 3 years for the assaults on young girls - which says a lot about the priorities of the British establishment at that time.

The cases of Bettaney and Prime revealed not only just how incompetent MI5 was - but also how sad and seedy the secret world of spies really was.

But even in the midst of all this treachery - a surprising thing happened.

Rhona Prime decided to stand by her husband. Here is Rhona describing how her deep christian beliefs gave her the strength to stand by her husband. She is very calm and composed, and somehow her dignity makes you realise just how odd the whole spy thing was. A strange hysteria driven by totally inadequate men - both agents and journalists - who were incapable of dealing with real human emotions like love and loyalty.

Rhona talks about something else - unconditional love. Receiving unconditional love, she says, makes us whole and beautiful people because we are totally accepted. The very opposite of treachery.

At the same time, one of the original traitors - Kim Philby - died in the Soviet Union. The BBC cameraman Phil Goodwin has given me the unedited rushes recording Philby's funeral in Moscow. He found it in the back of a cupboard in the BBC's Moscow office.

It's an amazing record of a weird communist state funeral - held for an upper class Englishman in a Moscow graveyard in 1987. Standing all around are the faces of the Russian side of the spy world - and it is great to look at their faces, peeking out for a moment from their traditional secrecy.

Then Philby's coffin arrives accompanied by a military band and members of the KGB holding all Philby's Soviet medals on orange cushions. It's an extraordinary scene. But also watch the woman with red hair. She is Philby's widow - Rufina - who had lived with him and helped him through alcoholism and depression.

Watch what Rufina does. It's really moving. Love and loyalty breaking through again into this narrow, nasty world.

And even Michael Bettaney found love. Marion Johnstone, who was a research scientist and also a communist, wrote to him in prison in 1985. She began to visit him - and they became engaged.

But in 1995 there was a security scare that reawakened all the spy journalists on papers like the Mail - and made them huff and puff again. Marion was found to have taken some photos and made some drawings of the landscape on Sheppey around the prison and given them to Bettaney.

The prison authorities confiscated them, and the journalists immediately said that this was part of an escape plan to spring the traitor from jail. Marion denied this - she insisted that because Bettaney was kept in solitary confinement she just wanted to show him how beautiful the landscape was outside.

And she is right. The landscape around Bettaney's prison, Swaleside, is extraordinary and beautiful. A little while ago I managed to get onto Deadman's Island which is nearby on the river Swale.

It is a moody place because it is where prisoners from a long time ago - the Napoleonic wars of the 1800s - were buried. They had been held on the "hulks", floating prisons off the coast of Sheppey.

What makes the island so strange is that it is covered by water every high tide - and that washes away the mud and opens up the prisoners' graves. It means that the island is littered with human bones.

The warden of Deadman's Island very kindly showed me round - and here he is showing me the open graves and the bones of prisoners, other kinds of traitors, from a very different war of long ago.

But what really did for all of the intelligence agencies at the end of the eighties is that none of them predicted the collapse of communism.

Mrs Thatcher's advisor - Charles Powell - summed up the extraordinary failure:

"The biggest single failure of intelligence of that era was the failure of almost everybody to foresee the end of communism. It caught us completely on the hop. All that intelligence about their war-fighting capabilities was all very well, but it didn't tell us the one thing we needed to know - that it was all about to collapse.

It was a colossal failure of the whole Western system of intelligence assessment and political judgement."

But the real reason that the intelligence agencies didn't predict the collapse of the Soviet system was because many of the people at the top of the agencies couldn't believe it was true.

Sir Percy Cradock was one of the most powerful figure in the British establishment. He was the chairman of the Joint Intelligence Committee - which co-ordinated the activities of MI5, MI6 and other intelligence groups. Even at the end of the eighties when everyone else was realising that the Soviet Union was collapsing, Sir Percy remained convinced that this was all a trick. That the Soviet Union was still aiming for communist domination of the world.

Here is Sir Percy

Cradock - along with a number of others high up in the intelligence agencies - really believed that Gorbachev's reforms were just a cunning ruse to deceive the West. And - as Mark Urban has pointed out in his book UK Eyes Alpha - Sir Percy used his position to make sure that this view dominated the Joint Intelligence Committee.

But as Urban also points out - Sir Percy and his allies had no secret evidence for this. They relied on what was pompously called "analysing open source data". Otherwise known as reading the newspapers and watching TV. Except they interpreted that data in a mad way - driven by their own fevered imaginings of a world completely possessed by infinite levels of deception.

Mrs Thatcher realised this was bonkers - and she finally gave up on the spies.

And that really should have been that for MI5.

Except ten years later it was saved by the War on Terror - and since then MI5 has grown massively. But what no-one seems to know is whether MI5 has changed.

For most of the twentieth century the combination of ineptitude and secrecy created an organisation that retreated more and more into a world of fictional conspiracies in order to disguise it's repeated failures. The question is whether the same is true today?

Disasters like the total intelligence failure over the WMD in Iraq would suggest that nothing much had changed. But the trouble is there is no way we can ever find out. The spies live behind a wall of secrecy and when anyone tries to criticise them, the spies respond by saying that they have prevented attacks and saved us from terrible danger. But they can't show us the evidence because that is secret.

It was recently revealed that back in the 1970s - at the height of the obsession with traitors - MI5 trained a specially bred group of gerbils to detect spies. Gerbils have a very acute sense of smell and they were used in interrogations to tell whether the suspects were releasing adrenaline - because that would show they were under stress and lying.

Then they tried the gerbils to see if they could detect terrorists who were about to carry a bomb onto a plane. But the gerbils got confused because they couldn't tell the difference between the terrorists and ordinary people who were frightened of flying who were also pumping out adrenaline in their sweat.

So the gerbils failed as well.

Perhaps MI5 shouldn't have given up so easily. Maybe what we need is a better class of gerbil to find out the truth? But maybe we have them already - they're called journalists.

But the saddest thing in this whole story is that Rhona Prime did not stay with her husband Geoffrey. In 1995 she met and fell in love with someone else.

MASSIVE ATTACK V ADAM CURTIS

2013-06-19T14:00:22+00:00

A LIVE SHOW STARTING IN TWO WEEKS TIME IN MANCHESTER

The show is a collaboration between myself and the brilliant Robert del Naja of Massive Attack.

What links us is not just cutting stuff up - but an interest in trying to change the way people see power and politics in the modern world. To say to them - have you thought of looking at it like this?

We've used film, music, stories and ideas to try and do this - to build a new kind of experience. The best way we can describe it is "a Gilm" - a new way of integrating a gig with a film that has a powerful overall narrative and emotional individual stories.

The show will be a bit of a total experience. You will be surrounded by all kinds of images and sounds. But it is also about ideas. It tells a story about how a new system of power has risen up in the modern world to manage and control us. A rigid and static system that has found in those images and sounds a way of enveloping us in a thin two-dimensional version of the past.

A fake, but enchanting world which we all live in today - but which has also become a new kind of prison that prevents us moving forward into the future.

Along with Massive Attack the show will star two great singers.

The amazing Liz Fraser.

And the wonderful Horace Andy.

And some of the music will be surprising - from early Barbra Streisand to Siberian punk from the 1980s.

Here's a two-dimensional trailer that will give you an idea of what the show is about - and the stories it tells.

MRS THATCHER - THE GHOST IN THE HOUSE OF WONKS

2013-04-26T13:03:17+00:00

I'm afraid I haven't been posting any stories recently. The reason is that I am in the midst of putting together a live show with Massive Attack. It's a joint production between the BBC and the Manchester Festival.

I've had quite a few requests to put up a film I made a while ago about Mrs Thatcher - called The Attic. It's about how she constructed a fake ghostly version of Britain's past, and then used it to maintain her power. But also how she became possessed and haunted by this vision.

I'm putting it up as a bit of a corrective to the terrifying wonk-fest that took over after Mrs Thatcher died. A conveyor belt of Think Tank pundits and allied operatives poured into the TV studios and together they built a fortress around Mrs Thatcher's memory that was rooted in theories about economics.

They did this because economics is the only language that wonks understand. It's a view of the world where they see the voters - the people who put Mrs Thatcher in power - as simplified consumption-driven robots.

What was missing was the fact that Mrs Thatcher was also a powerful romantic politician who created a strange but compelling story about Britain's past that connected with the imagination of millions of people. It was fake, but it was incredibly powerful because she believed it. And the power of her belief raised up ghostly dreams from Britain's past that still live in people's imaginations - long after she fell from power.

The problem with wonks is that they can't deal with emotion and feeling, and they don't like stories. It means that they cannot connect at all with the feelings and imaginations of the voters. Yet the think-tankers have built a sarcophagus of economic discourse around Westminster.

What we are waiting for is a politician to come along who can connect with our imaginations and inspire us about political ideas instead of boring us to tears.

The film stars a wide range of characters - including Flanagan and Allen:

Members of the Political Wing of the Irish National Liberation Army:

Deborah Kerr:

PC Claude Morrell:

Mrs Thatcher and her friend Airey Neave:

And the ghost of Winston Churchill:

Here's the film.

YOU THINK YOU ARE A CONSUMER BUT MAYBE YOU HAVE BEEN CONSUMED

2013-03-05T16:07:14+00:00

One of the guiding beliefs of our consuming age is that we are all free and independent individuals. That we can choose to do pretty much what we want, and if we can't then it's bad.

But at the same time, co-existing alongside this, there is a completely different, parallel universe where we all seem meekly to do what those in power tell us to do. Ever since the economic crisis in 2008, millions of people have accepted cuts in all sorts of things - from real wages and living standards to benefits and hospital care - without any real opposition.

The cuts may be right, or they may be stupid - but the astonishing thing is how no-one really challenges them.

I think that one of the reasons for this is because a lot of the power that shapes our lives today has become invisible - and so it is difficult to see how it really works and even more difficult to challenge it.

So much of the language that surrounds us - from things like economics, management theory and the algorithms built into computer systems - appears to be objective and neutral. But in fact it is loaded with powerful, and very debatable, political assumptions about how society should work, and what human beings are really like.

But it is very difficult to show this to people. Journalists, whose job is to pull back and tell dramatic stories that bring power into focus, find it impossible because things like economic theory are both incomprehensible and above all boring. The same is true of "management science". Mild-mannered men and women meet in glass-walled offices and decide the destinies of millions of people on the basis of "targets" and "measured outcomes".

Like economics it pretends to be neutral, but it isn't. Yet it's impossible to show this dramatically because nothing happens in those glass-walled offices except the click of a keystroke that brings up another powerpoint slide. It's boring - and it's impossible to turn it into stories that will grab peoples imaginations - yet hundreds of peoples' jobs may depend on what is written on that slide.

I want to do a series of posts that will go back and reveal the forgotten roots of some of this fake objectivity that surrounds us today. They will be a series of stories that show how over the past fifty years both the political Right and the Left have gnawed away at the idea of objective truth. Sometimes almost colluding together to help bring about today's uncertainty and confusion about where power and influence really lies in our society.

The first is an odd story - with a very strange character at its heart. It is about how in the 1950s the richest man in the world, an oil billionaire in Texas, invented a new form of television journalism. It pretended to be objective and balanced but in fact it was hard core right-wing propaganda. It was way ahead of its time because, in its fake neutrality, it prefigured the rise of the ultraconservative right-wing media of the 1990s - like Fox News, with its copyrighted slogan, "Fair and Balanced"

The billionaire was called H. L. Hunt - Haroldson Lafayette Hunt. He made his fortune in the early 1930s by getting hold of one of the biggest oil fields in America - in the pine forests of East Texas. He was a ruthless, driven man and from early on he became absolutely convinced that he had superhuman qualities that made him different from other humans.

Here is a picture of Mr Hunt which gives you a sense of his conviction about himself.

From the 1920s onwards Hunt was a bigamist. He married two women and raised two families that were oblivious of each other. He told his second wife, Frania, that he was called Major Franklyn Hunt. There was a rocky moment when his picture was on the front page of all the Texas papers because of his spectacular oil deal. Frania asked Hunt if that was him - he told her no, that it was his uncle who had been so clever.

Hunt was part of a group of extreme right-wing oil men in Texas who had enormous influence because of their wealth. There is a brilliant book written about this group - The Big Rich by Bryan Burrough. Burrough describes how they had first risen up in the 1930s because they loathed President Roosevelt - "a nigger-loving communist", as one oil man called him. They were convinced that Roosevelt's New Deal was really run by Jews and communists - or "social vermin" as they politely put it.

A Texas congressman called Sam Rayburn summed up this group of right-wing oil men. "All they do is hate" - he said.

After the Second World War H L Hunt did two things. He added another, third, family to his bigamist's collection. And he also turned to the new medium of television to promote his ultraconservative views. In 1950 he wrote a pamphlet putting forward the idea of what he called an "Educational Facts League" - its purpose, Hunt wrote:

"will be to secure a impartial presentation of all the news through all the news channels concerning issues of public interest"

It would, said Hunt, be an organization where ordinary Americans would be supplied with the true facts of political life.

Hunt announced that the organization would be called "Facts Forum" - and he found a man called Dan Smoot to be its public face. Smoot had been an FBI agent - and he was smooth and reasonable. Starting on radio, but then moving to television, Smoot presented a show called Facts Forum which every week would give you, the audience, a balanced presentation of the facts behind the news. Very reminiscent of the later catch-phrase on Fox News - "We Report, You Decide".

In fact this declaration of balance and fairness was rubbish. Smoot would begin by presenting the left or liberal viewpoint on a subject in a dull, bland way. Then would enthusiastically put forward the alternative, or what Hunt called, the "constructive" view. This view was simple - all government was bad, business should be left alone - and anyone who disagreed was a communist trying to take over the world. And was probably a Jew as well.

The programmes were radically skewed to promote an ultraconservative agenda while pretending to be neutral and balanced.

There was lots of implied racism in the shows. In his book Bryan Burroughs quotes from one episode where Smoot argued against fair employment legislation - and said:

"Remember that the negroes when first brought to America by Yankee and English merchants were not free people reduced to slavery. They were merely transferred from a barbaric enslavement by their own people in Africa to a relatively benign enslavement in the Western Hemisphere."

Facts Forum became a successful media enterprise - with two syndicated radio shows and three TV shows produced from their own studios in New York. They were backed up by books and pamphlets paid for by Hunt. One was called "We Must Abolish the United Nations" - written by Joseph Kamp. His previous "balanced" books had included one called "Hitler Was a Liberal".

Here is a wonderful documentary profile of H. L. Hunt. It was made in 1968. By now his first wife had died, the second had got fed up and moved away, and Hunt was now left with only his third wife - Rita Ray.

You get a very good sense of Hunt's obsessive drive to promote his conservative views - sending out endless pamphlets, training young men and women to become part of his League of Youth Freedom Speakers, and even insisting that his whole family sit at the dinner table to listen to one of his new radio shows. It was called LIFELINE. Again Hunt was ahead of his time - because the show fused right-wing anti-communism with fundamentalist religion.

What you don't see is the tragedy of Hunt's life - his eldest son Hassie. He had originally followed his father into the oil business, but had then become violent and paranoid. Hunt had tried his own treatment - bringing in lots of women for Hassie to have sex with. But what had worked for the father didn't do much for the son. Doctors tried ECT - but that didn't work. In the end Hunt was persuaded to let them give Hassie a prefrontal lobotomy and his son spent the rest of his life wandering the Hunt estate like a strange ghost.

At the end of the film Hunt and his wife get up in their living room and sing together "We're just plain folks". It's very spooky. And it's not true.

Hunt's Facts Forum was the model for much of what was later to come with the rise of the right in the media in the 1990s - both in radio and TV. But Hunt didn't just shape the future of the right, he also had a profound effect on the way the Left too attacked and corroded the idea of objectivity and neutrality in journalism.

It happened because of some pieces of paper that were found in the jacket pocket of Jack Ruby - the man who shot Lee Harvey Oswald. Two of them were scripts from Hunt's radio programme called LIFELINE. The third had a telephone number of one of Hunt's sons.

Many of Lifeline's programmes had attacked John F. Kennedy as a communist dupe who was destroying America - and Jack Ruby had apparently been outraged by such vicious propaganda against Kennedy.

Then it was discovered that a full page advertisement placed in the Dallas Morning News on the day of the assassination had been partly paid for by another of Hunt's sons - Bunker Hunt. It was surrounded by a black, threatening border - and was titled sarcastically "Welcome Mr Kennedy to Dallas"

Like his father, Bunker Hunt was an ultraconservative - and the advertisement was placed under a title that echoed Facts Forum. It was called "The American Fact-Finding Committee" who described themselves as "An unaffiliated and non-partisan group of citizens who wish truth". And it accused JFK of all sorts of treasonous acts against America - including:

"Why have you ordered your brother Bobby, the Attorney General, to go soft on communists, fellow-travellers and ultra-leftists in America, while permitting him to persecute loyal Americans who criticize you, your administration, and your leadership?

We DEMAND answers to these questions, and we want them NOW."

As a result newspapers across America attacked Hunt's operations for creating the "climate of hate" in Texas that might have contributed to the President's death. And Hunt and his sons became targets in the FBI investigation that would then become part of the Warren Commission.

And it got worse. In 1967 the ambitious District Attorney in New Orleans, Jim Garrison, opened a new investigation into Kennedy's killing. Garrison started talking about how there had been a conspiracy that might have included certain unnamed Texas oilmen.

Hunt's head of security managed to get hold of a diagram drawn out by Garrison's team where "H L Hunt" was at the heart of a complicated network of lines drawing connections between the Dallas police, Ruby, Oswald, plus all kinds of small-time players in Dallas. And although Garrison's investigation folded in 1969 - it, and its diagrams, became the template for the growing conspiracy theories from the left.

One of the earliest - and most powerful - expressions of this was a film called Rush To Judgement made in 1967 by a left wing filmmaker called Emile de Antonio and a lawyer-turned-investigator called Mark Lane. De Antonio is a fascinating character - he came out of the avant-garde art world, and had worked with Andy Warhol and Robert Rauschenberg - and he shared their knowing distrust of the media world of two-dimensional images that was then becoming so prevalent.

Rush to Judgement sets out to propose an alternative explanation for Kennedy's assassination. At the heart of this other story is the idea that there is a group of powerful, shadowy men in Texas who used their wealth and power to create a distorted fiction - Oswald the lone nut - to disguise their conspiracy. A fiction that the public then believed.

The film interviews a whole host of extraordinary bit players from the Texas world and builds up a very powerful mood of uncertainty and suspicion. Underlying this is a message that says these hidden forces in America will never allow you to know the truth. Which means that what you are told by the media may be a lie. That you are being manipulated.

Just as H. L. Hunt himself was gnawing away at the idea of objectivity and truth through his own TV programmes, so too were the left also using a demonic caricature of H L Hunt to do the very same thing. He and other shadowy figures, the left said, will never let you know the truth.

Here is a section of the Rush To Judgement film. It had its world premiere in 1967 on BBC television - broadcast for an hour and a half at prime time. The section starts with the presenter in the studio introducing it - and framing how the viewer should interpret it. Then I have cut straight to the latter part of the film - which is all about how intertwined Jack Ruby was with the Dallas police and establishment.

It is long, but I have left it like that deliberately, because I think it is important to see how Emile de Antonio uses a particular technique to persuade you that he is presenting the real truth. The interviews are held long, and an archive interview with the Dallas police chief is used repeatedly to counterpoint them. It has a cumulative power that feels real and also feels like it is allowing you to judge the characters. That technique would rise up and become central to many of the more mainstream liberal documentaries of the last thirty years.

But it is also very much a technique borrowed from avant-garde cinema and in that sense is as artificial a language as anything you see on Fox News.

We report. You decide.

Mark Lane went on to help write a film in 1973 called Executive Action. It was about how a group of Texas oilmen kill President Kennedy. It was the same idea that resurfaced in Oliver Stone's JFK. But the best, and earliest, caricature of Hunt is in the film Billion Dollar Brain - also made in 1967. It was written by Len Deighton and directed by Ken Russell. The villain is a raving right-wing Texas oilman called General Midwinter who runs an organisation called Crusade For Freedom - modelled on Facts Forum and Lifeline - and wants to use his giant computer to bring down the Soviet Union.

Here's a short clip of General Midwinter in full-on Hunt mode.

But H L Hunt was far more than a caricature right-wing nutbag. The roots of so much of the distrust of the media today lie back with him and his ideas - with his Facts Forum in the 1950s and the strange role he played in Dallas in the 1960s.

In later posts I want to trace how what Hunt started, spread out from the dark pine forests of East Texas and began to develop into a much more powerful force undermining the idea of neutrality and objectivity in our age.

PARADIABOLICAL

2013-01-30T18:52:48+00:00

The West is worried about the rise of Islamism in Africa. There are two big fears - one is that there is a new international terror network that will come and attack Europe and America. The other is that sneaky Islamist groups like the Muslim Brotherhood will get themselves elected - and then promptly abolish democracy.

But behind these fears is an incredibly simplified - almost fictional - vision of the world. It possesses the minds of many western politicians, journalists and associated think tank "experts". And at its heart is a kind of filter that wipes away anything complex about power and the struggles for power in African countries - and replaces that with a simple picture of the world as divided between goodies (us in the west) and dangerous frightening baddies who are out to destroy us.

It's both blind and arrogant. And it's terribly dangerous.

To try and bring it into focus I want to go back twenty years and tell two dramatic stories. In them lie many of the roots of today's western fears - but also, in the details of both stories are keys to understanding two crucial things that we ignore today at our peril. One is the complex local power struggles that have helped the rise of Islamism in Africa, and the second is the way past western interventions have fuelled a hatred and distrust of Europe and America - that has in turn massively helped the Islamist cause.

One is the story of what happened in Somalia between 1990 and 1993 - the real events that led to Black Hawk Down or, to give it its proper name, "Operation Gothic Serpent". The second is the story of the weird and horrific events that happened in Algeria between 1992 and 1996 after the Islamist party called FIS was stopped from winning an election by an armed coup. A coup that had the implicit backing of the west.

There is an odd ghost that haunts not only Somalia's history, but has also lodged itself in the western imagination. He was called Mohammed Abdullah Hassan - and a hundred years ago he set out to try and unite all the Somali people in an Islamic state. The British called him The Mad Mullah and they battled against him for twenty years until they found a new way of getting rid of him. They bombed him from the air.

These are the forgotten ruins of the place that was going to be the capital of his Islamic state - he called it The Dervish state

For the next forty years the Somali people remained divided - ruled by the British and the Italians as part of their empires. Then, in 1960, Somalia was finally given its independence. But, like so many of the other former European colonies, all sorts of powerful remnants of colonial rule remained. Not just the arbitrary lines drawn on maps to make the new countries - but in the minds and imaginations of millions of newly liberated people.

Here is a film made in 1961 which captures this brilliantly. It's from a series called Africa Now, subtitled First Hand Reports from a Changing Continent and it is about life and politics and the new forces of power in independent Somalia.

The capital, Mogadishu, had been part of Italian Somaliland - and the film shows how strongly the Italian presence remains. Not just in the grand buildings that had been part of Mussolini's dream of a Second Roman Empire, but in the language. Not only is there no written Somali language - which means the Somalis use Italian - but they don't even have a word for "independence", so they use the Italian word - "indipendenza". I also really like the attempt to create a written Somali language. It was called "Osmania", and it wasn't a success.

The film also shows how Mogadishu has already become "the cockpit in the propaganda struggle in the Cold War". The film captures the ambassadors from all the different players - the Soviets, the Americans, the communist Chinese and the West Germans - going hither and thither in their gleaming cars in Mogadishu, all snuffling around trying to gain influence over the new President, Abdullah Osman Daar.

And the key to that influence is foreign aid. The film shows how the Soviets are offering to build a proper harbour, while fascinatingly the Chinese are already building a road system for Somalia. The Americans don't seem to be doing very well - but the West German ambassador is very keen, he spends his time walking around the desert looking for possible places for development projects.

But you can see who's going to win out. The Russian ambassador who is described as "a carefree agitator with boyish charm".

But in 1969 democracy in Somalia ended. There was a military coup led by Major-General Siad Barre who set up what he called The Somali Democratic Republic. But in reality it was a centralised communist state modelled on General Barre's interpretations of Marx and Lenin and Mao.

Siad Barre promised to wipe away the ghosts of the past that were holding Somalis back from being truly independent. And that meant not just the old colonial remains, but the crucial thing that was holding Somalia back, Barre said, was the clan structure.

Somali society was permeated by a complex clan structure. Somalis defined themselves and understood their relationship to each other in great part through this system of clans and sub-clans. Siad Barre said that it was the clans - or "clanism" - that had undermined democracy in the new Somalia - so he was going to wipe out this destructive and outmoded "tribalism" and replace it with a new, centralised society run by The Supreme Revolutionary Council.

In 1974 the Council published a book about the new society they were building. It has great images of revolutionary displays.

It also contained lovely colour pictures, like this one of modern Somalis dancing at the discotheque in the new Juba hotel in Mogadishu.

And it also summed up this glorious new revolutionary world and its beautiful future like this:

Algeria didn't get its independence quite as easily as Somalia. Between 1954 and 1962 revolutionary groups - the main one was called the FLN - fought a vicious terrorist war against the French who ruled Algeria. The FLN bombed French civilians in cafes and the streets, while they also killed many Algerians in the French controlled Algerian army. In response the French killed the guerrillas and also used widespread torture.

In 1962 the French gave up and Algeria became independent. Its first President was one of the leaders of the FLN - Ben Bella. But in 1965 he was deposed by a military coup led by one of his close friends from the revolutionary times - Houari Boumediene - who, of course, like Somalia, turned Algeria in the a copy of the Soviet Union.

It all worked fine for a while because Algeria had oil - and as oil prices rose the FLN used the money to subsidise their state socialism. But underneath everyone knew that power was really concentrated in a small elite group that came from the east of the country and excluded everyone else.

There was growing resentment, but no real coherent opposition. But then, in November 1982, there were a series of battles on the campus of the University of Algiers between a group of Marxist students and a group of Islamists who killed one of the Marxists. The Islamists were protesting about the fact that the Marxists, who all spoke French, would get all the best paid jobs. While anyone who just spoke Arabic would find it nearly impossible to get a professional career. This meant that they were excluded from power in Algerian society.

The protests were immediately repressed - the Islamists were all arrested. But it was an important moment because it was the first public demonstration by an Islamist opposition, breaking cover and coming into the open in a country where all opposition was banned. The protests were led by a teacher called Abbasi Madani who had once been in the FLN. He was put in prison for two years - but he will turn up later in this story playing a very important role.

They key thing in the protests was the fact that the Marxists spoke French - and that was the route to power. Again it was a powerful example of how the remnants of French colonial times still exercised a powerful grip on the destinies of those who were supposed to be free and independent of that past. And it was that frustration that was a powerful fuel for the growing Islamist movement.

The most dramatic example of how the French Empire still possessed the minds and behaviour of Africans was in the Central African Republic. It too had got independence from France in 1960 - but in 1965 there was, of course, a military coup and Colonel Jean Bedel Bokassa took power. In 1972 he made himself President for Life, but then, in 1977, he decided to crown himself Emperor of the Central African Empire.

It was a weird and grotesque demonstration of how the European mind set still controlled Africans in a distorted way. Because Bokassa was directly modelling himself on the French emperor Napoleon - and his coronation was supposed to be an exact copy of Napoleon's coronation as emperor in Paris in 1804.

Here is a great film made about Bokassa as he prepares for his coronation. It's a wonderful picture of what happens when a mad dictator decides to spend lots of money - clutches of European designers and planners and "facilitators" flock around all taking it very seriously. While Bokassa spends his time in his palace watching film of other royal coronations and the British Queen's Silver Jubilee in order to get inspiration.

Bokassa is also interviewed. He explains why he cuts peoples' ears off - he says it's a lot less barbarous than the death penalty, which France still had at that time. I suppose he has a point. And then he tries to explain why he is establishing an Empire when in fact he hasn't got an Empire. It's a very odd explanation - and it's very funny.

To the Islamists in Algeria, a figure like Bokassa was a dramatic example of what their fundamental theory predicted. Modern Islamist ideas said that European and Western ideas of democracy were always going to lead to corruption. However well-intentioned at the beginning, the system gave enormous power to individuals and that always corrupted them.

It was a very pessimistic theory because it saw human beings as always being fallible and corruptible. Bokassa was an extreme example, but the Islamists believed that the same thing had happened in Algeria. The idealistic Marxist revolutionaries had morphed into a corrupt and repressive clique. The only solution was to an impose a rigid, incorruptible system of moral and political guidance on the politicians which they had to follow. And that should be drawn from Islam.

The Algerian Islamists’ chance came in 1988. Two years before - in 1986 - oil prices had collapsed and the effect on Algeria had been catastrophic. Half of the country's budget was wiped out and the whole socialist "experiment" collapsed. Out of the disaster came widespread corruption and soaring prices.

On October the 4th 1988 the dam burst and the young, angry urban poor started to riot in Algiers. The centre of the rioting was in the shopping mall called Riad al Fath, the Victory Gardens. It symbolized the elite that ruled Algeria and the rioters smashed it up. In the next days the rioting spread spontaneously. And the only organisation ready and able to ride the wave of fury were the Islamists.

And on March 10th 1989 the Islamic Salvation Front FIS was formed - with the aim of bonding together the chaotic rebellion and using it to create an Islamist state. The founders of FIS had different approaches, but the two key ones were Abassi Madani and Ali Belhadj. Madani had led the protests back in 1982 and he believed that it would be possible to Islamize Algeria without changing the fabric of the state, while Belhadj was more radical - he believed in armed struggle to create a new kind of state.

Here are the first reports of the rioting in the "Days of October" - followed by the meeting that announced the founding of FIS.

Meanwhile in Somalia things were also going very badly for the "Victorious Leader" - Siad Barre.

His problems had begun back in 1977 when he had decided to try and create what he called Greater Somalia. Barre started by invading an area of neighbouring Ethiopia called the Ogaden. Millions of Somalis lived there - but back in the late 1940s the British, under US pressure, had decided it was part of Ethiopia. But now Siad Barre decided Somalia wanted it back.

To begin with the invasion went well. But then the Soviet Union, who had been backing Siad Barre, suddenly decided to switch sides and back Ethiopia. Almost overnight they pulled out their advisers and troops, along with a bunch of Cubans who had also been helping Somalia.

The reason the Soviets switched sides was because there was a new dictator running Ethiopia who was more Marxist-Leninist than Siad Barre - and even more ruthless. This made the Soviets feel that he was more worth backing and they poured weapons, money and men into Ethiopia to help defeat Siad Barre, their previous friend. This included airlifting thousands of Cuban troops into Ethiopia.

Here is Siad Barre saying everything is going swimmingly with the Russians - followed by news footage of the Soviet Advisers leaving Mogadishu a few days later.

And here is film of the weird, frightening world of Colonel Mengistu in Ethiopia that the Soviets went off to help. It was shot just after Mengistu had taken power in 1977. He had just started what he called The Red Terror. It was the mass execution of what Mengistu called "counterrevolutionary elements" - otherwise known as the Ethiopian Peoples' Revolutionary Party who had already been running what was called The White Terror, which involved killing Mengistu's supporters - who were members of what was called the Derg Party

There is a very odd scene in a graveyard where there are hundreds of graves already dug for the future victims of the White Terror. There is a very good piece of deadpan dialogue from one of the gravediggers.

Q. What is the Red Terror?

The Red Terror is conducted by members of the revolution.

Q. And who are the victims of the Red Terror?

Those who conduct the White Terror

The solution for Siad Barre was simple. He switched sides too - and went to the Americans for help. The US started to pour arms and money into Somalia. But it came too late to help him in the war in the Ogaden. The Russians and the thousands of Cuban troops mounted a counterattack and smashed the Somalian army. The Ethiopians then displayed the arms they had captured along with captions saying where the arms came from.

and their ammunition

I particularly like - "Reactionary Pakistani Grenades"

The defeat in the Ogaden was a disaster for Somalia. Over one million people fled from the Ogaden into Somalia, a country that then had a population of about 4 million. Food prices soared, groups began to fight for access to precious water, and there was total disillusion with Siad Barre.

Over the 1980s the Americans poured all kinds of weapons along with hundreds of thousands of dollars into the country. And as the economy collapsed the country became increasingly dependent on American aid. But the aid then had a strange consequence - it brought the clan structure back to prominence and power. Siad Barre gave up any idea of ridding the country of "clanism" and started using the aid as a way of buying loyalty from different clans. Clans who supported him got the aid, those who didn't went without. This had a crucial effect because those who headed the Somali clans began to see aid as the route to power in Somalia.

And when the Cold War ended in 1990 the clans who had been excluded set out to overthrow Siad Barre. A war began and Barre was forced out. It didn't stop there though, "the liberators" split into sub-clans and then started to fight each other viciously. The ensuing civil war had terrible consequences because the terrible violence was the primary cause of a famine in the Bay area of the country that surrounded Mogadishu. The victims were hundreds of thousands of people who had been displaced by the fighting.

What then happened was one of the great scandals of the past twenty years. The United Nations promised to help - and then did nothing. A bureaucracy that had once promised to bring peace to the world had become corroded and corrupted by the politics of the Cold war - and it failed utterly. For over a year it did nothing - and thousands of Somalis died of starvation.

Then western television discovered Somalia. Crews began to pour in from Britain and America and sent back horrific pictures of dying children. There has been much criticism of TV journalists both from within the aid community, and from those who think that aid is a bad thing - they argue that television simplifies, emotionalises and thus distorts the reality on the ground.

Here is a film that I think is very important in this debate. It was made by the BBC journalist, Michael Buerk, at the time in Somalia, and it is about the difficulties of reporting such a complex situation. It's good because it shows how chaotic and incomprehensible things were - with different factions in the civil war trying to get control of the aid (because that was what they had learnt from the 1980s onwards). But it also shows how television was inexorably drawn to the two powerful images that were beginning to occupy a very big space in the western imagination.

The innocent, dying child.

And the evil frightening men on their "technicals"

Goodies and baddies.

In Algeria, in contrast, things seemed to be going very well. The riots had forced the ruling FLN leadership to give up on the one-party state - and bring in proper democracy. In February 1989 President Chadli announced a new constitution that would allow political parties to exist and compete in elections on both a local and national level. It was an extraordinary breakthrough - and everyone had great hopes of a future democratic Algeria.

But there was a lurking doubt in many peoples minds. The front-runners in any future elections were obviously FIS - the Islamist Salvation Front - because they already had a national organization. But the question was - did they really believe in democracy? Or would they simply use the elections to get into power and then create a new kind of Islamist state that abolished democracy? No-one knew.

But FIS had the high ground because they offered an alternative to the corrupt regime. For the first six months of 1990 they organized marches and meetings - and then on June 12, 1990 FIS won an astonishing victory in the local and municipal elections. It won control of the majority of the country's communes.

Here is the first report on the BBC of the shock result - and already you can feel the concern in the west beginning. The fear that this would be Khomeini Mk II, though this time it was Sunni not Shia. There is also a good interview with a Tunisian Islamist who tries to counter these fears. He says that western governments must not turn away from this new Islamism:

"otherwise they will be seen to be supporting the corrupt governments, and our people will make the link between the dictators in our countries and British and American and European governments"

The mystery about FIS was personified in the two men who led the Islamist movement. One was the urbane Abassi Madani who drove around in a Mercedes and spent his time reassuring the Algerian middle class establishment that everything was going to be OK.

The other was the charismatic Ali Belhadj who rode around on a small motorcycle and was an amazing public speaker. His followers were the young urban poor. They were called the "hittistes" - from the arab word for a wall - "hit". They were called this because millions of hittistes spent all day leaning up against the wall with nothing else to do.

There was euphoria in FIS after the local elections - but the effect was to deepen the mystery and the fears. Groups of young FIS followers started to try and impose bits of what they thought were sharia law. Women who worked for the local communes were forced to wear the veil, video stores and shops that sold alcohol were closed down. And the middle class who had supported FIS began to get worried.

It got worse. At the end of 1990 Ali Belhadj gave an interview where he zeroed in on how western culture - above all French culture - had poisoned the very minds of Algerians. This had to be wiped out. His intention, he said, was:

"to ban France from Algeria intellectually and ideologically, and be done, once and for all, with those whom France has nursed with her poisoned milk"

The Hittistes loved this - and groups of them started to go round trying to destroy the TV satellite dishes that were feeding the poisoned milk into the minds of the Algerians.

The Algerian middle classes really didn't like this. They felt they were becoming trapped in a regressive bubble - isolated from the world that they connected with via the French news that came in through their dishes. They also feared the growing thuggishness of groups of Hittistes who were going round beating up girls who didn't wear the veil.

Sensing this disenchantment, the FLN rulers started to play dirty. They designed the new electoral constituencies deliberately so they would weaken FIS's chances of winning. In return FIS got very angry and in June 1991 they called for a General Strike. Thousands of their supporters took over squares in the centre of Algiers and things got very Arab Spring.

At a press conference Ali Belhadj made a dramatic intrusion. He said that if the ruling elite tried to stop FIS then he would take up arms and fight them, just like his father had done when he had fought the French. It's a really powerful moment that shows how charismatic Belhadj was - and also why the middle classes were becoming frightened of FIS.

In response the Algerian government declared a state of emergency, sent in the troops and postponed the elections till December. Plus they arrested both Belhadj and Madani - and threw them in jail.

Algeria was gripped by a dramatic crisis. Here is just one moment captured on video that shows that intense mood. Ali Belhadj's young son addresses a mass rally of Islamists. The son has an amazing power and charisma - just like his father. The response from the crowd also show just what the Algerian government were unleashing through trying to trick FIS out of their election victory everyone knew was coming.

Here too are parts of a really good report shot during the crisis in June and the occupation of the squares. It captures the mood of the time - and also shows the growing fear among the middle classes about FIS and the suspicion about what the Islamists were really up to.

But just as FIS were trying to force the poison of France - and the West - out of the Algerian mind, the West - or more precisely America - came roaring back to invade and take over Somalia.

What happened in Somalia in the year between December 1992 and the end of 1993 was an extraordinary sequence of events - driven by a new idea. It was the belief that you could invade another country, not because it threatened you or because of any power politics - but because you were bursting with good intentions to save innocent people. And it all went wrong.

The UN's failure to save victims of the famine, combined with the TV images of starving children had cause outrage in America. In the face of that a powerful group of people at the top of some of the relief agencies proposed an alternative. This group have been called "the international humanitarians" and their solution, which would have been unthinkable only a few years before,was that you should go and occupy a country militarily on humanitarian grounds.

They were driven by the sincerest of motives - but it was also going to give them extraordinary power, especially as all sorts of other groups in Washington saw in their idea new opportunities for themselves now the Cold war was over. In December 1992 a wave of political pressure built up on the outgoing President Bush. It was led by the humanitarians - like Philip Johnston, the President of CARE-US who wrote letters to the press saying bluntly:

"the international community, backed by UN troops, should move in and run Somalia, because it has no government at all"

He was backed up by powerful newspaper columnists like Leslie Gelb of the New York Times who put it more bluntly - it should be a policy of "shoot to feed" he said. The US military also joined in because they were keen to prove that they could do OOTW - "Operations Other Than War". And there were powerful elements in the State Department who pushed for it, plus the UN who were feeling incredibly guilty and wanted to be seen to be doing good. UN officials told the press that 80% of food aid was being looted - which was completely untrue. The figure was more like 20% or less.

The writer and African expert Alex de Waal has written a fantastic book that details how this rush to intervention built up in the final weeks of the Bush administration - and how in the process it fatally simplified the country of Somalia. It's called Famine Crimes - and in it de Waal says:

"By the time the drumbeat for intervention reached it's crescendo, the vision of 'Somalia' in which the US marines were intervening was wholly different from the real Somalia experienced by Somalis."

And here is a fantastic, clear and thoughtful documentary about what happened next in Somalia. It was made during the first six months after the invasion by the British journalist Richard Dowden who knows the history of Africa well. He starts by going back to Victorian times to show how the British missionary David Livingston was the humanitarian interventionist of his time. Livingston was shocked by the Arab slave trade and persuaded the British government to intervene - but that inexorably led to the European takeover of Africa.

Dowden is convinced that the Somali intervention in 1992 - Operation Restore Hope - is going the same way and leading to what is in essence an imperial takeover of the country. He films fantastic detail - like the daily morning meeting of the humanitarian aid officials and the US military. It is in all but name the government of Somalia - but as Dowden shows, it completely ignores the Somalians.

And he shows all the other factors that are coming into play. The Pentagon who are desperate to keep their budgets, the ex-secretary of Defence - one Dick Cheney - who says that is not true, contrasted with the star of the show, a US marine interviewed on the street who puts it all so clearly:

"the place is filling up with American contractors all bidding to rebuild this joint. That's all the Defence Department is. We're bodyguards for American contractors ……………… You should know that - you've been to college."

Dowden's film was made in the first six months of 1992. What he missed was another factor that came into play with the new Clinton Administration in Washington which was going to give an extra twist to the story.

The Bush administration had been persuaded to invade - but they were fundamentally conservative in their outlook. Their aim was to protect the humanitarian groups who were feeding the starving Somalians. The only problem was that by the time the US Marines got there, the famine was almost over. Faced by this there was growing pressure to expand the mission - and with the Clinton administration came a new idea. Nation-building.

This meant trying to create a new peaceful state in Somalia. But that in turn meant taking on the warlords who were causing such havoc in the country. One of the most powerful of these was General Mohamed Farrah Aidid. He was the head of one of the groups who had overthrown Siad Barre and he and his militia now dominated Mogadishu.

General Aidid was far from being a simple gangster villain - which is how he was portrayed. He had served in Siad Barre's cabinet, he had been the ambassador to India, and had at one point been Barre's intelligence chief. The problem for the Americans in Mogadishu was that Aidid thought he was their friend. When they had first arrived they had turned to him to help protect the aid agencies, had told the press what a helpful person he was, and had even rented a house from him.

But now Aidid was an obstacle to nation building. This came to a head in June when a group of Pakistani UN soldiers under US control went to search some of Aidid's buildings - including Radio Mogadishu. Aidid's men ambushed them and 24 Pakistanis were killed. At that point General Aidid began to see the UN and their American sponsors in a different light. He realised that they were like a rival clan. And a vicious four-month war began.

In the war thousands of Somalis were killed. Alex de Waal has written a very powerful piece of journalism that says the US and other troops under its control were guilty of serious war crimes in Somalia. You can find it here - and it describes in detail how the months of bitter urban warfare allegedly involved the Americans firing rockets into a building where a group of Aidid's supporters were holding a meeting - killing 54 civilians, and knowingly firing missiles into a hospital full of innocent people because they though Aidid was hiding there.

This is part of a documentary made in 1994 that tells the story of this war - and how it ended with the events just outside the Olympic Hotel in October 1993. They are the real events later told in Black Hawk Down.

In Algeria a bloody terrorist war had also begun. But this was going to last for six years.

At the start of 1992 the Algerian Army stepped in and took power in a coup. They did this because FIS had won the first round of the postponed national elections - and the army argued that they had to cancel all elections "in order to defend democracy". FIS was dissolved and more than 40,000 people were arrested and sent to camps deep in the Sahara.

For a year things were relatively quiet. But then armed jihadist groups emerged out of the hardline remnants of FIS and other Islamists. The key figure who united a number of the groups into what was called the GIA - The Armed Islamic Group - was a car mechanic from Algiers called Abdelhaq Layada.

In an interview in March 1993 Layada spelled out his theory. He quoted Oswald Spengler on the decline of the West, and Bertrand Russell on how "the white man has had his day" - and said that a new Islamist society would arise instead. He then explained how those who stood in its way should be killed.

Layada justified this by using a theory which lay at the heart of modern Islamism but which was extremely dangerous because it was so imprecise. It said that those who had become involved with western style politics and power had entered into a state of barbarism or "Jahiliyyah" and that this meant they were no longer Muslims. That, in turn, could be interpreted as meaning that they were impious, or "takfir" - and that meant you could kill them.

The danger was that there was no objective way of defining who was impious or not. Layada said that it included not just politicians, but anyone having anything to do with the politicians. So, starting in March 1993, the Islamists began assassinating intellectuals, journalists, professionals - like doctors, and academics. Many had nothing to do with the regime, but in the eyes of the young Hittistes they were hated French speaking intellectuals, and that was enough.

Then Layada was arrested. The new GIA leader was called Djafar al-Afghani - because he had fought in Afghanistan - and he broadened the category of who could be killed even further. It was correct, he said, to kill godless foreigners as well as godless Algerians. The dangerous logic of Islamism's unthought-out theories was beginning to take hold of the movement.

Here is part of a film made about the Islamist terrorist movement in Algeria. It was shot in 1994 as the killing was broadening out. The programme went and filmed the Islamist fighters in a camp in the mountains, and it is very odd. They are all posing for the cameras, and it has the mood of a holiday camp. They all lie around watching TV, then get up to send death threats by fax, and settle down at the sewing machine to stitch leather bullet belts.

The weirdest bit is where "the bomb-maker" shows how he makes a giant bomb using a gas canister. It's like a grotesque cookery programme - even down to the seasoning he drops in at the end - horrible small bits of metal shrapnel to make it kill more people. Come die with me.

Then in September 1994 something very strange happened to the GIA. After two of it's leaders were killed, a new leader was appointed called Djamel Zitouni who was the son of a poultry merchant.

He escalated the violence - first of all he started a terrorist campaign in France. Then he followed the logic of who could be killed to include a new category. Zitouni declared that all the other Islamist factions that didn't agree with him were also godless - and so they should all be killed as well.

It didn't stop there. Throughout 1995 there were purges within the GIA itself - which seemed to mean that Zitouni had decided even those around him were impious too. His rivals in the GIA, and a number of journalists, were convinced though that what was really happening was that the Algerian intelligence agencies were manipulating him. That the intelligence agents were cleverly twisting the mad logic of Islamism so that it would end up destroying itself.

Or maybe not. Because it got even weirder and more horrific. In July 1996 Zitouni was shot and Antar Zouabri took over.

The GIA then began to kill hundreds of ordinary Algerians in a series of horrific massacres - culminating in terrible bloodbaths in August and September. The GIA put out a communique taking responsibility for the killing and saying it was justified because everyone in Algeria who didn't join the GIA was now impious and thus should be killed. As the historian of Islamism, Gilles Kepel, put it - the GIA had decided on the excommunication of the whole society. Except for themselves, everyone else must be killed.

It was mad, and the Islamist movement in Algeria fell apart.

Here are some extracts from a report on the horror that was being created both by the terrorists and by the government. It begins with a woman called Zora who is a very perky PR for the military rulers. What follows shows the reality.

At the heart of both these stories - in Somalia and in Algeria - is is a simple question. Could the chaos and the horror have been avoided? Was it inevitable?

In both cases there are fascinating clues.

In Somalia the evidence lies in a place that, as far as the rest of the world is concerned, doesn't exist. It is the independent Republic of Somaliland - which is the large northern part of Somalia that in 1991 seceded and set up as an independent self-governing state. Since then the people of Somaliland have built - from the bottom up - a safe, successful and democratic state without any foreign aid or outside intervention.

It is a fascinating story that has gone almost completely unnoticed and unreported. After 1991 thousands of Somalis who had fled abroad came back and began to set up in the new state all sorts of individual business initiatives that gradually rebuilt the state without any centralised guidance. They became known as "The Somaliland Pioneers". Then out of that emerged a new idea of how to take the existing Somali clan system and integrate it with the idea of multi-party democracy. This idea itself was developed democratically out of a series of grassroots meetings - not imposed by western 'nation-builders'.

The journalist Mary Harper is the BBC World Service African Editor and she has written an absolutely brilliant book about Somalia and its recent history. Anyone who wants to understand what has really gone on in that country should read it. It's called Getting Somalia Wrong and in it she is absolutely clear about what the story of Somaliland means:

"Because Western models of peacemaking and state-building have not been imposed from the outside, Somaliland has in many ways saved itself from the fate of Somalia. The example of Somaliland has demonstrated that, when left to themselves, Somalis can form a viable nation state.

After breaking away from Somalia in 1991 the people of Somaliland looked deep into their own traditions, building a system, which was initially based on clan politics, but over time incorporated more modern political institutions and processes. A hybrid system of government was designed, whereby Western-style institutions were fused with more traditional forms of social and political organisation.

And it was rooted in a popular consciousness rather than imposed from above."

This is the government of Somaliland - a country that is unrecognised by every other nation in the world

There are also practically no BBC television reports about what has happened in Somaliland - except for a film made by the intrepid reporter Simon Reeve in 2005. He went to look for this place that everyone in authority said didn't exist. It's a great film full of great facts. At one point he visits an airport that was originally built by the Soviet Union - which has the longest runway in Africa. The Americans then later used it for the most unexpected reason.

In Algeria the clue as to what might have happened lies buried in the chaos of 1991 when the Islamist Party FIS was heading for power.

In the first round of the national elections in December 1991 FIS lost nearly a million votes compared to the local elections the year before. FIS still won comfortably - but the big drop in support showed that the Islamists had begun to frighten and alienate large sections of the Algerian middle classes. The historian of Islamism, Gilles Kepel, has argued that this meant that FIS had passed its peak as a political force.

No one can know if that was true - but if FIS had been left alone they would have had to face the fact they were alienating a very powerful section of the urban middle classes in Algeria. Then they might have had to come to terms with the realities of power in complex modern societies.

Instead they were forced out into the barren wilderness - both literally, and in their imaginations. This led to violent illusions. A simplified vision of Algerian society took hold of the Islamists' minds - divided between good Hittistes and the bad westernised elites who had been poisoned by democracy. And that led them to believe they could use violence to force the kind of society they wanted into existence.

But maybe the same thing happened to the West in Somalia with the humanitarian intervention?

In 1992 a loose group of humanitarian internationalists, policy wonks, politicians and TV journalists invented a simplified vision of Somalia. It was a country full of innocent dying children and evil warlords riding around on their technicals. This was a picture almost completely detached from the complex questions of power that were causing such chaos in Somalia.

Then, when things didn't go as simply as the humanitarian vision predicted - and the westerners got inevitably caught up in the local struggles - they turned to violence to try and enforce a simplified vision of democracy and nation-building on the country.

And we still haven't learnt.

For ten years after 1994 Somalia descended into chaos. It wasn't caused by the failed western intervention - the Somalis bear a great responsibility. But then, in 2005 an Islamist movement emerged called the Islamic Courts Union which began to impose a new kind of order and stability in areas of the country using sharia courts. Mary Harper, and other journalists who know Somalia, have argued that the Islamic Courts were a local grassroots attempt to create order and essential services - similar to what had happened in Somaliland to the north. It's important not to romanticise them, but they were a fragmented and essentially local version of political Islam - with the violent extremists very much in the minority.

America's response was to immediately label the ICU as yet another part of a global jihad network linked to "Al Qaeda". Within six months Somalia was invaded by Ethiopia - backed by America and supported by giant US gunships and a naval fleet. The ICU fled - but then the Islamist movement re-emerged in a much more violent form in the shape of a group called Al Shabaab. Mary Harper says that America's intervention in Somalia had created the very thing it feared.

And the same thing is happening now all across the northern part of Africa. In Mali, in northern Nigeria with Boko Haram, and in Algeria with the remnants of the GIA. In every case what are local struggles for power are being simplified by Western politicians and commentators into part of a global battle against "Al Qaeda".

It is true that there are extreme Islamists involved who proudly announce that they are joined together into a global movement. But the reality is that that kind of extreme Islamism has failed everywhere. Ever since Algeria in the early 1990s none of the extremist salafist-jihad groups have managed to take power and create the kind of society they yearn for. The reason for their failure is simple - the growing urban middle classes throughout the Arab world don't want it. You only have to look at the battles now tearing Egypt apart to see that happening.

Instead our politicians and allied terror experts fall for the Islamists' attempts to aggrandise themselves - and in the process become the Islamists' PR agents. It means the western elites are helping to promote a failed revolutionary movement while ignoring the signs of what might be the future for Africa - the new systems of multi-party democracy being built from the grassroots in places like Somaliland. Without aid, and without the west imposing centralised forms of control.

Meanwhile most western aid agencies working in Africa have a very firm policy. They do not talk to the press or TV any longer. They keep what they are doing completely secret. Given what happened in places like Somalia it is a very sensible policy. But it leaves us and our leaders ever more lost in the wood looking for the baddies hidden behind the trees.

HEAVY PETTING

2012-12-20T15:19:40+00:00

THE POLITICAL USE AND ABUSE OF ANIMALS ON TV

Animals have been a central part of television from the very beginning. But over that time the way animals are portrayed on TV has varied enormously - not just in the way they are filmed, but in the stories they are used to tell the viewers.

And the truth is that the animal programmes are far more about us than they are about the animals. They are really about how we see ourselves. I have always been convinced that animal programmes are one of the most powerful ideological expressions of our time - telling stories that both express and reinforce how we understand our relationship to each other socially and politically in powerfully emotional ways.

Over the past thirty years the wildlife programme has been dominant, led by David Attenborough. The story these programmes tell is a deeply conservative one. The central, natural, unit that the films portray is the family - and they tend to follow that social unit through repeated cycles of birth, discovery, danger and tragedy - followed by the birth of the next generation who will repeat the cycle.

The backdrop to this story is the endless repetition of the seasons - "spring returns and the first green shoots force their way through the melting snows" - which gives the cycle a natural inevitability that reflects and echoes back to us the static conservatism of our age.

But it wasn't always like this - and for Christmas I want to tell the story of the far more larky and chaotic age of animal programmes that came before in the 1970s and early 1980s.

It is The Age of the Talented Pet. It was a way of portraying animals on TV that was not only very funny - but was also equally a powerful ideological expression of the politics and aspirations of the time. I don't think this has been properly recognised and I would like to set the record straight.

To put the age of the talented pet programme fully in context it is necessary to start with the way television portrayed animals - and pets in particular - before that, in the 1960s.

I have found in the archives an absolutely wonderful film made in 1969 about the relationship between pets and their owners. It is called Love of a Kind, and it is a series of scenes and stories about different owners and their pets. Some are very funny, others are odd and eccentric, and some are incredibly moving.

It is a really good film that is also brilliantly shot in that loose 1960s verite way where you get the feeling that the camera is just looking around as a normal person would. It also perfectly expresses the belief that underlay the counterculture notions of the 1960s - because at heart it is about eccentricity and tolerance of oddness and difference.

Everyone in it - whether animal or owner - is a distinct character who is being just what they want to be. I particularly love Flo the enormously fat and very grumpy cat who begins the film, and Benji the vicious Cairn terrier who just goes for everyone - including his owner and her close friend. Their dialogue as they discuss why Benji does this is great.

The film is about how individuals and animals can forge deep emotional relationships - yet still fully be themselves in all their awkward and grumpy ways. It shows these deep bonds in some incredibly moving ways. The scene where an old woman waits while her dachshund is operated on by the vet is just heartbreaking and so moving.

But then, at the end, the credits reveal that the film was shot and directed by Lord Snowdon - and you can't but help get the feeling that what the film is really expressing is a traditional One-Nation Tory fantasy about the world - where everyone can be funny eccentrics and be happy, providing that they all know their place on the estate. Perhaps that was always the idea that underlay the hippie dream.

The problem was that by the end of the 1960s more and more ordinary people didn't want to be patronised by the upper middle class elites in Britain and kept in their place. They didn't want to be told what was the right way to think and behave - because that somehow implied that the elites knew what was right, and so were cleverer than everyone else.

This rebellious feeling rose up among many ordinary people in the 1970s and would later be co-opted by the right under the term "aspirational". At its heart was a conviction among those people that they were just as clever as the patronising elites.

And as this feeling rose up so did a new type of animal programme on British television. Talented pets were animals who wanted to be as clever as their owners and took great delight in showing that they could do many of the things that humans could - like talk or sing or dance or even skateboard.

Here is one classic example. It is Meg the Counting Dog and her owner Mrs Martin. And Meg can not only count, she can do a lot more mathematically. Aspirational Dog.

From one perspective these short films - which were predominantly made by the programmes Nationwide and That's Life - can be seen as deeply patronising to the owners of the animals. But they didn't patronise the animals - what comes over in most of them is the sheer joy and liberation that the animals clearly feel as they behave in sometimes the silliest ways - just like humans.

Here is the one that I think is both the oddest and the funniest of all these short films. I don't want to give anything away except to say that I call it The Soda Dogs - and it makes me cry with laughter, above all because of the sheer eagerness and excitement on the dogs' faces.

And the animals got cleverer and cleverer. Here is one of the great talking dogs. He is called Domino. He only has one phrase but the film brilliantly repeats it in inventive ways. And the phrase is also a perfect expression of the world of 1980s and 90s consumer aspiration that about to come.

And notice how the power structure was shifting. Along with the talking dog is the non-talking husband, sitting next to his wife on the sofa. His only job is to feed biscuits to the dog as it talks.

But as well as being odd expressions of the new aspirations of the time, these films also express the sheer anarchic silliness of the late 1970s and early 80s.

I think that that silliness was one of the products of the economic collapse and political chaos of the post-war planned society - a free-wheeling individualism born out of a general realisation that the elites who were in charge didn't have a clue any longer about what was going on. And it was by no means inevitable that the right would grab hold of that individualism. If the left had had the imagination and courage - they too could have taken hold of it and steered Britain in a completely different direction.

And here is a collection of the best of these silly, talented, anarchic animals. The wonderful somersaulting dog, plus Shep the dog that that's going to play Salut D'Amour by Sir Edward Elgar on the piano the way he wants to - in a fabulous out-of-tune style, and the singing parrot who accompanies his policeman owner in Weymouth.

But in amongst all this new-found self-confidence among the pets of Britain there were still the ghosts of the old rigid owner-pet power structure.

Here is a beautiful moment I discovered in the live Election Programme from October 1974. It is 6.30 in the morning and the programme goes live to Downing Street. It is deserted except for one old man who is waiting to welcome Harold Wilson back as Prime Minister. With him is his dog - waiting mutely as his owner is interviewed, not allowed to do anything. He knows his place - very Old Labour.

(To be honest I have also put this in because the interview is great and the man's explanation of why he is there is beautifully logical and deadpan. Again very Old Labour)

And the talented pets got weirder and also began to exploit their special talents. Here is a short film about Balls the Bat - plus his owner Cherry Bramwell. The bat is beautiful - and I love the bit where it goes shopping - but he has begun to go commercial, having just had a starring role in a movie.

And you can also see how the old power structure between owner and pet is beginning to be reasserted by the owner. Cherry is a brilliant interviewee - because she has realised the basic law of all comic factual TV. That if an interviewee is serious about an absurd situation - they are funny. If they think it's funny - it's not funny at all.

Cherry is deadpan serious and thus funny. But in reality she is acting. The earlier innocence of the talented pets' owners is disappearing to be replaced by a controlled reality.

I want to end with a legendary moment from the Age of Talented Pets. It is a moment that millions remember - but it also shows dramatically how the silliness and anarchic stupidity was now beginning to be managed and controlled.

It is Prince, the dog from Leeds that said "Sausages". It is very funny – but, as the owner Paul Allen admits, he is manipulating Prince's throat to make the words. He has an elaborate justification for this - but, like a flash of lightning on a dark night, it shows how the individualism of the talented animals was now being increasingly institutionalised and managed. The age of innocence was over - and you could see the reality of what Thatcherism was going to become.

Then - in the 1980s - the talented pets receded in TV. They still exist, like Pudsey the dancing dog and Simba from Top Dog model, but their place at the top table of TV culture was taken by the epic, conservative moral stories of the wildlife programmes and series.

We have lived with that portrayal of animals for thirty years, mixed in with programmes like When Animals Attack - that was started by Fox TV in the 1990s, that also has an implicit conservative message - the eternal law of the jungle.

But maybe that age is coming to an end as the boosters for our conservative age sound ever more uncertain. And at the same time the animal programming on the BBC is weakening and being challenged by the kingdom of Youtube with its wonderful range of stupid animals doing very silly things.

If animals on TV are the innocent ideological expressions of our age - maybe it is possible to look to the sneezing panda and its allied operatives on Youtube as the harbingers of what is to come. The return of the revolutionary libertarianism that was glimpsed with the joyous, anarchic talented pets of the late 1970s, before that moment of silly freedom was co-opted by the forces of reaction and market conservatism.

And the first hero of this new breed is Loukanikos - the dog that turns up to all the riots in Athens. Naturally his name translates as Sausage.