If you were to play a game of word association with the term “anarchism” what would be the likely responses? Perhaps the anarchy sign, with the capital A over a circle. Black flags. The turn-of-the-century bombers immortalised by Joseph Conrad in The Secret Agent. Or maybe Johnny Rotten singing Anarchy in the UK.

What it would be unlikely to evoke is the image of an English diplomat, a veteran of the Foreign Office and the United Nations, a man schooled in the subtle arts of negotiation and persuasion. But that is the profile of Carne Ross, a former Middle East expert in the UK’s delegation to the UN, who is said to be the inspiration for a character in John le Carré’s novel A Delicate Truth. For Ross, as a new film shows, is now of one of world’s most active proselytisers for the virtues of an anarchist revolution.

With anarchism hardly top of the political agenda, that may sound like a limited claim to fame, akin to being the world’s tallest pygmy. In fact, anarchist ideas are taking root everywhere from Grenfell Tower to Rojava, the Kurd-run area of northern Syria.

Anarchism as a political outlook is rooted in the notion of direct democracy, a polity in which power moves from the bottom upwards. Many of those protesting at the Grenfell Tower fire argue that it was a symptom of a politics that goes in the other direction, from the uncaring top down to the unheard bottom. Ross not only wants to reverse what he sees as a failed kind of democracy, but believes the crisis of “neoliberalism” has created the conditions in which people are beginning to voice their disapproval of the status quo.

“Aberrational political events such as Brexit, Trump and even the rise of Corbyn are functions of this frustration,” he tells me when we meet in a cafe 10 minutes’ walk from Grenfell Tower. The grandson of one of Bletchley Park’s wartime codebreakers, Ross had wanted to be a diplomat ever since he was a boy. One of his motivations, he says, was a desire to escape the English class system. “I wanted to live abroad in a relatively safe way,” he explains.

His accent is now faultlessly demotic, but he says it wasn’t always like that. It’s not just the accent that’s changed. His politics were once firmly grounded in the liberalism of first the SDP and then the Lib Dems. What really altered his way of thinking were two major events: the invasion of Afghanistan and the Iraq war, specifically the role of the Blair government in leading the country into conflict.

In Accidental Anarchist, a new documentary that details Ross’s political transformation, the former diplomat speaks of his disillusionment with his job following a visit to the British embassy in Kabul in 2002. When he got back, he says in the film, he had lost his faith in the British project. “I felt that the system I’d battled for and believed in wasn’t working, capitalism, democracy, the western model, whatever you called it.”

He took a year’s sabbatical and read about political alternatives. Meanwhile the war in Iraq started, following an active campaign by the Blair government to ensure that Britain took part in the invasion. In the film, Ross is damning of this decision: “They had deliberately misled the public by claiming that Iraq was a threat when it wasn’t and that there were no alternatives to war when there were. To lie to the public and to the servicemen and women you’re sending to war is the gravest of disservices… that’s the worst thing any government can possibly do.”

Ross was a friend of the government scientist and weapons expert David Kelly, who took his own life after he was exposed for briefing a journalist about exaggerated claims in the government’s infamous “dodgy dossier” on Iraq’s weapons threat. It was a tragedy that personalised a much greater upheaval that had plunged Iraq into murderous turmoil. Ross went on to give evidence that explicitly contradicted the Blair government line in both the Butler and Chilcot inquiries and resigned from his job. His outspoken stand led Le Carré, who had befriended Ross, to say in the acknowledgments of A Delicate Truth that “his example demonstrates the perils of speaking a delicate truth to power”.

The perils in Ross’s case were largely that he had to give up his ambition to become an ambassador, but not before taking the senior management exam to prove to himself that he could have made it to the top. “Here’s an egotist’s confession,” he says. “I didn’t want my colleagues to think I’d left because I failed that exam.”

He passed on his second attempt. And then set up an NGO called Independent Diplomat, which advised non-state actors, such as the government of Somaliland and South Sudan, before it gained independence, on how to conduct themselves in international diplomacy.

It would be wrong to construe from all this that Ross is a staunch non-interventionist. He loathed Saddam Hussein, but believed there was a better way of removing him than going to war. By contrast he believes the west abandoned Syria to its bloody fate by ignoring democratic forces and allowing Russia and Iran to prop up Assad’s lethal regime.

“We worked with the Syrian opposition soon after the revolution with Independent Diplomat,” he says, “and their view has always been that the only way to stop Assad is to hit him militarily. He will only agree to discuss a transition to democracy when he believes himself to be under threat.

“People have got very confused about intervention, regime change, humanitarian intervention – all these terms are bandied about. But in all international law and moral law theory, the defence of people getting killed is legitimate, whether it’s in Srebrenica or Bosnia or in Iraq or Syria.”

He also supported intervention in Libya because Gaddafi’s tanks were at the gates of Benghazi and a massive war crime was about to be perpetrated. But on these points he is out of step with the majority in “progressive” politics, including Jeremy Corbyn and the leadership of the Labour party.

“I’ve got respect for Jeremy’s position,” he says. “If you look at the global history of western intervention in the Middle East, it’s pretty disastrous. But that doesn’t mean you reject intervention in defence of people who are subjected to daily violence. This is mass torture and civilians and children being killed indiscriminately every day. And that demands a little bit more thought and detailed analysis of what can be done rather than just wringing your hands and talking about diplomacy, because it’s just not enough.”

Aside from advising the Syrian opposition, Ross has another interest in the country, namely the Kurdish-run region of Rojava. Bordered by unfriendly Turkey on one side, and maniacal Isis on the other, it’s not an easy place to get to or a comfortable one to stay. That Ross makes the journey is testament to his commitment in what’s taking place there. Rojava is run according to what Ross believes are anarchist principles: there are no hierarchies, even in the military, and political decisions are apparently determined by public meetings. The reason for this is down to a man who has been held in a Turkish prison since 1999. Abdullah Öcalan is a founding member of the Kurdistan Workers Party (PKK), which, ceasefires aside, has been waging a war of independence against Turkey since 1984.

Öcalan used to be a communist but after being imprisoned by the Turks, and initially sentenced to death, he was greatly influenced by the late American anarchist Murray Bookchin, who combined an interest in ecology with libertarian socialism to create something that he called communalism. The imprisoned Kurdish leader drew heavily on Bookchin in developing his own political theory, known as “democratic confederalism”, which was adopted by the PKK in Rojava. There is an obvious irony, of course, in a leader such as Öcalan having the power to institute such a dramatic ideological about-turn in a political movement that supposedly rejects top-down decision making.

But the Kurds do appear to have brought about a level of participatory politics that goes far beyond a vote every few years. In the film, Ross cites Rojava along with Spanish civil war-era Catalonia as shining examples of working anarchism. He also visits a commune in Spain and refers to the Brazilian experiment in the city of Porto Alegre, whose system of participatory budgeting has been praised by no less an authority than the World Bank.

Nonetheless, the success stories of anarchism are few and far between. Why is that?

“Those who have power have a strong interest in retaining it and have done a lot to suppress alternative modes of the economy or of politics. Power is a zero sum. We can’t all be more powerful. If people at the bottom are to be more powerful it means people at the top have to lose power and people don’t like giving up power.”

That’s fine, as far as it goes, but it raises several questions. How is it possible to take hold of power without violence (and Ross is opposed to violent change, he believes it’s self-defeating)? And if you do get power, how do you guard against elites emerging and subverting the process?

The answer to the second question, according to Ross and many theorists, is that you ensure the “people” retain power through mass participation. But the problem with that proviso is that, by and large, most people don’t really like sitting around in meetings for hours on end listening to other people’s arguments and complaints. If representative democracy has one overriding advantage, it’s that we appoint other people to sit in the boring meetings we don’t want to attend.

That may seem like a facile objection, but it’s surely no coincidence that direct participation of Rojava and Catalonia flourished in wartime, when the narrowed space between life and death lends urgency to the decision-making process.

“Your point is very serious,” says Ross. “I’ve looked at this question very closely – it’s a serious reservation about anarchist models. But I genuinely believe people are apathetic because they feel there’s no point in participating, that nobody will bother listening to them and decisions are taken by a tiny number in Westminster.”

He believes that if people had a say over the future of their local hospital or local school then they would show up, “however boring the meeting, because you really give a shit about it”.

Perhaps, but after a while a certain kind of person tends to show up much more than anyone else, the kind that enjoy and flourish in meetings. I tell Ross about my visit to Christiania, the hippy commune founded in Copenhagen that prided itself on its participatory meetings. After a while, only the most dedicated inhabitants who were prepared to spend their Tuesday nights discussing questions of refuse collection and how to deal with alcoholism could be bothered to attend.

And that’s not to mention the fact that some people – the most articulate – have a natural advantage in getting their opinion across, while others like nothing more than preventing anything they disagree with from happening. Ross is familiar with both tendencies from his time working with the Occupy movement in America.

In the first instance he says there are solutions. In Occupy they used what’s called the “reverse stack”, whereby the usual suspects, white men, are placed last in order of speaking and minorities and women go first. That sounds rather a crude system of identification, but Ross believes that over time these distinctions would fade as people got used to the idea that they had the right and power to speak.

As for the second: “Whenever you get a public opportunity you always get a contingent of arseholes who show up. That’s what happened at Occupy,” says Ross. Again he thinks this is symptomatic of our current situation rather than reflective of the human condition. “People are so angry now that when they get the opportunity to talk they just rage. But if you continue with the meetings, that begins to stop and you’re left with the people who want to get on and work.”

Except that wasn’t his experience in the Occupy movement, where he attempted to set up an alternative bank called the Occupy Money Cooperative. What, I ask, happened to that initiative?

“It failed for lots of reasons,” he says, and explains there were legal problems with a national credit union and it was difficult to find decent advice because all the lawyers in New York were in the pay of the banks. “There was also a lot of hostility within the Occupy movement, which had a lot of purists,” he acknowledges.

He had wanted to set up the co-operative bank by initiating a banking card that could reach the poor and marginalised, as well as everyone else, and be owned by all members. But there were only two national platforms in the US – Visa and MasterCard – and many within the movement were suspicious of working with such glaring symbols of capitalism. Ross was of the opinion that it was better to compromise and create something that actually worked.

For his pains, he says, he was “vilified in the most unpleasant way” by some in Occupy, who said that he couldn’t be trusted because he had worked for the UK government. “I had all this ad hominem abuse from these purists who do fuck all. They love to sit on their high horses and condemn everyone else.”

That, of course, is the problem with working for “the people” – other people get in the way. Still, it’s easy to be cynical and do nothing, to ignore the obvious flaws of a sclerotic democratic system, and complacently accept that there is no alternative.

Ross’s idealism is the kind that strains to find some means of politics that recognises the deepest emotions. “With the most important things to humans we have only the vaguest terms to communicate them: love, community, solidarity, meaning, purpose, spirit, soul – the feeling of them is certainly not captured by a vote or a GDP statistic. I believe anarchism promotes these things.”

Perhaps it does, or perhaps it would if most people were as conscientious and motivated as Ross. What’s clear is that liberal democracy needs to reform and revive itself. Now relocated back in the UK, and continuing the work of Independent Diplomat, Carne Ross is doubtful about working to that end because it will help sustain the system he wants to see the back of, but realistically that’s exactly what he’s engaged in, which is no bad thing.

In an age when so many on the left have retreated into the realpolitik of the international status quo, it’s refreshing to hear a genuinely progressive voice who wants to change things at home and abroad. His brand of anarchism has little to do with the anarchy of lawlessness or failed states. Rather, he wants to see a world in which we all feel we have a stake.

I can’t see that a system of endless meetings will ever capture the popular imagination, but it’s an argument well worth having, and few will make it with more passion or intelligence than a middle-aged white man who spent a large part of his career making the case for the British establishment.

• The Accidental Anarchist airs on 23 July as part of BBC4’s Storyville series. Independent Diplomat by Carne Ross is published in paperback by Hurst, £9.99. To order a copy go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99