Photograph by Steve Schofield

“I have trouble leaving the twentieth century. And I’m sure I’m not alone in that.”

Jake Arnott enjoyed a more eventful twentieth century than most: born in Aylesbury 1961, his early professional life included stints as a mortuary technician, artist's model, actor (he played a mummy in, wait for it, The Mummy), theatre agency assistant, building site labourer and care worker. He arrived on the literary scene with a puff of smoke and a flash of light in 1999, landing a six-figure deal for The Long Firm, his hardboiled but richly imagined story of charismatic East End gangster Harry Starks and the fragile characters drawn into his dangerous orbit. This electrifying, 60s-set debut, later adapted into a BAFTA-winning mini-series by the BBC, led to Arnott being dismissed (or celebrated, depending on the company you keep) by many as a sort of New Lad crime writer. Nothing could be further from the truth: underneath the flinty surface of The Long Firm's prose lurks real insight, real depth of feeling. Arnott got his own back in the underrated Truecrime, a mordant satire on the 1990s fetish for "geezer chic", completing an informal Long Firm trilogy that also includes 2001's devastating He Kills Coppers.

In Johnny Come Home (2003), Arnott dialled back to the criminal milieu of the 1970s, this time attending to the political violence of The Angry Brigade and the suburban gender/sexual revolution ushered in by glam rock. Openly bisexual himself, Arnott has politely rejected the ‘gay writer’ tag he had foisted upon him in the early 2000s, but homosexuality is a recurrent theme in his work, most notably in 2005’s The Devil’s Paintbrush, which features as its protagonist Victorian military hero Hector McDonald, who killed himself in 1903 following allegations of gay activity with local boys from Ceylon.

Arnott's books have always been ambitious in scale, but his latest offering, The House of Rumour, represents a colossal enlargement. If Arnott has always been a historical novelist, then The House of Rumour is about the very nature of history; and despite being paced like a conspiracy thriller, it proffers no easy answers, no all-encompassing explanations. The events of this darkly suggestive story, which is structured according to the Tarot Arcana, bring us right up the present day, but its real focus is that ‘time and space, seventy years ago, when the whole world was on the edge.’

It opens amid the radical, idealistic SF scene of 1930s California, and before long the narrative has swollen to take in Rudolf Hess's unauthorised flight to Scotland at the climax of WWII, Ian Fleming, Aleister Crowley, Hollywood B-movies, L. Ron Hubbard, heartthrob rocket scientist Jack Parsons, UFOs, the Jonestown massacre in Guyana, a budding Cuban cosmonaut, Jorge Luis Borges, David Bowie, a transvestite prostitute, a drug-fried rockstar, the pre-war Bloomsbury literary scene, the closeted head of the British secret service, and so on. Arnott's achievement is that he explores such varied persons and terrain without falling into glibness or whimsicality; indeed, The House of Rumour is his most elegiac, thought-provoking and serious novel to date.

Earlier this year the author - a warm, inquisitive and self-effacing character, unrecognisable from his bullish press photos - was paid a visit by The Quietus at the office where he writes in Holborn, to discuss the extraordinary plot machinations behind The House of Rumour and his acute nostalgia not only for the past, but for the future as well.

Although your past novels have have been panoramic in scope, bringing together disparate characters, consciousnesses and plot-strands, in The House of Rumour you roam further than you ever have before, both temporally and geographically. And the action isn’t even restricted to Earth. It’s a story that takes place on an almost cosmic scale...

“There's been quite a lot of it about recently…the idea of a sort of open-structured novel. Roberto Bolaño's 2666 made quite an impression on me. You can tell it's written by someone who didn't have very much time [Bolaño’s health was in serious decline at the time of writing, and the novel was published posthumously]…because he just puts everything into it. You know, this is it, this is going to be the last big thing. So make it big. And I suddenly just thought god, maybe I'm never going to get around to doing something on that scale. Being told that you've got a limited amount of time isn't something that one hopes for, but in terms of establishing a deadline, it does help focus you [laughs].

I don't think 2666 is Bolaño's best work at all, it rambles and goes all over the place. But the scope and the ambition did inspire people. It’s no surprise that, say, Hari Kunzru's Gods Without Men came out just before mine. It’s big, it has that open structure. At a time when everyone seems to think that the novel's passé and everyone's going back to the 19th century and doing late Henry James, now's the time to say sure, postmodernism might be over, but modernism still lives. We haven't got over modernism. There are still so many things to try.

How did you hit upon the subject matter? And indeed, what would you say the book’s subject matter is? It takes in alternative religion, espionage, science fiction, Nazi occultism and much else besides...

I suppose I had this idea of American science fiction and belief systems in America and how they relate to each other... this idea of the American Gnostic, as Larry calls his novel within the novel. Sort of a play on American Gothic. I think there is this sort of American Gnostic movement in the post-WWII years - a strange new world that’s inventing its own mythology, and at the heart of it is the flying saucer. The UFO is the messenger from heaven, it takes on the role of the angel or messenger figure that you get in most religions - all of them extra-terrestrials, of course.

From the Crowley stuff I did in the last novel [The Devil’s Paintbrush] I came across the idea of Ian Fleming being involved in this strange plot, Operation Mistletoe, that still no one can get to the bottom of. Then through that I arrived at Katherine Burdekin and her strange novel Swastika Night - the first Nazi dystopia in novel form, really - where she talks about this Hess character coming to Scotland, and it's written in 1937. I don't believe that people can predict the future, but...

The Hess passages in The House of Rumour are very heightened, very lyrical… particularly towards the end, when he’s incarcerated and convinced that his jailers are trying to drug him. How did you formulate Hess as a character and as a voice?

I had to rely on secondary sources. Famously the American commandant, who I mention in The House of Rumour writes a book, did write a book [The Loneliest Man In The World, 1974] - Eugene Bird was his name - and he got into trouble for it. There are a lot of conversations with Hess that are recorded in that. A lot of the recorded conversations are about his obsession with the space race, and his observations on the Soviet cosmonauts. In the earlier stuff, there are a lot of references to his conversations with Hitler in Landsberg prison when they were locked up together for about six months, which I think cemented their non-homosexual homosexual relationship.

Hess had a very romanticised and emotional attachment to Hitler. And he had a very strange sort of life. He was mixed race; he grew up in Alexandria, had this strange colonial existence there. His mother taught him the stars; it was a seemingly idyllic youth. But then he wanted to prove himself, and to escape the family business as well - that's why the First World War was such a great moment for him. Then he wanted to fly.

His obsession with flight is of course very, very strong. His one great achievement was flying from Augsburg in Bavaria to just north of Glasgow, through god knows how many air defence systems - including his own, because nobody knows quite who knew and who didn't know about the flight. He could easily have been shot down by his own side, never mind the British. In the novel I have the radio station they use for their tracking in Denmark playing [Richard Wagner’s opera] Parsifal, which is probably far too whimsical an idea, but Hess did have this identification with Parsifal. It was a very Wagnerian interpretation of that story anyway…Wagner deliberately changed the spelling of Parsifal [from ‘Parzival’] for the opera, to imply that it meant 'pure fool' - this idea of innocence. I think that's one of the real horrors of Nazism: this obsession with innocence, with purity.

How would you describe your relationship to science fiction, particularly that of the post-war period? In The House of Rumour you concern yourself not only with the genre, but of its practitioners and their interconnected personal lives...

As often is the case, I got lucky. The Crowley-Fleming axis, with the Hess thing, then also connected with Jack Parsons, who's this ridiculously glamorous amateur rocket scientist in Pasadena, which then connects to L.Ron Hubbard as well. And you realise there is this sort of strange provenance of theosophical ideas: weird, wacky ideas transmitted through the science fiction world and into the mad, post-war American cult belief system.

Then you look what happens with what's called “the golden age of science fiction,” in the late 30s and early 40s, particularly in LA - there's this huge explosion of ideas. It's really the first time that literature engages with quantum-mechanics…they get there before anybody else, though maybe Borges gets there at around the same time, interestingly. In a really glib and often terribly pulp way, these Californian writers are trying to engage with big ideas. It reminded me of what I was reading in my early teens. I loved science fiction, largely because there were a lot of short story compilations - each story being a high-concept thought experiment with an absurd twist in the tail (something like, ‘We're all aliens’). That’s what's interesting about science fiction to a young person: the ideas.

For me [science fiction] was a gateway drug to other sorts of writing and literature. Writing that can take you somewhere even further than a distant galaxy; you know, somewhere more in the mind. The big one for me is Cat's Cradle by Kurt Vonnegut, which I read aged about 13: it begins as a science fiction novel but becomes something else entirely. I was very fond of Vonnegut, largely because he was a minimalist - he wrote short books but he wasn't holding onto ideas, he was letting them go all the time. Letting things drop, all the way through. I suppose Philip K Dick and Stanisław Lem were important for me too.

Then you have you have someone like William Gibson, still regarded by many as an SF writer but whose recent work is set in a contemporary world – the now rather than the next.

Well yes, I quote Gibson in the book - this idea that science fiction has become a component of naturalism. For me, the genre lost its edge when it started becoming nostalgic: you know, Star Wars is a thoroughly nostalgic film, it even starts with that line “A long time ago…”; and it looks back to the golden age, the pulp age. We have these strange fantasies of seeing the Holy Roman Empire in space.

You’ve pointed to the hazards of nostalgia in fiction, but The House of Rumour explicitly pines for a bygone era when individuals seemed to think and dream bigger...

I have trouble leaving the twentieth century. And I'm not sure I'm alone in that. What’s more, I was born in 1961, so a lot of what constitutes my life is already in the past. Evelyn Waugh said that after the age of 40, experiences no longer have that sort of intensity they do when you’re younger. Meaning it's very hard to write about anything that's happened after that age in your own life. And I do find that difficult, I really do. So perhaps I’m always looking back, now.

Is there a vanguard of science fiction today?

I don't know. I guess there are some interesting people. I’ve not read enough of China Mieville, but he's very ideas-led: something like The City and The City, I read that and I thought oh no, this is my least favourite literary form in the world, it's a police procedural, god, I won’t be able to bear it - but actually it's very interesting. It's less science fiction than speculative fiction, or fantasy even - in the sense of creating something out of nothing. One of the things I've realised over the years, but particularly while writing The House of Rumour, is that fantasy - if it's good, and it's got something to say - is the purest form of storytelling.

You mentioned your disdain for the police procedural, but presumably that disdain doesn’t extend to crime fiction at large? Your past work certainly borrows from that world...

Yeah, absolutely. And there are lots of crime writers that I admire. It all boils down to Dashiel Hammett for me. The first four novels [Red Harvest, The Dain Curse, The Maltese Falcon, The Glass Key] do everything you need. You know, he realised after The Maltese Falcon that Sam Spade is just not sustainable as a character; he's an appalling character. He's not like Philip Marlowe, he doesn't have this sort of existential angst; he's the Antichrist really, he's an appallingly bad person and he knows it. The Maltese Falcon is a preposterous story, but it's full of wonderful observations on how bad people are. Everyone’s trying to shaft everyone else.

Speaking of everyone trying to shaft everyone else, conspiracies are everywhere in your fiction; you’re constantly exposing hidden wiring, suggesting complex connections been apparently unrelated events. Conspiracy is, arguably, the very subject of The House Of Rumour.

There's a very long tradition of the counter-factual. Conspiracy as an overarching idea I take to mean as, there might be another - an other - world. Even going back to Ovid, where this image of the House of Rumour comes from, you see it: the idea that the line between history and mythology is slightly unclear. Historians themselves are very bitchy, very queeny about their territory - particularly now, now that the Right have taken over the historical narrative from the Marxists. Or so it appears. Certainly I think there has been a shift back to a much more reactionary view of history.

There's a moment in my last book where the head of the Ordo Templi Orrentis is giving singing lessons and breathing exercises to a young adept, and he tells her what ‘conspire’ means - it means to ‘breathe with’, quite literally. Meanwhile the internet has become the real House of Rumour - in fact Ovid obviously understood that there was going to be this thing in the future called the Internet [laughs]; his description of the House of Rumour is very, very close. It hums and reverberates; fiction is mixed with fact.

“This is the haunt of Credulity, rash Error, empty Joy and unreasoning Fear, impulsive Sedition and Whisperings of Doubtful Origin.” Speaking of credulity, did writing The House of Rumour affect your own attitude to the occult, the mystical, the unknown or unknowable?

The problem with a lot of modern discourse is that the doubt has become certainty – and I think for the writer doubt is much more important than certainty. So this rationalist, atheist, everything-must-be-proven sort of worldview is all very well, but I don't think it helps us examine the world very well, and of course we know now that physics isn't Newtonian, it's not just cause and effect; I don't know much about physics but I do know that quantum-mechanics tells us that what we observe isn't necessarily that easy to understand.

Niels Bohr, the great physicist, is said to have put up a horseshoe in his laboratory to bring him luck, and all his colleagues said, 'Niels, what are you doing with this stupid thing? How can you have this superstition in this place where we're trying to understand things rationally!?' And he said, 'Well, I've heard that one does not have to believe in it for it to work.'

Looking around at the crowdedness of your shelves, and at some of titles on them, there’s an abundance of non-fiction. That coupled with the historical detail - incidental or otherwise - in your novels suggests you’re something of a research junkie.

The joy and the frustration [of research] is that you never know what you're going to turn up. Take this book, Rocket To The Morgue by Tony Boucher. It's a murder mystery about the Mañana Literary Society, and it also has Jack Parsons as a character in it. And L. Ron Hubbard turns up and he's a pretty shifty character even then…he's called L. Lance Ripple or something like that. Boucher is interesting because you read this novel and the characters are discussing quantum stuff, and the forking path idea of their being more than one possibly reality to any event. And this is written in the year that Borges’ ‘The Garden Of Forking Paths’ was published and you think hmm, that's strange. But then you realise that Boucher was the first person to translate Borges’s story from the Spanish - and where does the translation appear? In Ellery Queen's Mystery Magazine. So Borges first appears in the pulp world anyway. Then Tony Boucher becomes the editor of the Magazine of Fantasy & Science Fiction, and his great protégé was Philip K. Dick, whose breakthrough novel [The Man In The High Castle] is a counterfactual novel about what happened if the Nazis won the war.

Space figures prominently in the novel. The Cold War space-race, the aspiring Cuban astronaut, Hess’s interest in the stars, and of course the more fanciful imaginings of Larry and his fellow SF writers.

It's very, very personal to me. As I said, I was born in '61, so some of my first memories are of watching the moon landings, following the space race, following the space race too in popular culture - because it did affect what we saw on television. Things like Star Trek were seen as being utterly relevant, because this is what it was all aiming towards.

I talk in the book about Posadism, a real Latin American Trotskyism movement which believed that contact with UFOs was actually very important, and tied it all to Marxist dialectics. For them Posadists, it was quite obvious that only an advanced society could achieve interstellar space travel, so any extraterrestrials visiting Earth are liable to come from a socialist society. Which I think is a wonderfully preposterous idea. But I think there was also this sort of general Posadism that occurred in the post-war period. In Star Trek the future is a very liberal American society, not quite Left, but it's democratic, it's multicultural.

Presumably like most people who grew up watching the space race unfold on TV, you imagined that you yourself would be living beyond the earth’s orbit one day?

It was a very strong driving force for the twentieth century. I remember very clearly wanting the Russians to win, for some reason. The cosmonauts - which sounds better than astronauts anyway - were better-looking, they had better-looking space-suits. I remember I used to, in quite a fetishistic way really, make my own space-suits, and it was all again to do with the breath...

The irony is that people talk about going into space they don’t really mean that, because they're not leaving orbit. The last time that happened was as early as '72 or '73. In the book Larry talks about what space is now, which is a really big surveillance system, essentially. All the satellites are looking back at us.

And of course the US shuttle program was shut down in 2011.

I was reading quite a good article about that somewhere on the web, then I looked at the comments and the first one was from someone in Nebraska, saying, "Shit, now we gotta ride bitch with the Russians!" - which says it all, really. It was always about that.

I had something else in the first draft of The House of Rumour that I had to get rid of, an extension of the Vita Lampada character, who’s based on a real transvestite hustler. I had heard about her doing a strange performance art piece about the space race, with this whole thing about - what's it called - the Androgynous Docking System. When the Russians and the Americans did the first link-up in space, there were very complex logistics to deal with because they'd developed very separate space programs - they were used to working with different air pressures, different this, different that. But the real problem was that docking is fucking, and neither side was willing to be fucked by the other. There were long debates about it and in the end they had to develop this sort of glory-hole in space for them both to connect to.

A climactic scene in The House of Rumour happens during the Jonestown massacre. What prompted you to incorporate this particular real-life tragedy into the narrative?

I was using the Tarot Arcana to structure the novel. A problem I have with people into the occult, or anything spiritual, is they usually don't want to take the bad stuff. So if they have a Tarot reading and the Death card comes up, they say, oh it's not about death, it's about transformation, it's a good card… No it's not! It's about death! You've got to accept it. So I thought, if I'm going to do Death, I'm really going to do Death..

The more I read about it, the more conflicting it becomes: it would have been easy to deal with if was simply a barking mad cult and they all were duped by this charismatic nutter, but the truth is that some of the people involved in it were deeply utopian and idealistic. They were very committed to anti-segregation. That's one thing you can say about Jim Jones - he really did fight against that. And it's one place where you did find in America a sort of integration. You look at films of the services: the congregation was about 60/40 black-white, and they saw themselves as being on the Left as well - this weird idea of Apostolic Socialism, which looking back is just terrifying. And of course it went horribly wrong, and Jones went completely mad…

Photograph by Cristian Barnett

Most of your previous novels have been very London-centric in terms of setting. As we’ve already discussed, the action in The House of Rumour takes us all over the world, and beyond; but it feels like California is its centre of gravity.

LA's interesting because it’s an imaginary city. All cities are cities of the imagination, but none so much as LA. So much has been written about it, it occupies so much of film culture as well, and I think noir, Western noir anyway, it always goes back to LA, where you get that very, very sharp shadow. It's also where the military-industrial complex was based, and there’s Caltech in Pasadena, and of course the Mañana Literary Society - which was this extraordinary explosion of speculative fiction writers, including L. Ron Hubbard. You had people like Asimov on the east coast - a very New York radical, Leftie, Democrat bunch - whereas the LA lot were a bit more extreme. People like [Robert] Heinlein - who started off as part of Upton Sinclair's End Poverty In California movement, but after the war becomes this very right-wing, militaristic, mad sort of libertarian - but libertarian-fascist, really.

You’re no stranger to filmed versions of your work, and I’m sure it won’t be long before The House of Rumour - despite its interiority and unfilmable structure - is optioned. Is your attitude to adaptation fairly sanguine, or are you quite protective over your work when translated into another medium?

It’s natural to be concerned with what can happen to what you write. But re-writing is interesting. The danger is when people start worrying about whether they're being 'faithful' or not. It's an interesting use of the word, because 'faithfulness', in the sense of fidelity - a sexual or emotional fidelity to someone - is the last thing you want in adaptation. You want a kind of promiscuous force. You want infidelity.

Is infidelity what you got from Joe Penhall’s adaptation of The Long Firm? It’s become something of a British TV classic.

I was very lucky indeed. Because it was BBC2, you didn't have to worry too much about the more difficult subject matter, or the bad language, or have to jazz it up too much. The only problem was that because the BBC accounts department were very clear about the fact that they do four-part drama series - and it was clearly meant to be a five-parter - so it feels a little fractured as a result. But in some ways it also gained from that: Joe made a decision to make the cut early on, meaning we went straight into the story of Teddy Thursby, the Tory politician - which helped establish that this isn't a crime story, it's about… well, if it is a crime story it's because everything's corrupt, right to the highest level. And of course having Derek Jacobi [who plays Thursby] as the first voice you hear …His response to the script was: “Pretty much the first thing I have to say is, ‘Get your cock out.'…wonderful!”

I must ask about Ian Fleming. He’s very affectionately drawn in The House of Rumour.

Fleming is such an interesting character. And he was killed by Bond. He was destroyed by this strange thing that he had to keep up with. He also had this strange relationship with Anne Fleming, who of course thought the books were utter rubbish and had this terrible snobbery. Fleming was an old Etonian, you can scarcely think of anybody with more class security than Ian Fleming, and yet he felt an oik compared to Anne - she was incredibly patrician, and they had this very charged relationship. I posit this idea in The House of Rumour that they were very happy when they were beating the hell out of each other.

All of Fleming’s intelligence training is pointed towards this new world where espionage becomes all-encompassing .You get it in Le Carré very strongly: this sense that the Second World War was a war they couldn't afford to lose, while the Cold War was a war they couldn't allow to happen. And so you get this movement into stasis. James Bond works because he's what we want spies to be like, when in fact they're not like that at all - they're completely static, nothing much happens. It's very duplicitous, but there's no real action.

In the novel I relay this story of Fleming joking to Kennedy about dropping some strange poison to make the hair in the Cubans’ beards fall out and then blaming it on the Russians or something…and the Americans are there nodding because they think he's being serious. The British were very good at lying in the Second World War: Perfidious Albion. The Germans should have beaten us but we were better at lying. Then the Americans came along and started calling it psychological warfare! They might not get the joke, but at least Americans are sincere. The horror of the British is that for us, it’s all a bit of a game.

The House of Rumour is out now, published by Sceptre

Follow @theQuietusBooks on Twitter for more