The short plane ride from city airport to the island of Jersey is predictably full of smug-looking businessmen, pensioners on holiday, tax evaders in brass buttoned blazers - and, less predictably, a bunch of bohemian looking actors chatting away in character. This must be the Branchage flight. Sure enough, later that evening when we enter the stunning Opera House for the opening night gala, our flying companions, wearing baby blue boiler suits with matching usher caps, are welcoming us to the festival.

Every year, the Branchage International Film Festival transforms the enigmatic island of Jersey into a magical place full of film, music art and immersive environments. But the island needs little help becoming a surreal setting. It is a bit like walking into a time warp, with all the good and bad that comes with it. There's the serene, old timey feel of a postcard Britain relatively untouched by chain stores or graffiti, but there are also the occasional local yobs who also seem to be stuck in a older, less tolerant world. But most of the locals of all stripes are charming and friendly and increasingly involved in the festival.

Morris Dancers at the Way of the Morris screening

Jersey is like a microcosm of English terrain crammed into a Manhattan-sized landmass, ranging from extremely rural farms and churches to opulent town halls, ruined castles, winding streets and industrial warehouses. The festival's organisers constantly search out strange and appropriate locations to screen films across these island settings.

Our hotel is clearly the pensioners' retreat. There is frequently a queue of old folks waiting for the elevators, but the location is great and the staff helpful. At breakfast we are positioned at a small table on the wooden dancefloor, which is still illuminated by a green spotlight from the Phoenix Nights-type entertainment the night before. Everything looks green so I can't even tell the tea from the milk. When we leave everything looks red and has trails.

A typical scene in the lobby of the hotel

Our first stop is the impressive looking festival Spiegeltent, situated in the main square of St. Helier, Jersey's biggest city. Outside, I am accosted by two bubbly blonde journalists who tower over me in their heels, asking of my involvement in the festival. One of them asks as part of a survey piece what my favourite 'boy toy' is. Several hundred inappropriate jokes flash through my head until I realise this is a shopping magazine and by 'boy toy' they mean 'gadget'. Somewhere there is a very boring interview with me about dictaphones.

The festival box office nearby is, as usual, full of wonderful flotsom and jetsam: old pallets, a mounted stag's head, dolls, grunge VHS tapes and a penny farthing leant up against the wall - the kind of quirky detritus that festival director Xanthe Hamilton and artistic director Molly Carroll seem to be able to summon at will.

photo: Danny Evans

It's this kind of scene-setting magic and eye for detail that makes the famous Branchage parties so memorable. At the Friday night party, a world music themed bash at the Havana Club, Quietus chief John Doran finds himself DJing in an annex to a red curtained room that looks like the Black Lodge from Twin Peaks. His room is covered in contrasting white silk curtains, which one of the volunteers named Jackson correctly articulated as “looking like the inside of a coffin”. It's the perfect size room and is quickly filled with both locals and film types dancing shamelessly to Fela Kuti, funk and even Indian proto-techno.

The Saturday in the Spiegeltent features an unhinged costume party based around the theme of 'mythical islands'. It's a bit like being in the 60s Prisoner TV show, with all its winking absurdity - there's a booth where personal fantasies are acted out, a brass band playing and plenty of bizarre entertainment, including an 80s-themed Maypole dance. We see an elaborate cockroach costume, an astronaut with a light up helmet and Teeth Of The Sea, fresh from their REAPER performance down the road, exploring the room in matching safari hats. Later that night, there's a grunge-themed party at the Live Lounge, featuring a Nirvana tribute band – quite convincing both in appearance and sound. Where they found a decent bass player as lanky as Krist Novoselic on this tiny island is beyond me.

Rob Da Bank's King Kong Live Score

Branchage is famous for its live soundtracks, and this year is no exception. We sadly miss Rob Da Bank's King Kong live soundtrack but manage to catch Serafina Steer and her brother Sam presenting a selection of films along with a new soundtrack collaboration in a remote church on the north of the Island. The new film, called Postman's Familiar, makes little sense but contains lots of great expressive characters and obscure occult imagery – the kind of stuff that your subconscious mind grasps much better than your conscious. For this film, Serafina's harp is put through effects on her laptop, drawing out the incidental textures of fingers on strings and knuckles rapping wood to great effect.

The Charming Betty

Perhaps the highlight of the trip is the excursion to Elizabeth Castle, an island ruin just off the coast of St. Helier, to see the ongoing weather-based sound installation by Variable 4. We all board the good ship (err, amphibious vehicle) Charming Betty, which charges down the beach before gently boating the rest of the way across the bay. Over the kazoo-like loudspeaker there's a pre-recorded safety announcement followed by the original Mission Impossible theme. On the way back it's Jaws.

Like many of the fortifications of Jersey, Elizabeth seems to be layered with history: there are medieval castles, 17th century manors and, on top, modernist Nazi bunkers. Hidden inside one of the many gun turrets is some scribbling in German, a kind of crude map of what's outside the view finder. Like one of those sightseeing guides on Waterloo bridge, but for things to shoot at. Predictably there's a man in 18th century garb firing a cannon on the half hour.

Amongst all this excitement, it's often easy to forget about the strong programme of films - documentaries, fiction and shorts - which are at the heart of the festival. The documentaries range in subject from the extremities of science (Donor Unknown and Project Nim) to the charming everydayness of an independent record shop (Sound It Out). There are also provocative political documentaries - Just Do It, about climate activism, and Inside Job. One Question & Answer session for Afghanistan War documentary Restrepo ended in a heated debate.

In contrast to most festivals, Branchage Q+As are lengthy and open-ended and the Jersey audiences are keen to get involved. In a panel which I moderated for Project Nim, James Marsh's film about one of the first chimps to learn sign language, an audience member challenged representatives of Jersey's Durrell Wildlife Conservation Trust about the morality of keeping animals in captivity at all, as well as the practice of using animal actors in a film about animal abuse. Documentaries in many ways demand this kind of interactivity. They need to keep being seen and talked about, or the arguments get lost and they become merely informative.

But the main films which bookend the festival are more than informative – they contain all the drama and entertainment of the best fiction films. The opening night film is Senna, the story of the ambitious and reckless Brazilian Formula One driver who reinvigorated the sport before tragically dying in a crash. The narrative feels so fantastical as to be the stuff of myth. The passionate, handsome Senna constantly struggles against his former teammate Alain Prost with his cold, point-scoring calculus and treacherous hooked nose. Senna's 'worldly deeds' – his amazing racing victories and even his charity work - are spoken of in hushed reverence like those of a saint. Even his death seems martyr-like and predestined. But there's also plenty of space for laughs with all the humorously dated 'sports music' and commentary, and the childish petulance of the drivers themselves.

The final film, shown, like Senna, in the packed opera house, is The Great White Silence, a silent documentary about Captain Scott's ill fated journey to the South Pole. It's shown alongside Simon Fisher Turner performing his new score to the film with the Elysian Quartet. It is not only an important document but a staggeringly well photographed, constructed and written film, far ahead of its time. The first act shows the cheerful crew of the Terra Nova setting off, the second is a wonderful study of Antarctic wildlife (full of charming English humour) and the third follows Scott's intrepid push to the pole. The soundtrack is subtle and resolutely modern – setting the tone and creating transitions without overpowering the imagery. The end, which features a stripped down choral hymn, is simply devastating.

Both films are about pushing the boundaries of what is possible for humans to do. Both involve essentially futile tasks (controlling faster and faster cars; planting a colourful flag in an uninhabitable part of the world) but this kind of uniquely human hubris make for great documentary films.

So once again we find ourselves on the Condor ferry home, exhausted, carrying lots of photos and stories. It's always a pleasure to visit Branchage, a festival which seems so unconcerned with the business and politics of running a film festival in a setting that's anything but your average multiplex.