Destination festivals are an odd and unwieldy beast: if you think about just how much 35,000 pairs of Hunters, a bunch of Pimm's sick and a pile of discarded Guardian crosswords can fuck up Henham park after only four days at Latitude, it's hard to imagine the effect a festival might have when it takes over an entire city – the population of which is actually less than the number of Glastonbury tickets sold in 27 minutes last year.

On Tuesday afternoon, having taken the near-hour long bus that trudges the distinctly lunar landscape from Keflavik Airport to Reykjavik City–face pressed against the window, Iceland's bleak and unshakeably beguiling siren song–there's not much sign of a festival in the traditional sense. There's no signposts, no fluoro-vested youth, no flags, no half-pitched tents, no misleadingly pleasant-seeming portaloos. While the red, blue and green of corrugated iron houses and the grey of their taller concrete peers make for a mesmerising urban patchwork of traditional Nordic and East Bloc-reminiscent architecture, they don't even hint at the kind of tribalism you find at British festivals: if, like mine, your hotel is nautical blue, you don't have to worry that you've pitched up in a hive of dickheads or a six-day shaggathon or a hive of dickheads having a six-day shaggathon – they're just the colour of houses.

By now it's Tuesday night, and we've been here for a good few hours. We're drinking in a pub where everyone speaks English but no one else seems to be English, and the only things on the Happy Hour menu are beer (not the IPA) and wine, and the toilets are clean even if the door is secured in the open position and there are no cubicles. It's still only Tuesday night. The festival isn't even "on" yet and won't even begin to kick-off until tomorrow evening at the earliest. This isn't Utopia: this is Limbo.

Unlike Dom Cobb in Inception, however–and this is more or less the only way in which my life to date has not mirrored the many faces of Leonardo DiCaprio–our limbo doesn't last long enough to exercise our architectural prowess on a mammoth scale, get old and commit suicide via decapitation by train. Unfortunately. By Wednesday night, at 1am, at the 24-hour shop alone there is a twenty-minute queue, a fight and someone throwing a plastic table at nothing and no one in particular. The hotdog stand at which we'd eaten the previous night in a silence so oppressive one of us had, in a moment of acute social anxiety–not left the vacuum, but–turned their back in a vain effort to dull the crunch of onion, was now a hub of activity. So it goes. Of course, even at this early stage, it wasn't all garden furniture and frankfurters.

Tuesday night there had been, among other things, music.

The first of the low-key off-venue amuse-bouches is Bast Magazine and Nicky Digital's Airwaves warm-up party at Paloma; a modest bar with ceiling rafters set to mimic the inverse of the belly of a Viking ship – a little like being in a cross between Valhalla Rising and The Life Aquatic With Steve Zissou.

Opened by Brighton-formed, Icelandic-fronted outfit Dream Wife, it's not the most auspicious start. This blend of gentrified grunge and electro-pop is not going to change my life (though the guy with the big camera and the even bigger flash just might, if he fires that thing in to my corneas one more time) – Dream Wife are, by turns, brash and ethereal, reaching for something between PIL and Warpaint but never quite getting there. They finish, are duly papped by the guy with the big flash and followed by Nolo, who are, by all counts, a total surprise in their brilliant combination of A-ha-meets-MGMT synth-pop.

Time, of course, is elliptical – cyclical – and post-Paloma we find ourselves, for the first time of many times, in a situation I have already described: the silence, the snap of frankfurter skin and the crunch of onion – the horror, oh the horror; the great and terrible wonder of the Icelandic Hotdog.

Wednesday is a new day, as it tends to be providing it's Tuesday you're taking as a springboard. And with it comes a fuller schedule, a hearty breakfast with not one but two kinds of fish and the chance to join a heinously long queue at Airwaves Media Centre in which I have no business being. The latter is better than it sounds: not only does it give legitimate reason to self-deprecate when a group of Australian festivalgoers ask if this is the queue for package deals ("I don't know. I'm British. I just joined the first queue I saw; fucked if I know what's at the other end"), but this is where the wristbands are. And it is with this wristband, and the ingestion of the can of warm Gull beer that came with it, that I'm able to traipse down to KEX Hostel in the rain for Kiasmos.

Brainchild of Iceland's own Olafur Arnalds – who, with the exception of maybe fellow Erased Tapes-er Nils Frahm, is just about the biggest name in contemporary classical composition and has a BAFTA for scoring that TV show where Doctor Who solves a murder – and the Faroe Islands' Janus Rasmussen of Bloodgroup, it's a pretty seamless amalgamation of their work as artists: Kiasmos is pared-down, bass-heavy and, ultimately, radio-ready. And therein lies the rub.

Listening at first in the gloom of the grey of KEX's concrete stairwell because, inside the main room, a room that puts one in mind of a cross between the Berghain and a vaulted chapel to J.G. Ballard, it's packed to the point of one in-one out, wet shoes are tapping dry floor more or less in time with the music. And it's easy to hear why: Kiasmos is intoxicating. The combined talent of the duo is evident in the music they're making – technically brilliant, concise and built for rooms like the one I've just about got two feet through the door of by the time their set finishes. It's clearly been a sweaty affair–even those faces sandwiched in the middle of the room, faces that have been here for a while to work their way to that position, possess and unmistakable sheen that logistically can't be from the same rain as my own. But, while it seems nonsensical to rag on music for being "too good" or "too clever", there is perhaps something missing from this set and "sheen" might just be, in two ways, the operative word.

Firstly, the tunes Kiasmos are playing out with such enthusiasm, twisting knobs and hitting buttons like no one's business, have the kind of sheen you might expect to find in an all-chrome kitchen: this stuff is beautiful, practical and put together with expert finesse – but it's also a little cold; a little more clinical than the gut-punches of, say, Arnalds' Eulogy For Evolution.

Secondly, in a more liberal sense of the term, vis-à -vis Charlie, Kiasmos undoubtedly have more than the necessary talent as artists but, frustratingly, as a project, it just doesn't yet seem to have the kind of tiger blood it needs to set itself apart from the mire of sanitised daytime playlist-approved producers abundant in mainstream contemporary music. But this is among their first outings together, and you're pretty fucking lucky if you see a tiger in the first 45 minutes of a safari – right?

Later that evening, London-based Norfolkite Mat Riviere opens his set, the first of an impressive three at the festival, with a combination of sombre piano and his Ian Curtis/Paul Banks-esque vocals that move deftly from the sinkhole of out-and-out crushed to what Frank O'Hara would call "blazing [his] tirade". In fact, Riviere's songs, most from 2013's not even doom music, are not incomparable to the New York poet's ability to build and build to the point of insurmountable height and silently, almost wordlessly push you over the edge of that building. It leaves a lingering sort of ache in the pit of the stomach – one that isn't easily shifted – and begins to build, live sampling instruments and smashing at a single floor tom, until once again, barely having noticed, you're there at the precipice again.

Bastardgeist, of Chicago but recently relocated to New York, though inaccurately advertised on the program with a paranthesised UK tag, follows after with Something Completely Different. Like Riviere – so not completely different, I suppose – he's using samples and a laptop, and recording a lot of those samples live on the fly, but the real power, the thrust of his music, is delivered not in the instrumentation but via an incredible falsetto that punctuates it. It's powerfully eerie in and of itself but, combined with the myriad instrumental textures that range from pure atmosphere to drum and bass and found EVP recordings collected from the internet, it becomes something less spectral and more like all-consuming.

When the two acts have played, and I'm sitting there in an uneven-legged wooden chair, I can't help but think of a line from Patrick Marber's Closer: "Have you ever seen a human heart? It looks like a fist wrapped in blood."

You could almost be forgiven for dredging the quasi-religious lexis in fumbling, for ways to best describe M-Band's sound: "spiritual," of course, seems the easiest, the safest and the laziest option. But it's more than that, more challenging than labeling Hörður Már Bjarnason as some incense-swinging Jon Hopkins sound-alike; it's deeper, and more entrenched and resonant at the root on a very different level. What M-Band is producing with his music is something more like the feeling of the unheimlich than epiphany, more uncanny than numinous. Imagine waking up in the morning, in your family home, to find your family sitting around and having breakfast as they might any other day – except, when you sit down to join them, you realise that even though this is your family, this is definitely not your family.

Bjarnason sets up his equipment, his boards and his triggers, and I am sure of what's coming – except that isn't what comes. Laid against the backdrop of the ebb – the build and recede – of his synthesised melodies and the un-oiled bodyslam thump of his bass, the Gregorian chant-like tenor, performed live and with admirable prowess, owing as much to the most ancient kind of performed poetry as any religious call, which completes the "M-Band" modus operandi is ethereal and beautiful, but also jarring and unexpected. And it's all the more powerful for that than it would be for sonic picturesqueness alone.

There's a lot to learn from the awkward theatricality of Jaakko Eino Kalevi's performance, not least of all the myriad possible readings of dream pop: what makes the part-time tram driver's songs dreamlike isn't so much an ethereal or intangible quality, a lightness of being, but rather something transportitive about their pace and timbre. Like a dream, Kalevi's songs somehow stretch beyond their temporal means – Soft Cell-esque 'Flexible Heart', which by appearances seems to be his most popular song, clocks in at around four minutes, feels much longer, but never once loosens its grip. What Kalevi seems to have done is not so much look to emulate the feel of the dream but to apply its rules, the paradigm of the dream, to his music. (Other things learned from Jaakko Eino Kalevi are as follows: there is a right way to make overtures to your influences in the music of the past; stereotyping is a terrible, terrible thing but the brooding sensuality of this music does little to dissuade associations of noirishness with mainland Nordic states; finally, perhaps most important, it is possible as a lone front man to air drum to your own music live on stage and not look a total dick.)

To borrow philosophy from One Tree Hill, "people always leave" – and leaving Gamla Bio feels an instantly regrettable decision given the incredulity and zealotry of the Icelandic wind this evening. Thankfully, it's a regret that can be eased to some degree simply by climbing the stairs, less than ten minutes later, to Gaukurinn – a venue that feels unabashedly and tautologously venue-esque.

Bands, it turns out, are a lot like books – there are a lot of them in the world and most of them you've never heard of. Taking recommendations is a good way to go, and that's why we've slogged it down Hverfisgata and across the crossing with no buttons at which everyone waits patiently despite the fact that if you even look like you're going to cross the road in Reykjavík cars will stop as if their whole world has become a pedestrian crossing: to see Börn. And they're fucking brilliant. I'm not planning to eat any of the Hákarl (that's putrefied shark to you and I) while I'm here, so this may be as close to an Icelandic delicacy as I'm going to get – lo-fi feminist post-punk of the Bikini Kill ilk with lyrics that I can't understand but am assured by a local I'd love because they're about "being a woman, feminism, not shaving, that kind of thing." This is music that knows where it comes from, where its roots are, but is also capable of pulling up those same roots with a palpable and genuine ferocity. It's a short-lived, rapturous and disruptive set and when it's over I'm not sure how to feel – which, come to think of it, is exactly how I should have been feeling.

Brain chemistry unbalanced by punk and brennivín we leave Gaukurinn to check out the Harpa Concert Hall – the festival's main venue – for the first time. It's very much a building of contrast: on my first visit to Iceland construction had been abandoned in light of the financial crash and Harpa was considered something of a blight on the city – six years later and it's the shimmering axis of an international music festival. Its façade is beautiful – a dense, effervescent honeycomb designed by Danish-Icelandic artist Ólafur Eliasson (of 'The Weather Project' fame) – but internally, walking through, it feels more like Heathrow Airport.

Saturday evening we find ourselves once more at Harpa, back in the eye of the storm – in its Kaldalón concert hall – for Reykjavík-based Americana-tinged, quasi-psychedelic, morose-folk trio Just Another Snake Cult, joined for the night by Joel Midden of Bastardgeist. I'm not sure what I'm expecting, but it's not exactly what I get: the cello and staccato bow-bashed guitar on 'Spell Of Platonic Reversal' recall the despair of Who Will Survive And What Will Be Left of Them?-era Murder by Death and could easily provide an alternate soundtrack to the opening scene of Wim Wenders' Paris, Texas. The set culminates in a cover of Wilco and Billy Bragg's 'Way Over Yonder In The Minor Key', from their collaborative album of unheard Woody Guthrie material, and brings to it a haunting quality that is – if anything – better and more fitting than the original. Thor Bogason's vocals – darkling, intense, straining, swirling vortex-like around Kaldalón – give authenticity to what, by all rights, should be an echo of an echo. When he sings 'Ain't nobody that can sing like me', for a moment, I really do believe him.

We don't stay for the full Future Islands set – I've really only conceded to heading over to Reykjavík Art Museum, its share of the festival's concerts taking place in an imposing cavern something like the Tate Modern's Turbine Hall, to have my suspicions about the band either confirmed or debunked. As it turns out, unhelpfully, I'm provided with evidence for both cases. The band themselves aren't bad at all, and that bizarre honking sound Samuel T. Herring makes upon the microphone has a certain charm in a live setting – though on record it sounds like someone stuck an omnidirectional microphone in the Baboon cage during mating season – and the songs themselves are catchy and well put-together. Suspicion debunked: Future Islands are not shit. But, although we can't see what's happening from this far back, every once in a while a loud cheer goes up from those at the front and it's clear that Herring is throwing his trademark shapes – strange, unnamed shapes that you couldn't draw on squared paper because they're so fucked up. Suspicion confirmed: novelty value is at play. There's no denying it, though, there is something touching about this band – showmanship is a big part of their art, but Christ alive they're some groovy fuckers.

In theory it should be quite a neat segue into How To Dress Well, back over at Gamla Bio, because he's a bit of a groovy fucker too – but opinion is divided, even in our party of three, on his Sad White Male R&B shtick to the extent I feel apologetic afterward for having enjoyed it so much. Tom Krell's vocals are far from potent even on the reverbed mic and the movement of his gangly frame doesn't have anything like the allure of a Herring or a Kalevi. Yet these are all observations with the benefit of hindsight and, at the time, I know for sure that I'm digging it – throwing my own shapes you'd need three-dimensional pyramid paper to properly sketch out – it's only when he stops playing, stops clicking his fingers and trying ever so hard to be Justin Timberlake that you wonder for a moment if he just might rip off his skin and reveal the JT inside, that the doubt starts to loom.

At around 2am we leave a basement venue that I can't remember the name of or the name of what we'd thought about going to see there and decided against – I'm not sure I ever knew, come to think of it, that moniker forever lost, like tears in rain. But I spot it then, milky and dull and we walk up a small hill with a statue on it and stand there as the aurora unfolds into pale green ribbons – not the in-your-face time-lapse postcard shot, but majestic and demure in a clear Reykjavík sky.