In her 33 1/3 book on Neutral Milk Hotel's In The Aeroplane Over The Sea, Kim Cooper paints an appealing picture of the early days of the Elephant 6 collective, a bohemian utopia of communal living and pot-luck dinners, set to a soundtrack of psychedelia, jazz, Balkan folk, Tropicalia and modern classical. Caps duly doffed, then, to NMH's Jeff Mangum for putting together an ATP programme which reflects this vision. An unfortunate, if inevitable, consequence of Aeroplane's canonisation is the way its sound has been shamelessly appropriated and diminished by lesser artists, from The Decemberists with their bleating vocals and literary pretensions, to Arcade Fire with their empty kitchen sink bombast.

The upshot of the event's postponement from December is that it ended up with an arguably stronger line-up. It's hard to bemoan the loss of saxophone Olympian Colin Stetson when you're getting Roscoe Mitchell and Matana Roberts instead. ATP's own selections, including Earth, Blanck Mass and Demdike Stare, helped bring a darker, contemporary edge to proceedings. And I doubt anyone is mourning the absence of Fleet Foxes' Herbal Essences Americana.

Before going any further, a confession: huge queues mean the Quietus missed Jeff Mangum. While I'm not a religiously devoted fan, I would have liked to see Mangum, even without the stirring bagpipe solos and wonky brass fanfares of the albums. Reports of the set were overwhelmingly positive. Several Neutral Milk members made appearances, including the ubiquitous Julian Koster on bowed saw and Scott Spillane on sousaphone. At several points Mangum encouraged the crowd to join in. Mass singalongs can be an annoyance, but with Mangum they seem right, an embodiment of Elephant 6's communal spirit.

Besides Mangum, the talk of the festival was the magnificence of the Boredoms. Their live shows are always spectacular, and this new 90 minute set, in which Yamantaka Eye conducts five drummers, 14 guitarists and a bassist, was an incredible feat of co-ordination and stamina. But what I wasn't quite prepared for was the emotional impact of these performances, particularly on the second day, when the tone of the first piece was set by a minute's silence to mark the first anniversary of the Tohoku earthquake and tsunami. With its central sequence of rolling and crashing cymbal swells, the piece is distinctly oceanic, leading some to read it as a response to the disaster. Eye is an intense, even sombre, presence throughout the piece, guiding the drummers with graceful, almost ceremonial hand and body movements.

He becomes much more animated with the second piece, an extended reworking of 'Super Going', which he blasts into the stratosphere by slamming a staff across the strings of his multi-necked guitar as the drums explode around him. There is wonderful vocal interplay between Eye and a giddy, yelping Yoshimi during the third piece, which, with its samba-like rhythms, screeching whistle and distorted steel pan samples, recalls her other band OOIOO. It's the final section, however, which utterly slays. After several minutes of poly-rhythmic play and electronic sorcery, it breaks down into a pounding tattoo played by Yojiro. Then in come the massed guitars with a monumental riff, like something Led Zeppelin might have written for a Kurosawa movie. This is 'Acid Police' from 1994's Chocolate Synthesiser, re-imagined on an epic scale. It builds and builds, as the other drummers pile in and layers of guitar roar and fizz. Eye flails around the stage, singing the refrain over and over. It's overwhelming in its awesome beauty: a truly sublime experience.

Less spectacular, but just as transporting, is Roscoe Mitchell. The Art Ensemble of Chicago saxophonist is one of the most uncompromising avant-gardists to appear at ATP. Alternating between soprano and alto saxes and recorder, Mitchell is a model of control, using circular breathing to conjure long, exhilarating streams of sound. Rather than scrabble wildly around the scale, Mitchell rigorously works through his own musical language. It's as if he's reworked standard notation as a star map on which an infinite number of constellations can be traced: tight microtonal clusters and freewheeling derives, ascending scalar runs which are punctuated by rude blasts or burn up in a jet of rasping multiphonics. It's an intense listen, but well worth the effort; by the end I was levitating.

In comparison, Sun Ra Arkestra sound relatively conventional, their set based on the more accessible, big band side of their repertoire, rather than the seriously out likes of Strange Strings or Magic City. It matters not, however, because these cats swing. Cosmic fire burns through the lustrous post-bop solos of Tenor saxophonist James Stewart, while leader Marshall Allen, still spry at 88, unleashes freaky blurts of alto sax and mind-bending synth tones on his Electronic Valve Instrument. Vocalist, percussionist and alto player Knoel Scott is the hype man, picking up the mic to deliver interplanetary wisdom and entertaining with his 'space dance', a limber display of body flips which, for a 55-year-old, is quite astonishing. The straight-edge Sun Ra lifestyle clearly has its benefits.

A gorgeous set on the Friday from Matana Roberts & Seb Rochford completes the ATP jazz trilogy. Roberts is a major talent, last year's Coin Coin Chapter 1, being an instant classic. Taking in free jazz, blues and even post-rock, the suite explores Roberts' ancestry and the exploitation of women in slavery. Roberts does not attempt anything on that scale here, instead focussing on the interplay between her soulful alto sax improvisations and Rochford's sensitive percussion. Her tone is luminous, moving from graceful Ornette-style free blues to raw gospelised multiphonics. By the end of one piece, she's clearly knocked out by Rochford's subtle invention and coiled funk. 'Beautiful!', she beams.

'He's just playing the same two notes over and over' complains one uncouth young fellow of Charlemagne Palestine. Ah, but what notes! For several minutes they're all Palestine plays, but machine-like repetition is not this pianist's aim; stick with it and you begin to notice the deliberate fluctuations in speed and attack, lulling you into into a woozy trance that is only deepened with the gradual introduction of further staccato notes. Over some 40 minutes, Palestine generates a thrumming mass of extended harmonics from his Boesendorfer grand, its relentlessly hammered strings producing buzzing metallic drones and reverberating overtones. The point of the cognac he quaffs and the toys he bedecks the piano with becomes clear: they reflect the sensuous extravagance of of Palestine's compositions.

An open communion around the altar of modern classical, American Contemporary Music Ensemble's Sunday lunchtime recital of Gavin Bryars' 'Jesus Blood Never Failed Me Yet' is a moving experience. The quartet, augmented by Julian Koster's eerie bowed saw, provide luminous accompaniment to the foggy tape loop of a homeless man's dignified declaration of faith.

It's testament to the eclecticism of Mangum's curation that it's taken this long to get to the guitar acts – and I haven't even mentioned Joanna Newsom yet, who invests her beautiful songs with real guts. Earth are on fine form, their stately desert-blues rendered uncanny by Lori Goldston's wraith-like cello. 'His Teeth Did Brightly Shine' is an lustrous crawl, while a fuzz-free 'Tallahassee' is quite magnificent. The Fall are piledrivingly good, Mark E Smith even cracking a smile as he fiddles with Elena Poulou's Korg. 'Theme From Sparta FC' makes for a rousing closer.

David Yow's reformed pre-Jesus Lizard band Scratch Acid are an ugly, seething howl of post-hardcore blues, a rough alloy of the Stooges, Sabbath and Black Flag. It's exactly the kind of primal rock action the audience have been craving and a serious moshpit soon develops. Yow gamely joins in, crowdsurfing with his microphone cable wrapped around his body. The following night Lou Barlow jokes that Sebadoh are unlikely candidates to bring the macho rock energy to proceedings. But rock they do, their melancholy tunes driven by a robust rhythm section. Indie classics abound, including 'Skull', 'Careful' and a rousing 'Freed Pig'. An even less likely candidate for rock glory is Half Japanese. With his straggly grey locks and plaid shirt, Jad Fair resembles the love-child of Neil Young and Garth from Wayne's World, and he leads the band through an amped up set of garage-pop, complete with awesomely sludgy guitar leads.

After a slightly shaky start The Raincoats hit their stride with a strident 'Black & White', Electrelane's Verity Susman playing the Lora Logic sax part with gusto. Their spirited cover of the Kinks' 'Lola' inspires much singing and dancing, while the rickety punk of 'Life On The Line' is a blast. Based around Vicky Aspinall's droning violin and Ana De Silva's howl, 'The Void' is a revelation. 'No One's Little Girl' glides on Gina Birch's supple, sensual bassline, while the eerie dub-folk of 'Shouting Out Loud' is magical. A triumphant return.

Disappointments? Both Jon Spencer Blues Explosion and Magic Band sound far too much like conventional blues-rock bands. Where's the weirdness, where's the mystery? Eschewing the synths of new album, Love At the Bottom of the Sea, Magnetic Fields appear in a chamber pop configuration, featuring harmonium, cello and grand piano. The sound is elegant, but somewhat precious, lacking in tension or weight. The main problem, however, is Stephin Merritt. His baritone, to quote Art Ensemble of Chicago's 'Theme de Yo Yo', is as ripe as camembert, oozing from its skin, turning the genuinely lovely 'Come Back From San Francisco' from a wistful lover's lament to a needy moan. During the awfully clever 'I've Run Away To Join The Fairies' he risks being borne away in a miasma of smugness. Following an afternoon of mindblowing drum rituals and ecstatic jazz, the droll musings of Manhattanites with ukuleles just isn't going to cut it.

These duds fail to spoil a fantastic weekend. That the festival should end with an impromptu all-star Sun Ra jam, featuring members of the Arkestra, assorted Elephant 6 alumni, ACME and even a few Boredoms is entirely appropriate; it's a perfect embodiment of Mangum's spontaneous, communal vision.