"Giant Sand is a mood," Howe Gelb has said. That mood has been sustained by Gelb for 25 years, resulting in one of the most richly idiosyncratic catalogues in American music. Chaotic rock, tender balladry, ramshackle country and wayward jazz have all found their way into Gelb's vision. It's an approach that has attracted collaborators like PJ Harvey, Victoria Williams, Evan Dando, Kristin Hersh, a Canadian gospel choir and a band of Spanish gypsies. Following the departure of Calexico's Joey Burns and John Convertino in 2002, Gelb recruited a handful of Danish musicians to become the new Giant Sand (Gelb is married to a Dane and spends his summers in Denmark). Their latest album, Blurry Blue Mountain, is another rough-hewn gem, graced with bleary-eyed piano ballads, dreamland waltzes and rollicking frontier rock.

Wry, laconic and sage-like, the 54-year-old Gelb is a highly engaging interviewee. His thoughts unfold in long, rolling sentences that don't always follow conventional rules of grammar or syntax. Words are toyed with, bent out of shape with unusual inflections and easy drawls. Sometimes he'll drop in British slang or even make up a word. In describing his art, Gelb can be both mystical and practical. He'll talk in romantic terms about capturing the 'gaseous state of song', then describe himself as a maker of things, a mechanic, or a farmer. It's a philosophy perhaps best captured by the title of one of his finest albums, 2000's Chore of Enchantment: magic exists, but it has to be worked at. Over a phone-line that only accentuates his grainy Arizona tones, The Quietus attempts to harvest Gelb's thoughts.

The last couple of Giant Sand records have featured such guests as Vic Chesnutt, Neko Case and Isobel Campbell. Being the 25th anniversary of the band, some might have expected this album to be an all-star affair, but in fact, you've stripped things back to the core group.

Howe Gelb: I enjoy not having the guests. I love my friends, but sometimes you think people are expecting guests, now you've got a reputation for having guests, and you don't want a reputation if you can help it. So pull back, just enjoy the band. It's the 25th, let's go that way. In fact, let's underscore. You like minimal production, so let's make it more minimal. Do what you like. As you get older, one of the perks is that you just gotta do what you like, what you think is good, what you thought was good all along. But now you like it even better, because you can form conclusions once you're over 50 - you feel secure about them, instead of wondering, 'Did I do the right thing?' It's a record with a new member, Nicolaj Heyman. It's spelt 'hey man', which is beautiful, because even if you forget his name and you say 'hey man', you'll be right. That's kinda lovely. Great Wurlitzer, great guitar. Fits in seamlessly. He's also Danish.

You've stripped things back, but there's nothing austere about the album. It has a warm, dreamy atmosphere...

HG: The record has a little bit of an overall atmosphere that is inbetween worlds. It's inbetween being awake and being asleep. It's that moment where you're just waking up. And it was bugging me for a long while, because I was thinking, 'There is something wrong with this shit'. Most of it's done live in the studio, it's not like it's assembled to be such. It just so happened that the three times we got together to record we were all so tired that we were falling asleep in the studio. First two times the studio was so hot we were like 'Uuhh', falling asleep. And then hearing it later, it'd be like, 'What is with this stuff? It's got this weird, bizarre flavour'.

At first I didn't like it, and thought, I don't know what to make of this stuff, because some of the songs - the short songs - are the most melodic I've ever written, and concise. And I thought, we need to redo everything and get it straighter. And then, when we did the third session and the same shit happened, it was like, okay, this is what the fates are demanding. I have to look at this and ask why is it going down like this?

So where, or what, is Blurry Blue Mountain?

HG: The title is taken from some artwork by this woman. She paints on wood and I liked her stuff. So I'm trying to come up with a title that was reflective of her work, kinda like what I did with the Provisions record. I liked the photograph. My daughter took it and I just loved it and I thought, 'What would be a good title for this photograph?', and I thought, 'Provisions'. There's the fish this person is providing and it's hard to do, but she's doing it. And this time it was 'Oh, it's a blue mountain, but it's not really clear, it's a little blurry'. And I thought 'Yeah, yeah, yeah, yeah, it's Blurry Blue Mountain.'

Now the music really sounds like the title, because when your eyes are half open everything is blurry. And it's not saying the album sounds sleepy, because there's some rockers on it, but everything in there sounds like it exists in that state of reasoning that lies at a place where most people spend no time - they move past it. Describing it to you now it sounds to me like a town that nobody stops in, but it's there and it's lovely, and it's got a landscape all its own. But everybody, they're either busy waking up so fast and being in the cold, hard logic of daylight, or they're just knackered and need to slam down into sleep, just get past that town too.

When you're sleeping you have an absurd reasoning power that makes no sense when you're awake, but when you're sleeping it's totally corresponding to reality there, it makes all the sense in the world. And when you're awake, it's also a different reality; it's not the opposite, but it's a different form of logistics that hold you fast to the power of reasoning and what we call logic here, and how to interact on this planet and with each other. But there's that point where both those reasonings, the absurd reasoning and the concrete reasoning mixes, start to go into the other, and that's where you're waking up or you're going to sleep. I remember writing one of my favourite songs like that, and that was 'Robes of Bible Black' (from The Love Songs, 1988), and all those lyrics. It was almost like being under anaesthesia where I found it really hard to get up to write those lyrics down, because I just wanted to sink into sleep. And I did it, and I'm really glad I did it, because usually you tell yourself you'll remember it later and you never do. But this whole record sounds like that little favourite spot, that's all I can tell ya.

The funniest part of it is that once I figured that out, and felt good about it, I realised that the first track starts with a lullaby where I'm playing... I think it's an old Brahms piece, on guitar. And I completely forgot I'm doing that. That's the intro to that song and it was cracking me up; perfect. But time and space have distorted measurements in that place. There's a seven minute song that I call the centrepiece, the third song 'Monk's Mountain' that sounds like it's three minutes long somehow. I don't know how it does that. Sometimes you play a three minute song that sounds like it's seven 'cos it's bad. But here that's the opposite. That's pretty cool.

You've described your approach to performing as 'a discussion with tunes attached'. I like that. It seems to capture the loose, improvisatory feel of Giant Sand's music.

HG: I used to say that we're like a jazz band, except without the talent. We're not using solos to improvise, but solos are problem solving within the song. As for problem solving within the set, if there aren't enough problems we'll make some. Meaning that if a song is in 4/4, maybe we'll try it in 6/8. Stuff like that.

The way you describe the creation of Blurry Blue Mountain suggests you're happy to just let things happen.

HG: I think it's organics. No, that's an overused term for food, duh. I just mean naturual, without the planning. No matter what you plan you don't know what crop you're gonna yield - you don't know the extreme weather conditions that might occur, or what the market's gonna allow. Nature provides one way or other. It knows better how to balance things, even though they might seem extreme at the time. So that's always been the message. That's why it's lasted so long, and also why it's never become wildly marketable or anticipated.

It's hard to pin down, package neatly?

HG: Yeah, it's basically asking too much of people, saying 'It's gonna be good, just come'. And they go, 'Aaaaww, are you sure?' 'Yeah, it's gonna be good. Even if it's bad it's gonna be good.' And it's like, 'Aw, can you give us something more to go on?' And it's like, 'Alright, it mixes Western sensibilities with pre-Western sensibilities, and it's gonna add some post-Western sensibilities on top of that.' And they go 'Alright, maybe we'll show up...'

Another element that makes your music unique is your lyrics. The love of what Seamus Heaney calls 'mouth music', the sound of words, suggests you've taken inspiration from the Beats, as well as Dylan. Any other influences?

HG: There are those out there I'd consider an inspiration, and anybody can be an inspiration those times when they deliver something I call 'a perfect harvest' and you hear it and you go 'oof, that's good', and you just can't help but feel that jazzle and go and write your own song in celebration of being so duly inspired. In the very beginning it's gotta get down to Bob Dylan. Neil Young was the guitar sound, Bob Dylan was the lyrical involvement. I chose to go the path where I don't revel in the lyrical content - in fact I find lyrics abhorrent, and I dare them to keep me interested, I dare them to exist. If I want to smash 'em around a little bit, it's not a love-hate thing, it's mostly a hate thing. And then what I'm left with is the least amount of detestfulness for a lyric.

And once I've calmed down a bit and listen I go, 'Oh, that's pretty good'. But at the time, it's just 'Uh, lyrics, just so don't get it'. But the music gets it. Now if the music is taking a break, or being cliché, or not causing any bumps, not trying to be new or different or involved, then that allows for more lyric involvement, to try to use the music as a platform for lyrics that are evolved more or are going off in a direction that's maybe new.

Your wordplay is a constant delight. I particularly like your propensity for made-up words. You coined one just there, 'Jazzle'. That's great.

HG: Yeah, the made up words – every word is made up. The made-up words are just words we haven't agreed on yet, as a people. But they're probably coming. There's probably gonna be a word, like 'tangulation', that's comin' any minute. But not for the sake of it. I don't want to feel like I have to have it, or get it in there every time. When it's useful. Right about now, they haven't built a word for this yet, so you have to come up with something, keep it moving, problem solving, keep moving.

How do you find working in Denmark and Spain compared to Arizona?

HG: I dunno, it all seems like Planet Earth to me. Yeah, I don't see really the difference in these places. If you're going to write about them then maybe you can see the borders. But the borders are all somebody else's ideas, and people think they have to remain in there or not, play the numbers game with the stamps and the computers. And it's just different. It's like, ah, the rain in Denmark is a beautiful, soft, velvety rain, that's often slanted; the smiles in Cordoba are just so lulling and welcoming that you're gonna have a hard time leaving. You harbour the most poignant points of interest on Planet Earth and they're just everywhere.

I like your phrase 'sonic nations' - the idea that music creates an imaginary space which people from around the world can share.

HG: I'm glad you picked up on that. You don't realise it until you get it, and the first couple of places you write it off, like it's not a phenomenon. And then the more you go out there, you realise, this is far flung, it's not only New Zealand, it's not only New Jersey, it's also in Sussex. Then you're reporting back like a scout. This is goin' on out there, you guys are all united and you don't know it.

With that in mind, do you think the idea that the Arizona desert influences the sound of your music is overplayed?

HG: All the time. Way too much. That's when my rhythm section [Burns and Convertino] figured out how to capitalise on it, and that was cool. I think many times, especially in the rest of Europe - [but] not so much here - we got a lot of our gigs because we come from that part of the world and I think people thought it was an exotic place. But none of them, those other countries in Europe, could understand what I was talking about. That's one of the drags, to put in all that time in those territories which I enjoyed very much, but to have those lyrics be so playful, so slang-ridden and then never know it.

The same on the Alegrias record [the 2010 album credited to Howe Gelb and a Band of Gypsies]. They are embracing that record in Spain because it comfortably melds the indie-rock mentality with flamenco - real flamenco - without it becoming a tourist trap. The exact same thing I heard about the gospel record ['Sno Angel, from 2006, recorded with a Canadian gospel choir]. Too many records with gospel choirs sound a little bit forced, and every record when someone tries to do this with the flamenco the same thing happens, it just turns into a joke. I dunno about that, because I stayed away from all the flamenco beats, but the guitar playing was so beautiful and it stems from the roots of flamenco, and how they attack it, and handle it, and hold it, and the rhythm – that's all infused there. But any exotic beats there were more Cuban or Brazilian. That was a place where both groups met, because I love to play that stuff on piano more and more, listen to it, and they were really at home with that beat.

Tell me about how Alegrias came about. I understand you've been a regular visitor to Cordoba since 2003.

HG: I enjoyed my stay there... my visit, the hospitality of Fernando Pacas. He put me up at his house. He enticed me there by saying he would gather the gypsies to play with me to see how that would be and I found that impossible to imagine, so I had to go check that out and it turned out better than I could have imagined. We started that about three years ago, I guess. But none of these records take that long to make. They're made very quickly. I sit on the songs for a while until a point where I say ok, let's bundle these up and take them to market. And call it an album.

There are new songs on there, but, as with 'Sno Angel, you've revisited older songs, such as 'Blood Orange', which had a strong Hispanic flavour in the first place.

HG: Well, the lyric had to do with the heat. It's about trying to make love when it's so hot. And then Guy Garvey pointed out that he nicked one of my lyrics, or he borrowed it, for Elbow, from that song, the one about 'kissing like it's our last meal'. I think it was on their last record maybe, or whatever song was the single. He said that way back when he had the radio show in Manchester, and he invited me down to apologise for it, or whatever, cos he said he wrote it and he thought it was great then he realised, 'oh is that mine?' And he had his people look into it, they couldn't find it anywhere. He thought it was maybe a Nick Cave thing. Everything was fine, he got the okay, nobody came back with any information, he felt good; and then he had his iPod on shuffle and the original version of that song came on from The Listener and he went 'aaawww'. So that was kinda cool, that was sweet of him to think it was so good. It's the only way to know. How else do you know it's a good lyric or not unless somebody else wants to use it?

So! The same thing happened down in Spain as it did in Canada with the gospel choir: when the session begins to happen you don't make any plans for it, it just happens. You need songs, you need something to play, so either you have to make something up on the spot, or you're so inspired by the possibilites you write something right before it, or even maybe after it. The first day, you're still jazzed and you go to your room and write. And, fortunately I have all those old songs, so I can take a quick look back and think what songs fit the format, or what songs I think fit the format, as I did with 'Sno Angel. I figured out some of my chords that were more flamenco, and thought, give that a shot.

You've said that when you started playing with the gypsies, it was your stride piano playing that attracted them.

HG: I don't know if that's my imagination or what, but man, when you meet those guys, you get the feeling that... we couldn't speak the language together, so it's like, alright, I can go brrrum, I can go ching, I can play any instrument in this room - now what can you do? Why am I here to work with you? I don't know you. And they go, 'This piano, check this out', and I start going into 'Ain't Misbehaving' or one of my original stride pieces, and they go, 'Oh, haven't done that yet. Haven't got there yet'. And I go, 'Okay, that's just an American thing, I had to grow up with it, just like you guys had to grow up with flamenco'. It makes us not so much equal, but it's some kind of allowance.

I can't just play guitar with those guys. I can't hold a candle to the way they play guitar. My method of playing guitar to begin with is bizarre, and if anything it kind of entertains them. The best thing about playing with them is a lot like what happens in jazz: when you play a lick, or you sing a cool line in the song that they haven't heard before, like some kind of an improvisation, they comment on it in the middle of the song right away, like 'Ole!'. And 'Oh Howe' - they call your name. Their voices are so low and gravelly. It's pretty fantastic. It's like a dream.

Some of the musicians on Alegrias have connections with the flamenco guitarist Tomatito, who you were a fan of back in the '80s. And then the drummer on 'Sno Angel was Jeremy Gara, now of Arcade Fire, whose grandfather was Alvino Rey, one of your early guitar heroes. It seems like there's some kind of curious serendipity, where your past influences turn out to have connections with you current collaborators.

HG: Yeah, but you know these don't make any sense in the real world. They perhaps make more sense in the dream world, the sleeping world. The drummer wasn't in Arcade Fire at the time, it was the gig he got directly after it, and then I found out that Alvino Rey was his grandfather. He was the guy that first thrilled me with guitar when I was eight. And with this, I had listened to Tomatito, but he didn't play on the record. Raimundo Amador, he and Tomatito used to get together and play, they were from the same town. There was something about the intro Amador did for 'Cowboy Boots' that sounded so Tomatito-ish - it was like, wait a minute, there's something encoded in there, taking me back to where I lived in Joshua Tree in 1989. I used to listen to that all the time up there. Tomatito, Tom Waits's Swordfishtrombones, and Miles Davis's Lift To The Scaffold soundtrack. Those three records are the ones that got played over and over out there, blaring. And they, together, sounded to me like the landscape of the desert.

To mark the 25th anniversary of Giant Sand, there's an extensive reissue programme underway. Do you have any particular favourites, or do you prefer not think like that?

HG: Looking back is never really all that healthy or good. I don't think it serves as body of work. It can be called that for want of any better term. But I think that it's kinda like an endless and infinite spin of yarn that's been out there for the ages since man was alive. And you get together and you make what some people call magic or voodoo, or tapping into some other world without gravity. You cut little holes in gravity and it makes you feel buoyant. Something about the element of song in all of this, that has more to do with leaking in that gaseous state of song into the solid state of gravity or reality.

That's quite a romantic conception of the creative process. On the other hand, on an album like Arizona Amp and Alternator, you liken yourself to a mechanic, making songs from parts in your toolbox...

HG: You might find a better use for it later, or you might find an actual need, like ah, I wish you had that old tool I once threw away, but I could really use it right here. That's like when you shut a song and go, that's going to fit perfectly.

So what does the future hold?

HG: Well, I don't know yet. That's always a loaded question. The future takes care of itself. I want to do something with strings attached. I've been doing things for so many years with no strings attached that now I'm ready to attach some strings.

There are a couple of songs on The Listener which feature a string section...

HG: Right, that was a glimpse, so I think maybe it would do for some... 'cos something happened this year where two different string projects wanted me to do something with them. I tried and the point is that they both ended up happening live on the same night at the same time. I only had to run three blocks between them, that was really funny. But for me it's also like an omen. It felt really cool, so probably that.

Giant Sand play at Queen Elizabeth Hall on November 4. Click here for more information and tickets.