Being in a group with a devoted if limited fan base has had its share of challenges, says Stephen Duffy. "We're not Felt. But that's probably got something to do with the fact I keep working with people who alienate my core audience. The kind of people who would love The Lilac Time are put off by the fact I made a record with Alex James; I made a record with Nigel Kennedy; I started Duran Duran; I wrote with the Barenaked Ladies. People think, 'I'm not going to like this guy. He's a prostitute!'"

But whereas others in his position might have become embittered, the 54-year-old ringleader of one of Britain's least predictable folk-pop troupes is admirably nonchalant. It's a mood you suspect reflects his surroundings in Cornwall, which he fell in love with while visiting his brother Nick and where he relocated from London in 2008. He now lives there with his wife Claire, their daughter and their two dogs ("It's very relaxed. It's almost Jamaican"). "The songs I wrote and the records I made with these people, they're good," he says breezily. "I'm happy with the choices I've made." And as his group re-enter the public arena after eight years of self-imposed silence with their ninth studio album No Sad Songs, he's taking nothing for granted. "Hopefully we can get people back into the story and the music again. I successfully killed it so I'm very happy to have another opportunity to breathe life back into it."

You can add to the roll-call of names above, that of Robbie Williams, whose 2004 number one single 'Radio' and album Intensive Care from the following year were sprinkled with Duffy's stardust. As Duffy says, hardly the sort of connection his fellow Brummie Lawrence might have forged, but a collaboration that typifies the strength of instinct required to survive more than 30 years in the music industry.

"When you get together and write one song, you don't think it's going to be two songs or an album," he explains, "and the last thing you're going to do is think: 'This is going to sell eight million.' Maybe some people do but it's only going to make it sound mercenary. You have to do it for the work, but obviously it did very well.

"Before that, I'd always been lucky with the people I'd written with. Songs have done well and been in films and stuff so the publishing side has always been supportive of the gentler things like The Lilac Time."

The song that started the ball rolling was 'Kiss Me', Duffy's first solo hit almost exactly 30 years ago, but it wasn't his first foray into the world of pop. In 1978, on his first day as an art student at Birmingham Polytechnic, Duffy befriended John Taylor and his "16-year-old buddy" Nick Rhodes. "So art was a thing of the past almost immediately," he recalls. "In the first week we started making music, but that was the thing, that was the tradition. Keith Richards, John Lennon – go to art college, start a band, steal the PA."

Duran Duran played their debut gig on April 5, 1979. Three performances later, though, Duffy bailed out. "We'd have struggled to have been as big as Echo and the Bunnymen," he says by way of an explanation for such a swift exit. "It would've been fun to have done a bit more but they were ambitious in a way I wasn't. I went off and started another band then completely caved in and did 'Kiss Me' because when all your friends have been on Top Of The Pops you're like: 'I've got to do something.' The Beat, Dexy's Midnight Runners, The Specials – every time you turned on the TV there was somebody you knew."

The fruits of his fleeting spell in Duran Duran eventually emerged in 2002 when Duffy and Rhodes issued an album, Dark Circles, under the alias The Devils. "It was what might have been the first Duran album if I'd stayed. Nick and John were estranged at the time and it would have been interesting if it had been the three of us. As it was, Nick and I had enormous fun pretending it was 1979."

In the aftermath of Duran Duran, Duffy learned to play the game, for a while at least. "You thought: 'I'll put out a record and be on Top Of The Pops,'" Duffy recalls. "But 'Kiss Me' was around for three years before it was on Top Of The Pops. I kept signing with different record companies and saying: 'Can I move on now please?' and they'd say: 'No. Record another version of 'Kiss Me'.'

"I was happy with the first version, which was a dance hit in America. I wrote it in 1979 or 1980, so by 1985 it was such an old song — when you're 25 that seems like donkey's years — that as soon as it was a hit I thought: 'I can do what I like now' and started The Lilac Time, and Virgin said it didn't want anything to do with that."

Taking their name from a line in the song 'River Man' by Nick Drake, Duffy, his brother Nick and their friend Michael Weston found another label and moved to rural Herefordshire, where they began plotting a path that might seem congruous with contemporary music but which at the time was anything but. "Halfway through the eighties it was as if I'd spent all the time pretending the eighties weren't happening, escaping from the big drum sounds and the synthetic thing and trying to make small, intimate acoustic music," Duffy says.

"When we started it was only 12 or 13 years since Nick Drake had died and many people thought we were insane, playing the acoustic guitar. Now there's a greater appreciation of it. We were only 30 years too soon.

"We were signed to PolyGram, this huge international record company, and they kept saying: 'Go to America and let's record some drivetime hits.' And we were in this farmhouse without a phone, without any heating. We'd walk across the fields to the pub to use the phone box outside."

The Lilac Time called it a day in 1991 after four albums, the last of which, Astronauts, saw them briefly signed to Creation Records. Duffy spent the nineties making three solo albums (including 1993's Music In Colours with Nigel Kennedy), co-writing songs with Steven Page of Canadian rock group Barenaked Ladies and having a Britpop moment with Blur's Alex James in Me Me Me (their solitary single, 'Hanging Around', peaked at number 19 in 1996). "I always knew Alex was a Tory," Duffy laughs. "Gore Vidal put out his first memoir and Alex wouldn't read it because he thought it was the work of a left-wing beast."

Then, in 1999, a new line-up of The Lilac Time coalesced, including Duffy's future wife Claire Worrall. "By then we were on an independent label and we were like: 'Now we can do it as we wanted to do it.' And we've pretty much stuck to that. We've made the same record over and over again." Duffy laughs and swiftly adds: "I don't mean that!"

Does he find music a more pleasant, less aggressive environment now? "Ah, the great democracy of the internet," he says. "It's just all there, isn't it? A great big mess. There are no editors or A&R men. When MySpace started I'd put up music and then I'd get some nutter who'd put up terrible demos talking to me. It's like: 'Hang on a second; we're not equal. You're terrible! Don't you know who I am? I've been doing this since 1979, young man …' It's democracy on a hellish scale."

After his reformed group's eighth album, Runout Groove (2007), Duffy had grown disillusioned with releasing his own music. "It occurred to me how much things had changed," he says. "You couldn't depend on the fivers you were getting from people buying your record – you were only getting 80p or something because people weren't buying all of the tracks. You'd stream it and people would say, 'I don't need to buy it, I'm happy listening to this terrible copy.'

"There was a fair bit of time where I thought we'll make music for ourselves… but then it got good." The tone of his voice begins to rise. "And that moment comes where you think: 'I have to share this with the world. It would be criminal to keep this to myself.'"

Thus was born No Sad Songs, a gathering of sublime vignettes touched by folk, Americana and soul influences, an uplifting record born of sessions in Duffy's home studio which yielded sufficient material for a companion album he wishes to release next year.

"The other reason I felt obliged to release it is that it is a lot happier than the last record. When you sit down and write a song called 'Oh God'… You think: 'Why didn't I realise that wasn't perhaps what people want to listen to?' So No Sad Songs is like the happy ending. We all like a happy ending, don't we?"

No Sad Songs is out now on Tapete