Data Driven: Spotify’s Former Data Alchemist Explores How Streaming is Changing Music

From algorithmic recommender systems to on-demand access, Spotify’s former Data Alchemist Glenn McDonald breaks down how streaming influences music discovery and listening behaviour

The website Every Noise at Once is any music geek’s dream: an interactive map of Spotify’s music categorized into over six thousand distinct genres, sortable by popularity, release date, listener demographics, and musical characteristics. You could spend a lifetime navigating this ocean of music, diving into genres like Javanese dangut, avant-garde black metal, and Maskandi. It’s difficult to fathom so much music – and so much data about music – being accessible without streaming.

Every Noise at Once is the brainchild of software engineer and algorithmic designer Glenn McDonald, Spotify’s former “Data Alchemist” and previously an employee at the music intelligence company the Echo Nest (which Spotify acquired in 2014). At Spotify and the Echo Nest, McDonald harnessed the listening data of millions of users to help develop everything from music recommendation engines to fraud detection.

McDonald has consolidated decades of listening to, thinking about, and working with music into You Have Not Yet Heard Your Favourite Song: How Streaming Changes Music. He cautions readers that “this isn’t a Spotify book. I’m not there anymore, and my opinions here are definitely not Spotify’s corporate opinions, even if you believe a company can have opinions.” But for anyone interested in how streaming services are changing the business and culture of music, this book is a compelling – if occasionally idealistic – peek behind the curtain from someone who helped build the biggest one in the world.

You Have Not Yet Heard Your Favourite Song consists of twenty-six bite-sized chapters organized into five larger sections: ‘The Disconnected Age’, ‘How Streaming Works’, ‘New Fears’, ‘New Joys’, and ‘New Questions’. McDonald makes complicated topics such as algorithmic recommender systems and royalty models easily digestible to a non-specialist audience with prose that is approachable and frequently funny. The book targets curious listeners and music enthusiasts rather than artists, academics, or industry experts, but McDonald’s insider perspective still offers plenty to those well acquainted with the current state of streaming.

For McDonald, streaming’s biggest innovation isn’t digital circulation, playlists, or mobile listening – we have Napster, the iTunes Store, and the iPod to thank for those – but rather algorithms. On streaming services, algorithms power search engines, music recommendations, and personalised playlists, among other things. McDonald claims that “algorithms and personalisation have … made their way into virtually all features of all major music-streaming services, and since the services now mostly have the same available music, similar features and similarly-priced subscription plans, algorithms are the main point of competition.”

Like anthropologist Nick Seaver in his book Computing Taste: Algorithms and the Makers of Music Recommendation, McDonald attributes agency to the humans who build algorithms rather than the algorithms themselves. But this doesn’t make them any less dangerous. “Algorithms are just tools,” he writes, “but sometimes their makers hand them to you and sometimes we strap you to a spinning board and throw them at your head.” Algorithms and machine learning not only potentially encode the biases of their makers but can also reproduce societal inequities, as McDonald demonstrates through the notorious gender imbalance on Spotify’s algorithmically generated ‘Country Music’ playlists.

McDonald celebrates technology’s potential for music discovery but is sensitive to its limitations for understanding and serving culture. Some of his most engaging anecdotes involve parsing puzzling patterns in listening data based on cultural phenomena. For example, he explores why the Philippines and the Middle East account for so much Christmas music streaming in September and how the genre russelåter is connected to Norwegian high-school seniors forming groups, getting drunk, and blasting commissioned theme songs in cartoonishly painted tour buses.

Another one of McDonald’s central preoccupations is genre, which he defines as “a community of artists, practice and/or listeners.” Many musical genres are rooted in physical communities, but streaming’s global access to music results in others based purely on listening patterns –including ‘escape room’, one of many Spotify genres that McDonald named himself. McDonald boasts an impressive command of obscure subgenres, but while he insists that streaming provides opportunities to learn about different communities, his data-driven approach to genre sometimes feels more like musical tourism than deep cultural engagement.

Much of the book is dedicated to countering common criticisms of streaming services. In a blog post announcing the book, McDonald remarks that “I almost always end up annoyed that the story [about streaming and music] is most consistently told from the point of view of an outsider with speculative information and dour grudges.” Certain chapters, such as those addressing Spotify’s relationship to surveillance capitalism, streaming fraud, and mood playlists, come across as defensive dismissals of critiques by journalists and academics like Liz Pelly, Eric Drott, and Maria Eriksson and colleagues.

Yet these rebuttals are often too cursory to be fully persuasive. In one chapter, McDonald posits that “difficult” genres – such as jazz, his hatred of which becomes a running joke – are not declining in popularity due to streaming, but because they are less culturally relevant or based on different forms of listener engagement. Not only is this a vast oversimplification that seemingly misdirects blame toward artists themselves, but it does not adequately account for how streaming services’ metadata frameworks and features were not built for niche genres, adversely affecting their viability. Classical music’s streaming problems in particular have been extensively documented, leading to standalone classical services like Apple Music Classical, IDAGIO, and Presto Music.

In other cases, McDonald more convincingly nuances or debunks streaming myths. The chapter ‘Ed Sheeran is Taking My Money’ clears up misconceptions about music licensing and royalties, outlining the difference between the standard “pro rata” royalty model (in which all subscription revenue is pooled and distributed to rightsholders based on the number of streams they receive) and the “user-centric” model (in which an individual user’s payment goes to the specific artists they listen to). While user-centric royalties are often touted as more ethical, McDonald uses Spotify data to argue that independent artists make more money on average through the pro rata model because their fans tend to listen to more music than mainstream listeners.

McDonald acknowledges that neither system radically redistributes royalties. As he puts it, “It would take a much more dramatic and intentional application of music-industry socialism to redirect enough of Ed Sheeran’s … money to musicians who still need to pay for apartment rent, or maybe better soundproofing, for it to be life-changing for them or Ed.” Despite this awareness, he often downplays streaming’s negative effects on artists’ livelihoods, which is disappointing given the extensive critiques of Spotify’s payout rates from organizations such the United Musicians and Allied Workers and artists ranging from Taylor Swift to Thom Yorke.

The final section of the book explores methods for improving the future of streaming, ranging from ethical listening practices to morally responsible algorithms. But the main suggestion is seemingly to subscribe to a streaming service, listen to it as much as you possibly can, and embrace the joys of music. McDonald’s enthusiasm for music is contagious, but there’s a tension between his utopian vision for streaming and the current reality of artist precarity.

While McDonald is unabashedly optimistic about streaming’s potential, he’s not uncritical. At various points throughout the book, he reflects on the troubling treatment of artists, the biases baked into algorithms, and the perils of power being centralised in streaming services. He admits that it would be “insanely naive to imagine that capitalist structures like labels and streaming companies (especially public ones), which hoard power by their inherent nature, will automatically align with the collective human interest in the future of music by pursuing their individual commercial interests.”

McDonald was one of the hundreds of Spotify employees affected by the company’s 2023 layoffs. Every Noise at Once is still operational, but McDonald no longer has access to the internal data required to keep it updated. The website remains a reminder of music streaming’s potential for discovering, falling in love with, and sharing new music across communities and cultures. But it’s also a reminder that access to this music and data is contingent on large, profit-seeking tech companies and record labels. The future of music streaming can be wonderful – but will it be?

<i>You Have Not Yet Heard Your Favourite Song by Glenn McDonald is published by Canbury Press</i>

The Quietus Digest

Sign up for our free Friday email newsletter.

Support The Quietus

Our journalism is funded by our readers. Become a subscriber today to help champion our writing, plus enjoy bonus essays, podcasts, playlists and music downloads.

Support & Subscribe Today