Combining two homemade GPTs
I built two GPTs at home. Both of them are bad. Combining them into one working product anyway is the biggest experience leap I've had since I started seriously studying this field. That sentence would have made no sense to me in April of 2025, when I first started mulling over whether I could use AI for music at all. It took me this long to figure it out — and the figuring, it turns out, was the whole value.
Where it started
I didn't begin with my own models. I began with other people's — source separation that strips a song into vocals, drums, bass, guitar, even the crowd noise, so I can mute the guitarist and be the guitarist. That work convinced me AI belonged in my music room. Then I watched Karpathy build a GPT from scratch in a talk, and I was listening to a Hinton lecture around the same time, and the obvious question landed: I've been applying other people's models — could I make my own do something?
I'm not trying to compete with anybody's frontier lab. My goal is different: I want to be genuinely expert at applying this technology — data to training to safeguards to shipped product — and there's a level of that you only reach by building the whole thing yourself, badly, at least once.
GPT number one: the songwriter
The first model writes lyrics with chords over the words, songbook-style. It started as a character-level toy and grew into a 126-million-parameter model trained on the GPU in my kitchen. The neural-network part was maybe five percent of the education. The rest was everything around it: scraping and scrubbing tens of thousands of chord charts (the internet's tabs are full of rating pleas and legal boilerplate a model will happily learn to write), designing a tokenizer that treats a chord as one atom, watching validation curves betray me, running a reinforcement-learning pass, and learning why you judge a model with blind A/B listening instead of the loss number — because the metrics will lie to your face.
It also taught me safeguards. A model trained on real songs will sometimes just quote one, so every output gets checked for long verbatim runs against the training data before it's allowed out. Nobody puts that part in the demo videos. It's half the job.
GPT number two: the riff intern
The second GPT is a tiny 14-million-parameter model trained on guitar tablature. Feed it the riff you just played and it proposes what comes next. The hard half wasn't the model — it was writing a parser for ASCII guitar tab, the least standardized format humans have ever loved. The model trains in about fourteen minutes and absolutely memorizes its small corpus; I can show you the chart. But prompt it with a lick and the continuations come back riff-shaped — right register, sensible phrasing, power chords where power chords belong. Not good yet. Starting to prove out.
The ears
There's a third piece that isn't a GPT at all: a browser tool that listens. Hum a melody or play a single-note line into the mic and it comes back as playable tab. That's the input side — my song ideas start as sound, not text — and it's how my own playing gets into the loop.
Combining them
This is the part that felt like the leap. Models in notebooks are homework; the education compounds when they have to work together, in a product. So I built a small studio app: sketch a song one section at a time — type it, generate lyrics and chords with the songwriter, play a riff in through the mic, ask the riff model to extend it — then drag finished sections into the song. It renders real sheet music at the end: tab for guitar, notation with proper key signatures for piano. One chorus can hold a riff I hummed, lines the songwriter drafted, and an extension the riff intern proposed.
I didn't build that product alone — most of the credit for turning three research toys into one working studio goes to Claude Fable, which pair-built the app, the serving plumbing, and more of the glue than I'd like to admit. That's its own lesson in applying AI: the models are teammates on the building side too, not just features in the product.
Where this leaves me
Honest scorecard: my models aren't that great. The songwriter still writes clunkers and the riff intern leans on its memory. With much more data and an architecture shift or two, I think this could genuinely make good music — the early signs are there. For now it's fun, and it wrote a song about cows this morning.
But step back from April 2025 to now: spatial audio, stem separation, and now generation — I feel good about where I've taken AI with music. Better than good, actually: ready. I'm starting my own company, and replaying this exact experience on its subject matter — the data discipline, the training scars, the safeguards, the shipping — is going to be very gratifying.
Two bad GPTs. One product. That's the education I was after.