Introduction: A Language Model of My Own
We are surrounded by large language models: systems trained on the vastness of the internet. Models like GPT, Claude, and LLaMA can write essays, answer science questions, explain math, generate stories, and even simulate personalities.But as I’ve written in my blog, I’m not chasing scale. I like to experiment with small language models (SLM)—or, in this case, tiny language models (TLM). Partly because I’m burning midnight oil alone and not a green pile of venture capital money. Training even a mediocre LLM can cost tens of millions of dollars. More importantly, I wanted something that runs locally, privately, and fast. A model that lives on my laptop, not in a server farm.
Yes, I know how to fine-tune Mistral 7B and use RAG and CAG. I’ll explain later why I trained a model from scratch instead.
I also wanted something more playful: an AI Zen Garden where I cultivate a Collegium of AI personalities. Each is trained to reflect a slice of my inner world with their philosophies, moods, and voices.
TLM is the first of these. She does not know all the facts ever written about the world. Instead, she is trained to understand how I see the world—to help me think, reflect, and write in my own voice. She speaks from my blog posts and personal notes collected over the decades. She is not artificial in the corporate sense. She is authentic, with her Japanese spunk and thoughtful presence.
This book tells the whole story: how I came up with the idea, how I built it, why a small model can still be wise, and how you might make one, too.