I object to this opinion.
Not because the models are flawless, but because the criticism often misidentifies the problem's source.
Modern large language models represent a compressed statistical summary of an enormous portion of what has been publicly written.
Not because the models are flawless, but because the criticism often misidentifies the problem's source.
Modern large language models represent a compressed statistical summary of an enormous portion of what has been publicly written.
They do not store documents. They encode patterns, structures, distinctions, and relationships across domains at a scale no individual human can internalize.
From the model’s perspective, there are only two things to work with.
One is the representation space shaped during training, which might be called learned meaning proximity in representation space.
The other is the context provided at inference time, the text we place in front of the model and ask it to continue.
The learned representation is undeniably powerful.
I have seen this directly when the model explained subtle distinctions in eighteenth-century Japanese poetry, including how classical grammar and social context are distorted in many modern translations.
That kind of response is not generic. It reflects the deep structure of the training data.
Where things often fail is the other half of the equation.
The context we provide as users is frequently vague, underspecified, or internally inconsistent.
We ask broad questions.
We mix goals.
We omit constraints.
We rely on unstated assumptions.
Then we blame the model for producing broad answers.
From the model’s perspective, a weak prompt defines a broad, blurry region of the meaning space.
When attention has nothing precise to lock onto, the output naturally gravitates toward statistically common, default responses, intellectually equivalent to the bad prompt that produced them.
This is not a flaw unique to LLMs. The same dynamic exists in human conversation.
Precise questions invite precise answers. Poorly-formed questions invite platitudes.
Seen this way, prompting is not a cosmetic skill.
It is the act of placing the model in the right neighborhood of meaning space.
Good context allows latent structure to surface.
In practice, when approaching a complex or scientific question, this means preparing a laser-focused conversational context.
That context may include notes that frame the question, references to recent innovations such as papers, articles, or news, and clearly stated constraints.
Very often, for fifteen or more minutes, I ask GPT to pose clarifying questions one at a time.
Only when I am confident it understands the problem precisely do I proceed and ask GPT to write a prompt for itself that summarizes the problem's mental framework as it understands it.
Only then do I have a real chance of receiving not a generic answer, but a concept or idea that has never been explicitly written down, yet emerges naturally as an intersection of existing concepts, algorithms, or solutions.
In other words, something genuinely novel.
It is well known that after an important speech or a major publication, multiple scientists often independently arrive at the same invention nearly simultaneously.
This is what it means to share the essential context.
From the model’s perspective, there are only two things to work with.
One is the representation space shaped during training, which might be called learned meaning proximity in representation space.
The other is the context provided at inference time, the text we place in front of the model and ask it to continue.
The learned representation is undeniably powerful.
I have seen this directly when the model explained subtle distinctions in eighteenth-century Japanese poetry, including how classical grammar and social context are distorted in many modern translations.
That kind of response is not generic. It reflects the deep structure of the training data.
Where things often fail is the other half of the equation.
The context we provide as users is frequently vague, underspecified, or internally inconsistent.
We ask broad questions.
We mix goals.
We omit constraints.
We rely on unstated assumptions.
Then we blame the model for producing broad answers.
From the model’s perspective, a weak prompt defines a broad, blurry region of the meaning space.
When attention has nothing precise to lock onto, the output naturally gravitates toward statistically common, default responses, intellectually equivalent to the bad prompt that produced them.
This is not a flaw unique to LLMs. The same dynamic exists in human conversation.
Precise questions invite precise answers. Poorly-formed questions invite platitudes.
Seen this way, prompting is not a cosmetic skill.
It is the act of placing the model in the right neighborhood of meaning space.
Good context allows latent structure to surface.
In practice, when approaching a complex or scientific question, this means preparing a laser-focused conversational context.
That context may include notes that frame the question, references to recent innovations such as papers, articles, or news, and clearly stated constraints.
Very often, for fifteen or more minutes, I ask GPT to pose clarifying questions one at a time.
Only when I am confident it understands the problem precisely do I proceed and ask GPT to write a prompt for itself that summarizes the problem's mental framework as it understands it.
Only then do I have a real chance of receiving not a generic answer, but a concept or idea that has never been explicitly written down, yet emerges naturally as an intersection of existing concepts, algorithms, or solutions.
In other words, something genuinely novel.
It is well known that after an important speech or a major publication, multiple scientists often independently arrive at the same invention nearly simultaneously.
This is what it means to share the essential context.
No comments:
Post a Comment