When I Fed Poems to an LLM, I Realized I Was Measuring Temperature with a Screwdriver

I spend my days building AI applications that turn chaos into structure. Give me a messy dataset, some business logic, and a powerful enough model, and I can extract signal from noise.

So when I wanted to understand what large language models really comprehend, I did what any engineer would do: I ran an experiment. I fed a book of poems into an LLM and asked it to analyze what they meant.

The results were technically perfect. The model identified the rhyme scheme, explained the metaphors, mapped the structural tension between opposing concepts. If there had been a decoding exam, it would have scored 100%.

But watching it work, I realized I had made a category error. I was using a screwdriver to measure temperature.

What the Model Couldn't See

Take a line like "Dark Pillar Growth / Shadow Lack." An LLM can tell you these words create a juxtaposition of presence and absence, growth and loss. It can reference similar patterns in its training data. It can generate a sophisticated structural analysis.

What it cannot do is feel the specific weight of growth that is shadowed by loss. It cannot sit with the quiet discomfort this phrase creates in a human nervous system. The meaning isn't in the words themselves. It's in the resonance they create in you.

The poems were not asking to be decoded. They were asking to be experienced.

The Problem That Actually Matters

This isn't just about poetry. It's about the products we built in today’s world of ubiquitous llm integration.

How many times have I approached a user problem exactly the same way I approached that poem? Looking for what can be extracted, measured, logged, and fed into a dashboard. Building systems optimized for quantifiable metrics: engagement rates, conversion funnels, time-on-page.

We become obsessed with decoding user behavior. And in the process, we go numb to user experience.

The most important truths about a product aren't found in A/B test results. They're found in the things that resist measurement: the subtle frustration of navigating a confusing interface. The unexpected delight when something works more intuitively than expected. The trust built by a single honest error message.

These aren't data points. They're human experiences. And our best tools are often blind to them.

What This Means for What We Build

The AI systems we're building will get exponentially better at pattern recognition and analysis. They'll dominate any task that can be broken down into a logical, data-driven process.

But the things that resist this reduction: intuition, meaning, the felt sense of trust; those remain profoundly human. And they're often what determine whether our products actually matter to people.

Our job isn't just to build more powerful decoders. It's to have the wisdom to recognize what lies beyond the data. To build systems that leave space for the messy, unquantifiable experiences that make technology worth using.

Not everything that matters can be decoded. Some things must be lived.

And that includes the products we're asking people to live with.

\