Web Directions Year round learning for product, design and engineering professionals

Who Needs a LoRA?

There's an assumption embedded in how people work with image generation models: if you want the model to do something new or specific, you fine-tune it. You get the pretrained model, you gather a dataset of examples of what you want it to do, you run a training process, and you get a specialized version of the model. This is called a LoRA (Low-Rank Adaptation) and it's the standard move in the playbook.

Except what if you don't actually need it? What if, with the right framing, a pretrained model can already do what you want? What if the barrier isn't the model's capability but how you're asking it to work?

The Avatar Editing Problem

Charli Posner faced exactly this constraint when building a production system to edit hand-drawn character avatars in an educational platform. The system needed to do something very specific: take an illustration and edit it to match emotional expression in dialogue. A character speaks with anger, so the avatar's expression needs to become angry. A character speaks with joy, so the avatar's expression needs to shift to match.

This is the kind of problem that seems like a job for fine-tuning. You'd gather examples of avatars with different expressions, train a LoRA, and deploy it. But there was a hard constraint: every edit had to look like the original artist drew it. Not like a model interpreted what the artist might have drawn. Not a reasonable approximation. Like the original artist's style, linework, and approach to figure drawing.

Fine-tuning is actually the wrong tool for this problem. Fine-tuning creates a model that's good at generating images in a certain category. But it doesn't preserve style fidelity across edits. More importantly, it requires a dataset of examples, and there wasn't a dataset of examples because the original artwork was bespoke.

Three Principles Emerged

The insight that made this solvable was to stop thinking about fine-tuning and start thinking about prompting. The image model already knew how to edit expressions. It already knew how to preserve style. The question was how to direct it effectively.

Three principles emerged from experimentation. The first is subtle but crucial: prompt entropy predicts drift. If your prompt is vague, the model's output will be unpredictable. It might give you a great expression edit or it might drift into something that doesn't look like the original artwork anymore. If your prompt is precise, the model's behavior is more reliable. This seems obvious in retrospect, but it's counterintuitive when you're working with language models—usually, being too specific over-constrains the model. With image models, being specific is the path to reliability.

The second principle is about what to describe: movements, not features. Instead of saying "make the eyebrows angry," which requires the model to understand your concept of what angry eyebrows look like, you describe the movement. "Bring the inner eyebrows closer together and angle them downward." Now you're describing geometry, not emotion, and the model's output is more consistent with the original artwork's geometry.

The third principle is architectural: parallel edits from pristine original beat sequential chains. If you start with the original artwork and generate the edited version directly, you get better results than if you edit the artwork once, then edit the result again. Sequential edits compound errors. Each transformation introduces a small drift, and after three or four transformations, you've drifted significantly from the original style. Parallel edits from a single source are more stable.

The Power of Better Prompting

What's interesting about these principles is that they're not about the model's architecture or capability. They're about how you work with the model. You don't need a specialized version of the model. You need to understand the model well enough to direct it effectively.

This extends beyond avatar editing. The implication is broader: as image models have become more capable, the frontier of value creation has shifted. It's not about having a model trained on your specific data. It's about knowing how to prompt a general model so precisely that it produces exactly what you want. That requires understanding what the model responds to, what directions it interprets reliably, and what ambiguities lead to degradation.

The LoRA approach made sense when image models were less capable and less controllable. You needed to push the model toward your specific use case because the base model wasn't reliable enough to do it. Modern image models are reliable enough that precise prompting can often achieve the same result with none of the training overhead.

This has implications for how people should be thinking about model customization more broadly. Before you fine-tune, experiment with prompting. Understand the space of what the model can do with the right direction. Sometimes you'll discover that fine-tuning is still necessary. But often, you'll find that the base model was capable all along—you just weren't directing it correctly.

Building Production Systems with Precision

Charli Posner works at Stile Education as an AI engineer, building production systems that span the full spectrum of the AI stack—from handwritten scanning pipelines to image editing workflows. This work requires deep understanding of what each tool can do and how to get consistent, reliable results.

Her work is grounded in computer vision, deep learning, and practical experience shipping systems that have to work repeatedly, on diverse inputs, without breaking. She's researched human pose estimation, worked on large-scale systems, and learned what it takes to make AI systems that are trustworthy enough for production use.

The talk will explore these principles in depth: how prompt entropy relates to model reliability, the difference between describing features versus movements, why parallel edits beat sequential chains, and the broader implications for thinking about model customization. It's a case study in getting the most out of models you can't fine-tune and building systems that preserve style and quality across multiple transformations.

Charli will be sharing these insights at AI Engineer Melbourne 2026, June 3-4.