Web Directions Year round learning for product, design and engineering professionals

Having Your Cake and Eating It: An Implementation Guide for Privacy with AI

There's a fundamental tension at the heart of modern AI deployment: the models that work best require the most data, and the data that makes models work comes with the steepest privacy costs. The larger the model, the more examples it needs to learn from. The more examples, the more likely it is that sensitive information gets baked into the weights. Use a frontier model and you're outsourcing your data to a vendor. Use a local model and you're limiting yourself to architectures and capabilities that can fit on your hardware.

Everyone wants privacy. But privacy and capability are in direct tension, and that tension has seemed almost unresolvable. Either you get a powerful model and sacrifice control of your data, or you keep your data private and accept reduced capability. The conventional wisdom says you have to pick.

But conventional wisdom is often just wisdom that hasn't been questioned carefully enough.

The Options That Exist

The landscape of privacy-preserving AI is actually larger and richer than it appears at first glance. There are more options than private models versus give-up-control-to-vendors. What's missing isn't technology. It's practical experience and understanding of the tradeoffs.

Private models are the obvious starting point—smaller models that run locally, that never transmit data, that keep everything inside your infrastructure. The tradeoff is capability. You get less performance for more privacy. But sometimes that tradeoff is favorable. A private model that's 90% as capable as a frontier model but keeps your data in-house might be exactly what you need.

Trusted execution environments go further. Instead of trusting that a service provider won't abuse your data, you trust the hardware itself. Your data runs on encrypted processors in someone else's data center, but the encryption is enforced at the chip level. Even the infrastructure operator can't see what's being computed. It adds latency and cost, but it extends what's possible if you need to use a larger model while maintaining cryptographic guarantees about privacy.

Differential privacy adds noise to training data in carefully calculated ways. The idea is that if you add enough noise, you can prevent the model from memorizing specific examples, but you can still learn general patterns. The gradient is noisy instead of deterministic, the results are approximate instead of exact, but the privacy guarantees are mathematical. A model trained with differential privacy can't be reverse-engineered to extract the names and details of individuals in the training set.

Secure multi-party computation is more exotic but more powerful. Multiple organizations each have sensitive data. Instead of pooling the data (and trusting a central operator), you can compute jointly. The computation happens across multiple machines, no single party ever sees the complete data, but the result is accurate. It's slower than centralized computation, but it enables collaboration between competitors or adversaries who need to trust the math more than they trust each other.

Federated learning distributes the training process itself. Instead of sending raw data to a central server, you send model updates. Each participant trains on their own machine, then shares only the gradient updates. The central server aggregates, produces a new model, and sends it back. This way, sensitive data never leaves the local system, but you're still building models collaboratively. It's slower than centralized training, but it works.

Homomorphic encryption is the most powerful and most exotic. You can run computations on encrypted data without ever decrypting it. The result comes back encrypted, you decrypt it, and you get the correct answer. From the server's perspective, it's computing on nonsense. The privacy is perfect and total. The cost is massive—homomorphic operations are orders of magnitude slower than normal operations.

What Actually Works in Production

The theoretical beauty of these approaches is one thing. Making them work in real systems, serving real users, on real business timelines, is another. Nick Lothian has spent four years running some of these approaches in production, learning which ones work, which ones don't, and what the actual cost-benefit tradeoffs are.

Private models work. They're straightforward, they integrate easily, they perform predictably. The constraint is capability—you're limited to what can run locally. For many applications, modern small models are sufficient. The privacy benefit is real and immediate.

Trusted execution environments work but require careful architecture. You're not just running code; you're managing the trust boundary between your code and the hardware. It's more complex than local execution, but less fragile than trusting a remote service operator.

Differential privacy works, but the tradeoff between privacy and utility is real and sometimes steep. Adding noise is easy. Adding the right amount of noise to maintain utility while providing meaningful privacy protection requires experimentation and expertise.

Federated learning works if you have multiple parties coordinating. It adds significant complexity to the training process, but it enables collaboration with privacy guarantees.

Homomorphic encryption is beautiful in theory. In practice, it's slow enough that it's primarily useful for specific, limited operations rather than end-to-end pipelines. But for certain high-privacy applications, the cost is worth it.

Building Your Privacy Stack

The framework that emerges from practical experience is this: privacy-preserving AI isn't a single technology. It's a architecture question. Which parts of your system need privacy most? Which parts benefit most from capability? Can you use a private model for some parts and a frontier model for others? Can you process sensitive data locally and send only summary statistics to the cloud?

The combinations matter. You might use federated learning to build a model collaboratively, then deploy it as a private model locally. You might use differential privacy to protect training, then add trusted execution environments to protect inference. You might use homomorphic encryption for specific cryptographic operations and normal computation for everything else.

Nick Lothian brings deep expertise in AI, ML, forecasting, NLP, and privacy-preserving technologies. He's won the National AI Centre Sprint (2024) and brings the perspective of someone who has thought carefully about how to preserve privacy without sacrificing capability. He's published in top venues (PNAS, ACM) and holds patents in NLP.

The session will cover practical experience with each of these approaches: what works, what doesn't, what the real costs are, and how to architect systems that achieve meaningful privacy while remaining practical and performant.

Nick will be sharing these implementation insights at AI Engineer Melbourne 2026, June 3-4.