Weekly reading
Recently I’ve been thinking a lot about the question ‘what is Software Engineering in an age of LLMs’. So much so that I’ve started a Linkedin Group for those similar interested–so if that’s you, please come join.
So this week we return to a grab bag of interesting things I came across the last week or so. Something for everyone I hope!
Comparing local large language models for alt-text generation
I tested 12 LLMs — 10 running locally and 2 cloud-based — to assess their accuracy in generating alt-text for images.
Source: Comparing local large language models for alt-text generation
Trusting AI to describe my photos wasn’t easy. But after 9,000 images, I had to admit: it often did the job better than me, and at a fraction of the cost.
Source: Trusting AI with my images wasn’t easy
The use of LLMs to generate alt text is somewhat contentious, with concerns about the accuracy of the descriptions generated. But automating a process that it has proved challenging to get humans to do, for whatever reason, may ensure far more alt text than we get at present, particularly on social media.
At Conffab we use LLMs to provide text descriptions for images in slides in presentations for our accessible slides feature. This is something we did originally by hand, but when a conference might have many thousands of slides, and a reasonable percentage contain images of some sort, this is a very costly and time consuming exercise.
An LLMs based approach very, very significantly sped this up. We do check the output to ensure these are good descriptions, but we have found these are often better descriptions than those done by humans–especially when they are of complex diagrams and charts. Here Dries Buytaert, the founder of Drupal tested 10 LLMs for doing this task, and concluded in a follow up post
Trusting AI to describe my photos wasn’t easy. But after 9,000 images, I had to admit: it often did the job better than me, and at a fraction of the cost.
Why I Like Designing in the Browser
It can be surprising for new clients to see just how much of our design process happens in HTML, CSS and (light) JavaScript. While we do plenty of ideation exercises, sketching, wireframes, mockups and more, we like to get our hands dirty in the browser as soon as we can.
There are business and process benefits to this approach, which we’ve written about before. In this article, I hope to answer a much smaller question:
What do I, a designer of 20+ years with many static mockups to his name, personally enjoy about designing in-browser with web standards in 2025?
Source: Why I Like Designing in the Browser – Cloud Four
The earliest web didn’t need design tools–all we had were simply a few heading levels, paragraphs, lists, a few inline elements. No colors or fonts or even images. For years these things were added to HTML piecemeal, while designers discovered techniques (better described as ‘hacks’) for creating page layouts and designs wth these rudimentary tools.
These approaches could be complex and error prone–using tables with images to create space was a technique that remained for years. Around the same time, though it took years to mature and longer to get widely adopted, CSS introduced more sophisticated layouts with absolute positioning (we discovered the use of float for creating layouts some years later).
And alongside these new technologies and techniques came hybrid web design/development tools like HotPage (I think it was called? From Adobe?), Microsoft Front Page, and then DreamWeaver, the killer app for Web Design. These WYSIWYG tools ushered in a new era of web design.
In the decades since, we’ve oscillated between coding designs directly, and using design tools, either to prototype or to implement these designs. Douglas Engelbart, a giant of human computer interfaces called WYSIWYG, ‘what you see if all you get’–observing that when we use such tools we are constrained to only what such tools enable.
When it comes to the web, many designers have continued the practice of designing in the browser. Perhaps first widely articulated by designer extraordinaire Andy Clarke, here Tyler Sticka talks about the benefits of designing in the browser in the age of Figma. A good companion read to the recent What do developers want in a design handoff? (see below).
Using & Styling The Details Element
You can find the element all over the web these days. We were excited about it when it first dropped and toyed with using it as a menu back in 2019 (but probably don’t) among many other experiments. John Rhea made an entire game that combines with the Popover API!Now that we’re 5+ years into , we know more about it than ever before. I thought I’d round that information up so it’s in one place I can reference in the future without having to search the site — and other sites — to find it.
Source: Using & Styling The Details Element | CSS-Tricks
HTML is full of useful elements, some of longstanding, that are much less used than they ought to be. Here Geoff Graham goes into detail about <details>, their use and styling.
Your first WebGPU app
Before WebGPU, there was WebGL, which offered a subset of the features of WebGPU. It enabled a new class of rich web content, and developers have built amazing things with it. However, it was based on the OpenGL ES 2.0 API, released in 2007, which was based on the even older OpenGL API. GPUs have evolved significantly in that time, and the native APIs that are used to interface with them have evolved as well with Direct3D 12, Metal, and Vulkan.
WebGPU brings the advancements of these modern APIs to the web platform. It focuses on enabling GPU features in a cross-platform way, while presenting an API that feels natural on the web and is less verbose than some of the native APIs it’s built on top of.
Source: Your first WebGPU app
Build Conway’s ‘Game of Life’ using WebGPU and vanilla JavaScript and web technologies. Being a bit obsessed with the Game of Life, and interested in learning more about WebGPU, and not being terrible at JavaScript, I have to make time to do this!
The UX Researcher’s Guide to Getting Started with Accessibility Research
Many UX researchers want to conduct more meaningful accessibility testing with their products – and that means including people with disabilities in the process. While organizational commitment to introducing accessible user research methods may be high, progress can easily get bogged down by endless discussions and brainstorming sessions. Despite best intentions, over-analysis can block the path forward.
The truth is that achieving your inclusive product design goals doesn’t have to be a massive undertaking from day one. You can start small and build at a pace that makes sense for your organization. But there are some foundational steps to get right before you jump into recruiting and screening research participants. This article is a great place to start.
Keep reading to access practical advice on how to build on your existing research skills, scope your accessibility research smarter, and achieve the early wins that keep enthusiasm high and momentum strong.
Source: The UX Researcher’s Guide to Getting Started with Accessibility Research
The excellent folks at Fable have an in-depth guide to Getting Started with Accessibility Research.
Replace JavaScript animations with View Transitions
For those rooting for the web, 2025 has now exciting news! Both Chrome and Safari shipped a new and fairly straightforward way to add animations and transitions to your sites — say “Hello” to the View Transitions API.Let me show you how view transitions work by recreating this animation effect with a few freckles of modern web technology. A web technology that is probably the most significant web platform update in years.
Source: Replace JavaScript animations with View Transitions
An excellent introduction to View Transitions, that goes beyond the basic. I cannot get enough of View Transitions–so sorry not sorry, you’ll be reading a lot more about them here.
Implementing a Product Model in a non-product enterprise
Product-centric companies like Facebook, Instagram, and Canva aren’t merely companies that happen to have a product; the product is virtually the whole company. There is no distinction between the company, its brand, and the SaaS product they offer to customers.
However, this model doesn’t translate neatly into the enterprise context. Digital products, particularly those delivered through a SaaS model, are essentially services. Furthermore, the approach needed for success varies greatly between contexts, which are many and varied in an enterprise setting vs the simplicity of a startup.
Enterprises like banks, media companies, or healthcare providers cannot simply adopt the product models that work for tech companies. Their challenges are more complex, requiring tailored strategies that provide additional methods and nuanced application in the spaces above and in-between these ‘products’.
Source: Implementing a Product Model in a non-product enterprise
Some thoughts from Mike Briggs on adopting a product mindset in organisations that aren’t product companies.
Digital Sovereignty
Digital sovereignty is a real problem that matters to real people and real businesses in the real world, it can be explained in concrete terms, and we can devise pragmatic strategies to improve it. To do this, I will go through the following steps:
First, I will briefly define sovereignty to make sure that we are on the same page, and explain how digital sovereignty is built from digital infrastructure.
Second, I will explain why it comes into conflict with democratic sovereignty, and how both the politics of tech companies (that have been authoritarian for a long time) and the current geopolitics make the situation particularly challenging.
Finally, I will offer a series of high-level strategies that can be deployed to improve digital sovereignty.
Source: Digital Sovereignty
Robin Berjon argues that digital sovereignty is important because control over digital infrastructure translates to power—both politically and economically. He sees digital sovereignty as a means of reinforcing democratic values, protecting autonomy, and ensuring that digital infrastructure serves the public good rather than the interests of a few dominant corporations or geopolitical powers.
Here he highlights the conflict between corporate control by tech giants and democratic sovereignty, noting the authoritarian tendencies of these corporations and the geopolitical challenges they present. Incredibly timely.
Everything you need to know about Invoker Commands | London Web Standards
command and commandfor attributes are a brand new web platform feature coming in 2025. Here’s everything you need to know about them, plus some things you really don’t.
Source: Everything you need to know about Invoker Commands
Invokers are something a lot of people have wanted in the Web platform for a long time–a way of declaratively triggering built-in behaviors without JavaScript.
For example, the <button popovertarget=”myPopover”> attribute opens a <div popover id=”myPopover”>, enabling native pop-ups without extra scripting.
Here Keith Cirkle goes into them in detail.
And keep an eye out for a fantastic talk on popovers coming up soon on Conffab
What do developers want in a design handoff? We asked them
When designers finish designing an app and deliver the design specs to the development team to build, mutual frustration frequently ensues. The design team is worried the development team won’t realize their vision and the development team is worried about all the back-and-forth that will be necessary to make sure they’re actually building what needs to be built.
Source: What do developers want in a design handoff? We asked them. – Blog – zeroheight
The folks at ZeroHeight asked developers what they wanted from their design teams in terms of design handoffs.
My LLM codegen workflow atm
I have been building so many small products using LLMs. It has been fun, and useful. However, there are pitfalls that can waste so much time. A while back a friend asked me how I was using LLMs to write software. I thought “oh boy. how much time do you have!” and thus this post.
(p.s. if you are an AI hater – scroll to the end)
I talk to many dev friends about this, and we all have a similar approach with various tweaks in either direction.
Here is my workflow. It is built upon my own work, conversations with friends (thx Nikete, Kanno, Obra, Kris, and Erik), and following many best practices shared on the various terrible internet bad places.
This is working well NOW, it will probably not work in 2 weeks, or it will work twice as well. ¯\_(ツ)_/¯
Source: My LLM codegen workflow atm
It is such early days of working with LLMs for code generation that we’re still just beginning to work out workflows–which is why it is fascinating to see people share theirs as Harper Reed does here. I’ve tried some his suggestions and suggest you give them a try too.
Thoughts On A Month With Devin
As a team at Answer.AI that routinely experiments with AI developer tools, something about Devin felt different. If it could deliver even half of what it promised, it could transform how we work. But while Twitter was full of enthusiasm, we couldn’t find many detailed accounts of people actually using it. So we decided to put it through its paces, testing it against a wide range of real-world tasks. This is our story – a thorough, real-world attempt to work with one of the most hyped AI products of 2024.
Source: Thoughts On A Month With Devin – Answer.AI
One of the areas where LLMs seem to do particularly well is in software engineering. In my experience it can make me far more productive, and as Simon Willison has observed, make developers more adventurous.
I’ve been starting many more projects I know could do, but didn’t know how long they might take, which otherwise I would not even have commenced. I’ve built numerous internal tools for streamlining the production of Conffab content using these technologies and a vector search engine for all of Taylor Swift’s lyrics using Python, a language I had only passing familiarity with. But these are quite constrained, and relatively low stakes problems–the ambition of the likes of of Replit and Devin are far greater-to replace the writing of code almost entirely, not simply to augment existing experienced developers.
The folks at Fast.AI put Devin to the test here–with less than stellar results, at least for now.
Toe Dipping Into View Transitions
I’ll be honest and say that the View Transition API intimidates me more than a smidge. There are plenty of tutorials with the most impressive demos showing how we can animate the transition between two pages, and they usually start with the simplest of all examples.
Source: Toe Dipping Into View Transitions | CSS-Tricks
I said there as more View Transitions content coming! I do agree with Geoff Graham it can feel like the step from simple to anything more is not trivial when it comes to this new web technology. Here he looks at taking the next step beyond the basic.
Introducing Web Applets, Rupert Manfredi
Natural language UIs are touted by many to be the future of user interfaces. But right now, they still look a lot like the past: a text-based prompt, with text-based answers. (If you squint, it could be MS-DOS.) While there’s growing support to bring richer, more graphical interfaces to these systems, they mostly rely on proprietary, centralized integrations. Will the future of computing look like a handful of privately controlled platforms, or an open thriving ecosystem like the web?
Web Applets are a new open standard for allowing language models to use rich, graphical software, built upon the web.
They are small, local-first, interoperable bits of software that can be used and read by both a human and a machine. This talk introduces the problem that Web Applets aims to solve, shows how they work, and reveals a few examples in action. Attendees will be encouraged to integrate Web Applets into their own applications and collaborate with us in shaping the technology’s future, to help keep software open!
Source: Youtube
Rupert Manfredi thinks a lot about the future of the human experience of the Web and technology. He’s worked at Mozilla, Google and elsewhere on the ideas and is now developing an open Web based platform for AI agents to interoperate. Here he talks about Unternet, that platform.
Getting Started | Web Components Toolkit
The Web Components Toolkit is a collection of tools to help you build web components that teams love to use.
This guide will help you get started with the tools used in the Toolkit.
Source: Getting Started | Web Components Toolkit
If you’re looking to dip your toe in the Web Components water, then these tools could be very helpful.
Why I’m Woke
This is almost certainly the first in a series of Not A Tech Bro posts that could share the same title. Today, I want to talk about diversity, equity and inclusion (DEI) programs, and why I worked so hard to establish them at Cloudera. MAGA hates DEI, and stole the term “woke” to insult folks who embrace it. I’m stealing it back to explain why they’re wrong.
I am, proudly, pretty damn woke.
If you’re the CEO of a tech company doing innovative work, you are in a constant struggle with every other tech company CEO to recruit and retain top-tier talent. Especially in its early days, the major threat to a start-up isn’t competition. It’s mediocrity.Received wisdom among the MAGA crowd is that DEI programs drive mediocrity. Of course if you elevated skin tone or gender above everything else in hiring, that’d be true. But of course that’s a dumb mistake and easy to avoid. It for sure isn’t how I used DEI at my company.There are three things that concentrating on DEI can do for you.
Source: Why I’m Woke
Right now a lot of incredibly powerful and wealthy folks in technology are distancing themselves from “DEI” initiatives (those terrible terrible things, diversity, equity and inclusion). But not everyone–Mike Olson, founder of Cloudera here articulates strongly the value of these initiatives. Initiatives we strongly support and actively embrace.
OK, that’s a fair bit to keep you occupied until next week!
Great reading, every weekend.
We round up the best writing about the web and send it your way each Friday.
