Tags: ai

From human hands. · Mass-Driver

This is beautifully written.

At The Web You Want event in Amsterdam last month, Léonie gave an absolutely fantastic talk. You can now read the whole thing: Accessibility is resistance!

She makes the very good point that generative tools are a necessary accessibility recourse when humans have failed to do their job properly.

Images without alt text? Somebody messed up. But this is damage that can be routed around with large language models.

So if you—like me—don’t like the idea of people using extractive generative tools made by the worst kind of people, pay heed:

If you want to resist AI, make accessibility part of everything you do, every decision you make, every product you design and build. Remove the need for people to use AI to compensate, and remember, accessibility is resistance.

Also last month, I wrote about enhancing images from a carousel to masonry using CSS Grid Lanes. Here’s the context:

Over on The Session, I added a little enhancement to the events and sessions listings recently. I make a call to the Google Places API to see if I can find a match for the venue, and if I do, pull in some photos.

Sidenote: right now there’s a major issue with this. None of the photos come with text descriptions. This is something I need to fix, and I’ve got some ideas on how to do that.

I was one of those people with a website that doesn’t have alt text for images. I was one of those people not doing their job.

The first fix I put in place was very much in keeping with the ethos of The Session. I added a form so that any member of the site can edit the alt text of the images in the carousel/masonry layout.

The Session is quite WikiPedia-like. Anyone can edit the track listings in the discography. Anyone can edit the details of any tune. Anyone can edit the details of any session or event. So it made sense to add one more thing that anyone can edit.

With that functionality in place, I put out the call:

If everyone were to take just one or two sessions or events and add alt text to the photos, it would really, really help make the site more accessible.

The community went to work and did a pretty good job.

But there are a lot images. I’d like to have a least some kind of alt text for all of them, even if it’s just a placeholder until someone gets around to adding something better.

I was going to have to use generative “AI”, wasn’t I?

Holding my nose, I investigated some alt-text-as-a-service APIs out there. I signed up for some trials and kicked the tyres. They were rubbish. I suspect most of them were vibe-coded.

The more I thought about it, the more I realised that I probably didn’t need the latest and greatest energy-guzzling large langauge models. Some good old-fashioned machine learning could do the trick here. I’m aiming to do a relatively straightforward transformation—image to text.

Check out the COCO project: Common Objects in Context. It’s a few years old now, but it’s basically a dataset that’s been trained on images.

Ah, but where did the training data come from?

All images in COCO were sourced from Flickr, a photo-sharing platform where users upload images under various Creative Commons licenses.

Works for me!

I found a model on Hugging Face that was created by Microsoft from the COCO data. Confusingly the model is called GIT. It’s nothing to do with version control. It stands for GenerativeImage2Text.

I downloaded the model and set to work creating a script to trawl through all those images and generate alt text if the image doesn’t already have one.

I should mention: the way that I store the alt text is not in a database, but in the file system. Every image has its own folder. Inside that folder there can also be a file called alt.txt.

So the script needed to loop through lots and lots of folders, generate alt text where needed, and save it into a text file in the same folder.

Python is the go-to language for working with models like GIT. But Python is not my strong suit, to put it mildly. And this was going to be a throwaway script that I was only going to use once. So I held my nose again and turned on Copilot in VSCode (I usually keep it switched off).

With the right instructions I was able to get a script that—crucially—I could read and understand. Then I switched off Copilot and had a shower to try to wash the shame away.

Long story short, the script worked.

The results are mostly fine. The alt text is functional. Sometimes it’s wrong, but not disastrously so. A human-generated image description will always be preferable, so any member of The Session can over-ride the placeholder alt text.

It took a few hours to loop through all those folders and generate all those text files. My laptop got hot. But doing it locally sure beats paying for tokens from the baddies.

Like Scott says:

While the utility is there, the current landscape of these frontier corporate models comes with massive ethical and environmental baggage. This is not a long term solution!

That’s exactly why my hope is that ethically trained open-weight models will eventually catch up and that in the next couple of years, we’ll have something with this level of capability that can run locally. That feels like a much better place to be with this technology.

The AI Resist List

Nothing about the current trajectory of AI development is inevitable. It was shaped by the thousands of subjective decisions of a tiny elite, and continues its march based on the active participation and tacit consent of people globally.

I give Apple a hard time. That’s mostly due to how they treat the web on their own mobile devices.

Though iOS ostensibly supports the ability for websites to be added to the homescreen, they make it so difficult for users to do, the functionality is practically worthless. It’s dispiriting to see the web so hamstrung by that decision.

The Webkit team has come in for other criticism too. For a while there, Safari was lagging so far behind in features that people were calling it the new Internet Explorer. Ouch!

But credit where credit is due. The upcoming version 27 of Safari is looking very good.

That’s not because it’s at the cutting edge of the latest web standards. Quite the opposite. Most of the changes listed for this release are bug fixes. That’s what I want to acknowledge and applaud.

Far too often a browser will rush out an implementation of an exciting new web standard that gets plenty of attention. But that initial implementation is rarely 100% correct. Then the next release rolls around and the focus has moved on to a different new web standard. The result is an ever-growing backlog of almost-but-not-quite-supported features.

Clearly the focus for Safari 27 was on that backlog. I bet that wasn’t an easy decision. Like I said, the kudos and recognition tends to go to the browser that ships new stuff, not the browser that goes back to fix long-standing issues.

Don’t get me wrong, there’s some exciting new stuff in Safari 27 too, like styleable select, but it’s great to see the focus on maintenance and repair:

If you look through the lists of features and fixes in Safari 27, you’ll notice that, although there are 58 brand-new features and 525 fixes — the largest pile of fixes in any Safari release in recent memory — most of what is released is not about new things.

Most of this work has been about existing features behaving more correctly, handling more edge cases, and fitting together with other features the way you’d expect.

This in sharp contrast to the most recent release of Chrome that shipped support for the prompt API despite opposition from other browsers and no positive signals from developers. I hope some Googler got a nice promotion for shoving a proprietary technology into a web browser, but they should be aware of the damage they’ve done.

At this year’s CSS Day, the represenatatives from Google Chrome were once again there to talk to developers and ask what we wanted them to prioritise. Those requests rang very hollow. Why should we waste our time and energy telling a browser team what we need if they’re just going to ship whatever crap they want?

The truth is that the folks from Google who were canvassing opinions from the attendees at CSS Day are not the same people who torpedoed the browser with unwanted proprietary tech. This team has spent years doing excellent outreach, documenting web standards, and meeting with developers. They built up an impressive amount of trust, respect, and goodwill.

That stock has now plummeted.

So well done to the Webkit team for Safari 27. And shame on the Chrome team for Chrome 148.

AI and the Rise of Mediocrity

Simply put: AI thrives when our need for originality is low and our demand for mediocrity is high.

AI will fill the world with grindingly average texts, passable but derivative illustration and video, and unoriginal but functional new product designs.

What is being mechanized by AI is our tastes—our ability to discern quality (or originality) at all.

The value is in the difficulty - Annotations

We’ve seen this arc before, and music is the richest analogy.

Like Bruce Sterling always says:

Whatever happens to musicians happens to everybody.

I knew my writing students were using AI. Their confessions led to a powerful teaching moment | AI (artificial intelligence) | The Guardian

AI writing reminds me of Tennyson’s description of the beautiful Maud in the titular poem:

Faultily faultless, icily regular, splendidly null
Dead perfection; no more

It’s Not AI. It’s FOMOnetization.

FOMO is a feeling. But it’s also a business model—and increasingly, one of the more successful ones. Fear, in general, makes people much easier to separate from their money. It’s perfectly suited to this moment of ubiquitous grift, where everything feels like a lottery ticket or a multi-level marketing scheme.

It’s even more perfectly suited for “the age of AI,” which squeezes economic FOMO from both sides. AI could make you wildly rich (the first person to start a billion-dollar company with zero employees!) or leave you hopelessly destitute (part of the looming “permanent underclass”). Which one do you want to be? Smash that like button, sign up for my online course, and use my new AI-powered business platform!

In the latest issue of Matthias’s excellent Own Your Web series, he describes the recent betrayal by Google:

The search engine no longer says “here, go read what this person wrote.” It now says “here, I’ve already read it for you.” The contract is broken.

He’s absolutely right.

But…

Have you ever clicked on a result from a search engine? Unless you’re lucky enough to land on a nice personal website, you’re more than likely to be confronted with pop-ups to allow tracking, or a desparate plea to subscribe to a newsletter, or just rubbish ads all accompanied by a slow page loading somewhere in the mix.

Don’t get me wrong. I’m not saying that what Google is doing is okay. But let’s not pretend that everything indexed by Google is just fine and dandy for people to visit.

And of course the main reason why websites are so terrible is because they’ve tied their business model to heaps of behavioral advertising driven by invasive tracking courtesy of …Google.

This reminds me of AMP. Remember Google AMP? It was a terrible solution to a real problem. Web pages were (and still are) bloated and slow. The correct solution would be to encourage people to fix that, but instead Google mandated a proprietary format for your content that had to be hosted on their servers.

AMP was a disaster, both in practical terms and in the reputational damage it did to Google’s developer relations.

Now they’re doing it again, powerwashing away any goodwill they ever had with site owners. Now Google doesn’t even send search engine traffic to the websites that host the ads that Google encouraged people to put on every page.

It’s almost as if Google is a company so large and with so many competing interests that it now suffers from an incurable split personality disorder.

Personally I think they’re missing a trick. They should be using “AI” summaries as a stick.

If your site is slow, or filled with user-hostile annoyances then it should be cockblocked by a hallucinated summary. But a nice fast respectful website? Send the traffic their way! Everyone wins—users, site owners, Google, the World Wide Web.

Could you imagine how quickly this would revolutionise the world of search engine optimisation? They’ve always told us that we should make websites for humans in order to get good Google juice. This would be a way of making it come true, without any of the over-engineered woefulness of AMP.

It’ll never happen of course. But I can dream.

Expansion artifacts || Matt Ström-Awn, designer-leader

Compression made the information age possible by stripping things down to fit the pipes. Expansion made the AI age possible by blowing data back up again. Both operations leave marks; we’ve learned to spot compression artifacts, but we’ve only just begun to reckon with expansion artifacts. Until we do, there’s a lot of risk to manage.

People talk about the effectiveness (or lack thereof) of large language models as though all tasks are comparable. But it strikes me that there are three broad categories of work that large language models are applied to:

1. Compression.
2. Transformation.
3. Expansion.

Compression is when you feed a large language model something big that you want to make small. Summarise this book. Give me the gist of this meeting. Large language models are generally pretty good at this, which makes sense given that they themselves are kind of like compressed artifacts.

Transformation is when large language models convert from one format into another. Turn this audio into text. Turn this jumble of data into structured JSON. A large language model can handle these tasks pretty well. There’ll probably be a few errors so make sure that’s not a deal-breaker.

Expansion is when you give a large language model a prompt to generate something from scratch. An image. A presentation. An email. A poem. This is where slop lives. The output inevitably betrays its origins, glistening with a sheen of mediocrity.

Laurie spotted this three-way split a while back:

Is what you’re doing taking a large amount of text and asking the LLM to convert it into a smaller amount of text? Then it’s probably going to be great at it. If you’re asking it to convert into a roughly equal amount of text it will be so-so. If you’re asking it to create more text than you gave it, forget about it.

I hope that when the bubble finally bursts, we’ll see the surviving large language models put to work on the first two categories. The boring stuff. The work that’s tedious for humans.

But tedious is as tedious does. Something I consider drudgery might be the very thing that gives you life. Like Giles says:

I have a feeling that everyone likes using AI tools to try doing someone else’s profession. They’re much less keen when someone else uses it for their profession.

The big exception seems to be programming. Apparently there are plenty of coders who never before expressed an interest in being managers who are now happily hanging up their coding spurs in favour being the overseer of non-human workers.

It’s a reasonable outlook. It could even be considered a user-centred approach. Users don’t care about the elegance of your code; they care about accomplishing their tasks.

Programming is something of an exception to the efficacy of large language models in general. Instead of relying on the subjectivity of painting, poetry, or prose, programming can be objectively tested. Throw enough money at the worst people in the world and they’ll give you tokens you can use to get the machines to test their own output. So you can get a large language model to create something reasonably good from scratch as long as that something is code.

If you had asked me about the threat model of large language models two years ago, I probably would’ve been worried for artists, writers, and musicians. I thought that software had enough inherent complexity to be relatively safe.

Now my opinion has completely reversed. Software is almost certainly the killer app for large language models.

I think the artists, writers, and musicians will be okay, or at least as okay as they ever were. It turns out that humans like things made by other humans.

And y’know what? If I had to choose which endeavour I’d rather see automated away—programming or art—it’s no competition.

Don’t get me wrong—it would be nice if everyone got paid for doing what they enjoy. It’s just that I’m okay with software engineers not being at the front of that line.

I remember when I first started getting paid money to make websites. “Really?” I thought, “Someone is willing to pay me to do something I’d do anyway?” I kept waiting for the jig to be up. Instead I saw my profession grow and expand.

Perhaps there’s a long-overdue compression happening.

Or maybe it’s more like a transformation.

Design and Engineering, As One · Matthias Ott

A thoughtful piece by Matthias that’s a must-read for both designers and developers.

No-stack web development – David Bushell – Web Dev (UK)

A stack is also technical debt, non-transferable knowledge, accelerated obsolescence, and vendor lock-in. That means fragility and overall unnecessary complication. Popular stacks inevitably turn into cargo cults that build in spite of the web, not for it.

The web platform does not require build toolchains. Always default to, and regress to, the fundamentals of CSS, HTML, and JavaScript. Those core standards are the web stack.

The AI Great Leap Forward

In 1958, Mao ordered every village in China to produce steel. Farmers melted down their cooking pots in backyard furnaces and reported spectacular numbers. The steel was useless. The crops rotted. Thirty million people starved.

In 2026, every other company is having top down mandate on AI transformation.

Same energy.

AI Might Be Our Best Shot At Taking Back The Open Web | Techdirt

Not sure I buy the argument here, though I do very much look forward to local language models getting better so we can ditch the predatory peddlars of today’s slop. But this trip down memory lane to the early web of the 1990s could’ve been describing my own experience:

But the thing I do remember was the first time I came across Derek Powazek’s Fray online magazine. It was the first time I had seen a website look beautiful. This was without CSS and without Javascript. I still remember quite clearly an “issue” of Fray that used frames to create some kind of “doors” you could slide open to reveal an article inside.

Fray was what made me want to make websites:

I distinctly remember sites like prehensile tales, 0sil8 and the inimitable Fray triggering something in my brain that made me realise what it was I wanted to do with my life.

I used AI. It worked. I hated it.: Taggart Tech

There’s a fundamental problem with these tools beyond the capacity of any deployment strategy to solve: the tool requires expertise to validate, but its use diminishes expertise and stunts its growth. How does one become an expert? There are no shortcuts; there is only continuous hard work and dedication. I was once told of writing, great writers learn how to break the rules in new and ingenious ways by first learning the rules.

But how is a new developer meant to learn the rules if their day-to-day work is nothing but the babysitting of models? How will they gain the hard-won experience that allows a human in the loop to be a useful safeguard?

These models alter cognition in ways deleterious to human prosperity. In other words, for as much output as they provide, they take something important from us.

The End : Focal Curve

I can’t remember the last time a blog post resonated with me this much.

Craig’s criteria on his job search:

• One: fuck offices
• Two: fuck AI
• Three: fuck React

And his conclusion:

Fuck work