An excellent interview!
From deleted government records to disappearing music, our digital culture can be erased overnight. The Internet Archive’s Mark Graham and Chris Freeland explain what it will take to save it.
At The Web You Want event in Amsterdam last month, Léonie gave an absolutely fantastic talk. You can now read the whole thing: Accessibility is resistance!
She makes the very good point that generative tools are a necessary accessibility recourse when humans have failed to do their job properly.
Images without alt text? Somebody messed up. But this is damage that can be routed around with large language models.
So if you—like me—don’t like the idea of people using extractive generative tools made by the worst kind of people, pay heed:
If you want to resist AI, make accessibility part of everything you do, every decision you make, every product you design and build. Remove the need for people to use AI to compensate, and remember, accessibility is resistance.
Also last month, I wrote about enhancing images from a carousel to masonry using CSS Grid Lanes. Here’s the context:
Over on The Session, I added a little enhancement to the events and sessions listings recently. I make a call to the Google Places API to see if I can find a match for the venue, and if I do, pull in some photos.
Sidenote: right now there’s a major issue with this. None of the photos come with text descriptions. This is something I need to fix, and I’ve got some ideas on how to do that.
I was one of those people with a website that doesn’t have alt text for images. I was one of those people not doing their job.
The first fix I put in place was very much in keeping with the ethos of The Session. I added a form so that any member of the site can edit the alt text of the images in the carousel/masonry layout.
The Session is quite WikiPedia-like. Anyone can edit the track listings in the discography. Anyone can edit the details of any tune. Anyone can edit the details of any session or event. So it made sense to add one more thing that anyone can edit.
With that functionality in place, I put out the call:
If everyone were to take just one or two sessions or events and add alt text to the photos, it would really, really help make the site more accessible.
The community went to work and did a pretty good job.
But there are a lot images. I’d like to have a least some kind of alt text for all of them, even if it’s just a placeholder until someone gets around to adding something better.
I was going to have to use generative “AI”, wasn’t I?
Holding my nose, I investigated some alt-text-as-a-service APIs out there. I signed up for some trials and kicked the tyres. They were rubbish. I suspect most of them were vibe-coded.
The more I thought about it, the more I realised that I probably didn’t need the latest and greatest energy-guzzling large langauge models. Some good old-fashioned machine learning could do the trick here. I’m aiming to do a relatively straightforward transformation—image to text.
Check out the COCO project: Common Objects in Context. It’s a few years old now, but it’s basically a dataset that’s been trained on images.
Ah, but where did the training data come from?
All images in COCO were sourced from Flickr, a photo-sharing platform where users upload images under various Creative Commons licenses.
Works for me!
I found a model on Hugging Face that was created by Microsoft from the COCO data. Confusingly the model is called GIT. It’s nothing to do with version control. It stands for GenerativeImage2Text.
I downloaded the model and set to work creating a script to trawl through all those images and generate alt text if the image doesn’t already have one.
I should mention: the way that I store the alt text is not in a database, but in the file system. Every image has its own folder. Inside that folder there can also be a file called alt.txt.
So the script needed to loop through lots and lots of folders, generate alt text where needed, and save it into a text file in the same folder.
Python is the go-to language for working with models like GIT. But Python is not my strong suit, to put it mildly. And this was going to be a throwaway script that I was only going to use once. So I held my nose again and turned on Copilot in VSCode (I usually keep it switched off).
With the right instructions I was able to get a script that—crucially—I could read and understand. Then I switched off Copilot and had a shower to try to wash the shame away.
Long story short, the script worked.
The results are mostly fine. The alt text is functional. Sometimes it’s wrong, but not disastrously so. A human-generated image description will always be preferable, so any member of The Session can over-ride the placeholder alt text.
It took a few hours to loop through all those folders and generate all those text files. My laptop got hot. But doing it locally sure beats paying for tokens from the baddies.
While the utility is there, the current landscape of these frontier corporate models comes with massive ethical and environmental baggage. This is not a long term solution!
That’s exactly why my hope is that ethically trained open-weight models will eventually catch up and that in the next couple of years, we’ll have something with this level of capability that can run locally. That feels like a much better place to be with this technology.
Put the kettle on. Marcin has another magnum opus on interaction design for you to read …and interact with.
This is excellent advice and I’m glad to see this getting addressed nice and early in the era of customisable select elements:
always provide text content or accessible text attributes for your option elements.
This is a great case study featuring a really useful HTML web component called validation-enhancer.
The results? When we launched, the number of people completing the form doubled. The analytics people didn’t even know where these users were coming from. Of course, your javascript-based analytics package doesn’t see the users you are bouncing because of javascript failures. It was a flood!
You cannot kill a federated thing by killing one node, the way you can kill a platform by changing one company.
Fixed an issue on iOS and iPadOS where
datalistsuggestions were presented directly over the associated input, obscuring it.
I really like the thinking that goes into this approach. It seems so counter-intuitive at first, but there’s no arguing with the snappy resilient results.
Turns out, if you have a website and you think of the browser as a way to navigate documents — rather than a runtime to execute arbitrary code and fetch, compile, and present them — things can be a lot simpler than our tools often prime us to make them.
This really gets to the heart of one of the biggest benefits of HTML web components: composability. You can nest your regular markup inside multiple custom elements; something that is can’t do.
The other exciting approach doesn’t exist yet: custom attributes. Again, they’d be a great way of using composability to turbo-charge your existing HTML in all sorts of ways.
In the latest issue of Matthias’s excellent Own Your Web series, he describes the recent betrayal by Google:
The search engine no longer says “here, go read what this person wrote.” It now says “here, I’ve already read it for you.” The contract is broken.
He’s absolutely right.
But…
Have you ever clicked on a result from a search engine? Unless you’re lucky enough to land on a nice personal website, you’re more than likely to be confronted with pop-ups to allow tracking, or a desparate plea to subscribe to a newsletter, or just rubbish ads all accompanied by a slow page loading somewhere in the mix.
Don’t get me wrong. I’m not saying that what Google is doing is okay. But let’s not pretend that everything indexed by Google is just fine and dandy for people to visit.
And of course the main reason why websites are so terrible is because they’ve tied their business model to heaps of behavioral advertising driven by invasive tracking courtesy of …Google.
This reminds me of AMP. Remember Google AMP? It was a terrible solution to a real problem. Web pages were (and still are) bloated and slow. The correct solution would be to encourage people to fix that, but instead Google mandated a proprietary format for your content that had to be hosted on their servers.
AMP was a disaster, both in practical terms and in the reputational damage it did to Google’s developer relations.
Now they’re doing it again, powerwashing away any goodwill they ever had with site owners. Now Google doesn’t even send search engine traffic to the websites that host the ads that Google encouraged people to put on every page.
It’s almost as if Google is a company so large and with so many competing interests that it now suffers from an incurable split personality disorder.
Personally I think they’re missing a trick. They should be using “AI” summaries as a stick.
If your site is slow, or filled with user-hostile annoyances then it should be cockblocked by a hallucinated summary. But a nice fast respectful website? Send the traffic their way! Everyone wins—users, site owners, Google, the World Wide Web.
Could you imagine how quickly this would revolutionise the world of search engine optimisation? They’ve always told us that we should make websites for humans in order to get good Google juice. This would be a way of making it come true, without any of the over-engineered woefulness of AMP.
It’ll never happen of course. But I can dream.
Hallelujah! Support for sizes="auto" is finally landing in Firefox and Safari! Praise be!
Here’s another nice progressive web component for your forms, this time for showing error messages.
Here’s an excellent progressive web component from Aaron—wrap a custom element around your exising form and your good to go:
At its core,
form-saveris a small web component that wraps a form, keeps an eye on it, stores values inlocalStorage, and restores them when the page loads again. Better yet, it clears out saved data after a successful submission so you’re not accidentally resurrecting stale information the next time someone stops by.
People talk about the effectiveness (or lack thereof) of large language models as though all tasks are comparable. But it strikes me that there are three broad categories of work that large language models are applied to:
Compression is when you feed a large language model something big that you want to make small. Summarise this book. Give me the gist of this meeting. Large language models are generally pretty good at this, which makes sense given that they themselves are kind of like compressed artifacts.
Transformation is when large language models convert from one format into another. Turn this audio into text. Turn this jumble of data into structured JSON. A large language model can handle these tasks pretty well. There’ll probably be a few errors so make sure that’s not a deal-breaker.
Expansion is when you give a large language model a prompt to generate something from scratch. An image. A presentation. An email. A poem. This is where slop lives. The output inevitably betrays its origins, glistening with a sheen of mediocrity.
Laurie spotted this three-way split a while back:
Is what you’re doing taking a large amount of text and asking the LLM to convert it into a smaller amount of text? Then it’s probably going to be great at it. If you’re asking it to convert into a roughly equal amount of text it will be so-so. If you’re asking it to create more text than you gave it, forget about it.
I hope that when the bubble finally bursts, we’ll see the surviving large language models put to work on the first two categories. The boring stuff. The work that’s tedious for humans.
But tedious is as tedious does. Something I consider drudgery might be the very thing that gives you life. Like Giles says:
I have a feeling that everyone likes using AI tools to try doing someone else’s profession. They’re much less keen when someone else uses it for their profession.
The big exception seems to be programming. Apparently there are plenty of coders who never before expressed an interest in being managers who are now happily hanging up their coding spurs in favour being the overseer of non-human workers.
It’s a reasonable outlook. It could even be considered a user-centred approach. Users don’t care about the elegance of your code; they care about accomplishing their tasks.
Programming is something of an exception to the efficacy of large language models in general. Instead of relying on the subjectivity of painting, poetry, or prose, programming can be objectively tested. Throw enough money at the worst people in the world and they’ll give you tokens you can use to get the machines to test their own output. So you can get a large language model to create something reasonably good from scratch as long as that something is code.
If you had asked me about the threat model of large language models two years ago, I probably would’ve been worried for artists, writers, and musicians. I thought that software had enough inherent complexity to be relatively safe.
Now my opinion has completely reversed. Software is almost certainly the killer app for large language models.
I think the artists, writers, and musicians will be okay, or at least as okay as they ever were. It turns out that humans like things made by other humans.
And y’know what? If I had to choose which endeavour I’d rather see automated away—programming or art—it’s no competition.
Don’t get me wrong—it would be nice if everyone got paid for doing what they enjoy. It’s just that I’m okay with software engineers not being at the front of that line.
I remember when I first started getting paid money to make websites. “Really?” I thought, “Someone is willing to pay me to do something I’d do anyway?” I kept waiting for the jig to be up. Instead I saw my profession grow and expand.
Perhaps there’s a long-overdue compression happening.
Or maybe it’s more like a transformation.
Four years ago I wrote about something that has long puzzled me in the world of front-end development. Trust:
The mindset I’ve noticed is that many developers are suspicious of browser features but trusting of third-party libraries.
Developers are more likely to trust, say, Bootstrap than they are to trust CSS grid or custom properties. Developers are more likely to trust React than they are to trust web components.
That post got some thoughtful responses but I never really understood the imbalance of trust and suspicion:
I’m kind of confused by this prevalent mindset of trusting third-party code more than built-in browser features.
But something happened recently that helped me understand that mindset better.
I wrote a while back about how the datalist element on iOS has been completely fucked up. It’s worse than if Safari simply didn’t support it.
Breaking the web like that should be a five-alarm fire, but nobody is in any rush to fix it. I recall a similar lackadaisical attitude when Safari completely broke their implentation of IndexedDB.
I had it in my head that browser features followed a forward path generally. They’d be iterated on and improved on to iron out any glitches, but it was reasonable to expect things to get better with each new version of a browser.
Now I see that’s not necessarily the case.
Had I used an over-engineered JavaScript library instead of the datalist element, I wouldn’t be facing the current situation of having to use browser-sniffing to avoid sending a standard HTML element to any browser on iOS.
Sure, that third-party JavaScript would mean that users are downloading more code, and it probably wouldn’t work well with assistive technology, but as long as I didn’t touch it, it would continue to work. That should be true of web standards—I should be able to use them secure in the knowledge that they won’t suddenly shit the bed.
Perhaps I should be grateful to Apple for dispelling my naïveté. I now have much more empathy and understanding for web developers who are suspicious of web standards and prefer to use third-party libraries instead.
Good job, Apple. Happy anniversary.
Web development follows a familiar cycle. First we glue together a solution with whatever we have — JavaScript, image hacks, Flash, anything. Then the platform matures, and CSS or HTML eventually makes that same workaround native. Rounded corners, custom fonts, smooth scrolling, sticky positioning: all of these started as JavaScript-heavy hacks before CSS turned them into a single declaration.
We are in another one of those transition moments. A new wave of long-requested CSS features is finally landing, and many of them are explicitly designed to replace patterns that used to require JavaScript. Not as approximations — as first-class platform primitives that handle the edge cases, run in the right thread, and need zero dependencies.
The choice isn’t between books and screens. The choice is between intentional design and profitable chaos. Between habitats that cultivate human potential and platforms that extract human attention.
I mean, I would ask for the country first (because not all countries have zip/postal codes), but the point stands…
The Session has been online in some form since the late 1990s. That’s long before web fonts existed.
To begin with, Times New Roman was the only game in town if you wanted serif type on a website. When Microsoft introduced Georgia it was a godsend. A beautiful typeface designed by Matthew Carter for the screen. I put it right at the start of my font stack for The Session.
Later, web fonts came along. Boy, does that short sentence belie the drama! There were very heated discussions about whether web browsers should provide this ability at all, and what it would mean for type foundries.
Microsoft led the way with their prorietary EOT format. Then everyone agreed on WOFF. Finally we got WOFF2, Electric Boogaloo.
Perhaps more important than that, we got intermediaries. Typekit, Fontdeck, and then the big daddy, Google Fonts.
That’s pretty much the state of play today. Oh yeah, and we’ve got variable fonts now.
I remember Nick Sherman presenting the idea of variable fonts at an Ampersand event years ago. I remember thinking “great idea, but it’ll never happen.” Pure science fiction. I thought the same thing when I first saw a conference presentation about a miraculous image format called Scalable Vector Graphics.
Sometimes I like to stop and take stock of what we take for granted in web browsers now. Web fonts. Variable web fonts. SVG. Flexbox. Grid. Media queries. Container queries. Fluid typography. And I haven’t even mentioned how we were once limited to just 216 colours on the web.
Given all the advances in web typography, you might be wondering how my font strategy for The Session changed over the years.
It didn’t.
I mean, sure, I added fluid typography. That was a natural extension of my love for liquid layouts and, later, responsive design. But the font stack itself? That was still Georgia all the way.
Y’see, performance has always been a top priority for The Session. If I was going to replace a system font with a web font that the user had to download, it really needed to be worth it.
Over the years I dabbled with different typefaces but none of them felt quite right to me. And I still think Georgia is a beautiful typeface.
“But your website will look like lots of other websites!” some may cry. That used to be true when all we had was system fonts. But now that web fonts have become the norm, it’s actually pretty unusual to see Georgia in the wild.
Recently I found a font I liked. Part of why I like it is that it shares a lot of qualities with Georgia. It’s Lora by Olga Karpushina and Alexei Vanyashin.
I started to dabble with it and began seriously contemplating using it on The Session.
It’s a variable font, which is great. But actually, I’m not using that many weights on The Session. I could potentially just use a non-variable variety. It comes in fixed weights of regular, medium, semibold, and bold.
Alas, the regular weight (400) is a bit too light and the medium weight (500) is a bit too heavy. My goldilocks font weight is more like 450.
Okay, so the variable font it is. That also allows me to play around with some subtle variations in weights. As the font size gets bigger for headings, the font weight can reduce ever so slightly. And I can adjust the overall font weight down in dark mode (there’s no grading feature in this font, alas).
Lora supports a lot of alphabets, which is great—quite a few alphabets turn up on The Session occasionally. But this means that the font file size is quite large. 84K.
Subsetting to the rescue!
I created a subset of Lora that has everything except Cyrillic, Greek, and Latin Extended-B. I created another subset that only has Cyrillic, Greek, and Latin Extended-B. Now I’ve got two separate font files that are 48K and 41K in size.
I wrote two @font-face declarations for the two files. They’ve got the same font-family (Lora), the same font-weight (400 700), and the same font-style (normal) but they’ve got different values for unicode-range. That way, browsers know to only use appropriate file when characters on the page actually match the unicode range.
The first file is definitely going to be used. The second one might not even be needed on most pages.
I want to prioritise the loading of that first subsetted font file so it gets referenced in a link element with rel="preload".
As well as file size, my other concern was how the swapping from Georgia to Lora would be perceived, especially on a slow connection. I wanted to avoid any visible rejiggering of the content.
This is where size-adjust comes in, along with its compadres ascent-override and descent-override.
Rather than adjusting the default size of Lora to match that of Georgia, I want to do it the other way around; adjust the fallback font to match the web font.
Here’s how I’m doing it:
@font-face {
font-family: 'Fallback for Lora';
src: local('Georgia');
size-adjust: 105.77%;
ascent-override: 95.11%;
descent-override: 25.9%;
}
And then my font stack is:
font-family: Lora, 'Fallback for Lora', Georgia, serif;
It’s highly unlikely that any device out there has a system font called “Fallback for Lora” so I can be pretty confident that the @font-face adjustment rules will only get applied to browsers that have the right local font, Georgia.
But where did those magic numbers come from for size-adjust, ascent-override, and descent-override?
They came from Katie Hempenius. As well as maintaing a repo of font metrics, she provides the formula needed to calculate all three values. Or you could use this handy tool to eyeball it.
With that, Georgia gets swapped out for Lora with a minimum of layout shift.
Even with the layout shift taken care of, do I want to serve up web fonts to someone on a slow connection?
It depends. Specifically, it depends on whether it’s their first time visiting.
The Session already treats first time visitors differently to repeat visitors. The first time you visit the site, critical CSS is embedded in the head of the HTML page instead of being referenced in an external style sheet. Only once the page has loaded does the full style sheet also get downloaded and cached.
I decided that my @font-face rules pointing to the web fonts are not critical CSS. If it’s your first time visiting, those CSS rules only get downloaded after the page is done loading.
And unless you’re on a fast connection, you won’t see Georgia get swapped out for Lora. That’s because I’ve gone with a font-display value of “optional”.
Most people use “swap”. Some people use “fallback”. You’ve got to be pretty hardcore to use “optional”.
But the next page you go to, or the next time you come to the site, you more than likely will see Lora straight away. That’s because of the service worker I’ve got quietly putting static assets into the Cache API: CSS, JavaScript, and now web fonts.
So even though I’m prioritising snappy performance over visual consistency, it’s a trade-off that only really comes into play for first visits.
I’m pretty happy with the overall strategy. Still, I’m not going to just set it and forget it. I’ll be monitoring the CRUX data for The Session keeping a particular eye on cumulative layout shift.
Before adding web fonts, the cumulative layout shift on The Session was zero. I think I’ve taken all the necessary steps to keep it nice and low, but if I’m wrong I’ll need to revisit my strategy.
Update: Big thanks to Roel Nieskens—of Wakamai Fondue fame—who managed to get the file size of my main subsetted font down even further; bedankt!