Having successfully created an initial javascript powered badges page for my website I wanted to move towards creating a more permanent solution.

So I decided to set up a new Eleventy project and copy over the badge related stuff. This was slightly complicated by the fact that the day I decided to do this was the day Eleventy version 3.0 was released. I’d been thinking about trying it out anyway, so this seemed like a good opportunity. So I switched most of my code over to ESM modules without any difficulty. The only ones that gave me any trouble were the CSP generation files, and I’d copied those from somewhere else, so I initially took the easy way out and let them stay as they were (making them .cjs files). Now I had a working project it was time to generate a real website.

Originally I had assumed that I would copy the URL structure from WordPress thus giving me the option of setting up page by page redirection, however that would mean paying WordPress money and after recent events I felt pretty disinclined to do that, plus paying them to leave felt pretty ridiculous.

So I had free reign to design a new scheme. So I decided on a simple structure
• /badges/badge-number/badge-title for the badges themselves as the date on which they were posted is largely irrelevant (and badge title is not always unique).
• /categories/category-name for categories
• /tags/tag-name for tags
• / for the main page

I started with the individual badge pages. I’d already written code to pull in tags and categories from WordPress at build time, so fetching the data was simple and creating a page from a data object is straightforward.

At this point though the complications multiplied.
I needed to create a ‘processed’ version of the badge data. WordPress provides the title and content as rendered HTML. It also throws in extra non breaking spaces into the title (presumably to prevent orphaned words) but in my masonry view l realised they had to go, as did the escaped characters which were spoiling the title slug (the he library was my friend here).
I wanted each badge to be identified by its collection number which wasn’t data WordPress knew, other than I’ve named all the badge images consistently. Time for a regular expression to retrieve it.
Similarly I wanted a very specific excerpt which needed several regular expressions to create.
The tags and categories come back from WordPress as numeric ids but I needed to look up the names and (worse than that) the names had also been rendered into HTML so again I need to decode them so that they could be used by slugify to generate a link.

(Yes, I’m going to have to rewrite a lot of this when I have static local data).

Now I had badge pages, I could move on. The sidebar links were next as it merely needed the links directing to my pages rather than WordPress.

(I skipped doing the search at this point, we’ll come back to that later).

Once I had some page pages built from data, it was time to move onto the tags and categories. These two things are basically the same, for each I wanted a page with a list of badges. In some cases though there are a lot of badges (the badge category is actually everything), so it was going to need paginating.

For the badge pages I simply used Eleventy pagination with a page size of one. For tags and categories though I wanted pagination for each individual tag, the ‘double pagination problem’. This requires creating a collection (in my case from the badges data) indexed by both tag (or category) and page number. This site was incredibly helpful in giving me an approach to follow.

I discovered a few interesting things (mostly by accident) here:

• You can create a collection of pages that are generated from data. You need to set.

  pagination:
    data: badges
    addAllPagesToCollections: true

  in the front matter and then you can do

  collection.getFilteredByGlob('**/badge.njk');

• You can create a collection from your global data directly using

  collection.getAll()[0].data.badges

I wanted to use the former method basing the collection on the pages but I couldn’t work out how to access the badge data for each one. So I used the latter. Then I built a double collection and created pages from that. I added next and previous buttons so all the badges for a category (or tag) could be viewed.

Javascript or no Javascript

Given that I had built this whole thing already in a single page you might be wondering why I went through all that (I certainly was). I had decided I wanted the pagination to work without javascript and only if javascript was on I would replace it with progressive loading.

I’d already written the progressive loading logic in my one page version, all I needed was something to progressively load. For this I created a second template for each category, tag and the index that output json instead of HTML. The trickiest bit was generating valid json containing my HTML excerpt string that I needed for each card. The solution here was to use filters in my template

{
"excerpt" : {{ badge.excerpt | dump | safe}}
}

did the trick. I had to add a little bit more code to handle switching over to progressive loading gracefully and I was done (except of course for a lack of search).

So now I have a working and deployable site. Cue cheering.

At this point I took the time to revisit the CSP generation, as it was pretty slow over the ~1500 pages I now had. I realised that the code I had copied used the JSDOM library and found a similar library, linkedom, which ran significantly faster. Rewriting my code slightly gave me the impetus to convert it into module format as well.

Searching

Being able to search my badges is incredibly important to me. After all the lack of a search box on my main WordPress page was one of the things that triggered all this.

I had been recommended PageFind as a search engine. The documentation made it sound ridiculously easy. Mark up your html, point PageFind at it and go!
At one level it was; in only a few minutes I had a searchable site. I created a search box in the sidebar to take you to a search results page to run the requested search and it felt nearly done.

The tricky bit now was that I wanted the search page to display the same badge cards as the rest of the site.

PageFind is set up to index html pages, and it extracts the text it finds and offers this up as search results. I wanted the same excerpt as elsewhere and that contains a table. Luckily PageFind allows you to add your own metadata to indexed content.

I added an attribute in my page to contain my excerpt but now I needed to insert html containing quote marks into an attribute string. Javascript escape function to the rescue. Creating an Eleventy filter to call that function provided a valid string.
Then my search javascript can unescape the metadata before displaying it. This all feels incredibly clunky and it is possible there’s a better way that I’ve missed but for now I have a search that works.

Creating a static site from WordPress has mostly involved a lot of escaping data in one way or another. I have a sneaking suspicion that the next stage of this project will involve a lot of rewriting, but this stage is done for now.

As you may have noticed I actually have two blogs, this one (obviously) and my badge blog. The badge blog acts as a reference to all my badges and is primarily to enable me to easily find out what badges I have on a specific topic.

I rather like:
• The masonry layout for displaying posts
• Progressively loading posts into the same page (rather than navigating to separate pages)
• The ability to search for badges from the sidebar

There are a few things that I am less happy about though:
• The main page doesn’t have a sidebar so doesn’t have a search link (and I can’t add it to the menu instead).
• Search results don’t show an image of the badge.
• WordPress in general have made the editing experience less enjoyable over time and generally nerfed things in a variety of ways.

In fact as well as editing on WordPress being increasingly difficult, it now seems impossible to pick any kind of modern theme and customise it without paying them money. So maybe I can do better. And because I’m busy, maybe I can improve things in stages, so I don’t have to recreate the entire blog all at once.

It turns out WordPress actually have a really good API which will serve public WordPress content (and private content if you log in but I don’t need that).
So my theoretical plan* is as follows :
• Create a single page with javascript to fetch and display badges (by category, tag or search) direct from WordPress via the API. This will give me a chance to play with layouts and functionality.
• Create a full blown site with pages/data pulling in the data from WordPress at build time (and allowing the pages to pull from that site).
• Work out what to do about blog comments (!)
• Export the content from WordPress into my site (so I can turn off WordPress).

This post details the first stage. As this is an experiment, I have decided to host a single page on my website to act as a front end. Eventually I will need to decide what URL the badges should live at, but for now this is fine.

Firstly I needed to extend my Content Security Policy to allow resources from WordPress. I took advantage of this to make the CSP policy configurable (and set a default for most pages) in the base layout file. Then I created a second CSP which includes WordPress.

I wanted a side bar to display search, categories and tags. Since I have over five hundred badges, the categories and tags aren’t going to change much so I can pull them in as static content (although I realise the post counts will get out of date). This gave me a chance to use the Eleventy Fetch plugin and play with creating the categories list and a tag cloud from the javascript data file.

The list of posts can be dynamically pulled from WordPress via javascript. There are parameters to choose the page size. I chose twelve, of course, because that will produce (roughly) even columns for the most number of columns (without loading them in sixties). There are also parameters to restrict the fields pulled back, so I can limit the data to id, title, link and content. There’s a little bit of jiggery pokery needed (a regexp) to handle the occasional ‘more’ tag in a post, but with very little effort we have a grid layout of posts, fetched by most recent, tag, category or search. The API also provides the total available number of posts in a header, so rather than having automatic infinite scroll, I have added a ‘more’ button to pull them down a page at a time.
At this point, it feels like I should be done. But I am rather attached to the masonry layout, although (I realise) less attached to the floaty animation WordPress does when it rearranges the posts. There are a variety of techniques to do masonry layouts, and I’ve considered several of them, but in the end I decided to keep it simple and go with an approach based on using a grid layout. The basic premise is that you adjust your grid layout by pulling items up (using top margin) to fill in the gaps. Given that my badges all have roughly the same length I feel this is good enough, but it does technically mean the badges won’t strictly be listed going down the page as they will always stay in their column regardless.

The result is a page that gives me a better search experience and has been a lot of fun to create. It might need some tweaking still, so I’ll try it out for a bit and who knows eventually I might move on to the next stage.

*I might stop at any point, who knows, or even refine the plan further.