Google Summer of Code '24 Final Submission

I'm so excited to be writing this. Some months ago, I indicated my interest in contributing to the Wagtail CMS project for the 2024 Google Summer of Code program, and my proposal to "improve alt text capabilities" was accepted.
Story time!!

Background

In the Wagtail CMS, "blocks" are the simplest units for storing content. They work similarly to Django's model fields and forms. For every type of data you would want to store in Wagtail, there is a variety of blocks. CharBlock for characters, DateBlock for dates, IntegerBlock for integers, and so much more for types that their prefixed names represent.

Apart from these primary blocks, Wagtail has some more advanced blocks such as PageChooserBlock for "choosing" page objects, and ImageChooserBlock for choosing images from the gallery of already-uploaded ones or for uploading and then selecting. Here's a full list of blocks in Wagtail.

My project involved working with the ImageChooserBlock that looked like the following in an empty state:

The Problem

When an image gets chosen, the ImageChooserBlock looks like:

Wagtail leaves the implementation of alt text mechanisms in the hands of the developers that build with it, and the only fallback for missing alt texts is the image's title. The title field gets auto-populated with the image's filename on the computer. This field is editable but is often ignored by editors. Here's a survey carried out on Wagtail sites where alt texts were graded on a scale of 0-5. About 54% of images on the web pages examined had poor alt texts, and 22% had completely meaningless alt text. Among the 22% were mostly alt texts that looked like computer defaults.

Project Goals

1. To provide a new and improved ImageBlock with support for handling contextual image alt texts.
2. To add a new description field to the image model.

What I did

1. I created an RFC to communicate my proposed changes in technical detail with the Wagtail core team.
2. I created a new ImageBlock with full backward compatibility with the ImageChooserBlock, hence removing the need for data migration.
3. I created a new description property for the AbstractImage class.
4. I made documentation efforts on both.
5. I wrote tests for all of the new features I introduced (included in their PRs).
6. I updated the official demo site's repository to replace the ImageChooserBlock with the newer and more recommended ImageBlock.
7. I collaborated with my mentors to develop a proposed image alt text flowchart for Wagtail.

The Current State

At the point of my writing this, the RFC has been merged, while the ImageBlock and description field PRs are still being reviewed. Feedback on those two PRs looks good but needs more reviews before a merge to the codebase. The expectation is that these changes will feature in Wagtail's 6.3 release, which should be on the 1st of November, 2024.

Challenges

Among some of the challenges I faced with the project were:

1. Scope change: Certain unexpected things came up, and adjustments were made. I was consequently also unable to work on the project's stretch goals.
2. Community requirements: My mentors and I always carried the community along in all the decisions we were to make. It came at a bit of a cost because feedback wasn't as rapid, and in some cases, the community opted for a different approach from what we had in mind. Regardless, the synergy was worth it.
3. Codebase internals: Some of the fixes involved deeper parts of Wagtail's internal code, and I found myself frequently reaching out to maintainers for more clarity and advice.

Lessons

I learned a lot about collaboration, time management, working as a community, and reaching out for help.
I also significantly strengthened my understanding of git, after frequently damaging my commit histories.

Overall, I learned much more about Wagtail and the magic behind what it does.

Potential Future Modifications

1. As is, the ImageBlock can be modified to have a suggestions UI. This suggestion UI can prompt the editor to re-use the image's description and then populate the alt text field with it.
2. A possibility of using AI models to help generate alt text for the images, and then prompting with the suggestion UI. AI models perform quite well with image alt texts in general, and suggestions from them would be of great help in nudging editors to be more thorough (and creative) with alt texts. For a proof of concept, I built a loosely coupled Chromium-based browser extension to show how AI models excel with alt text.

Credits

It took other people's additional efforts to achieve this project's objectives. I was privileged to work with the following people/groups:

• My mentors Storm Heg and Saptak Sengupta.
• Wagtail's GSoC coordinator Thibaud Colas.
• Wagtail's lead developer Matthew Westcott.
• LB, Tomasz, and Vince Salvino for their reviews of my RFC.
• Wagtail's Accessibility team.
• Wagtail's Core team.