DEV Community

Cover image for GenAI Image Alt Text
Peli de Halleux
Peli de Halleux

Posted on • Originally published at microsoft.github.io

GenAI Image Alt Text

It is a best practice to provide an alt attribute for images.
This attribute is used to describe the image to users who are unable to see it.
It is also used by search engines to understand the content of the image.

<img src="..." alt="describe the image here" />
Enter fullscreen mode Exit fullscreen mode

However, this task can be tedious and developer are often tempted to skip it, or provide a generic alt text like "image".

<img src="..." alt="image" />
Enter fullscreen mode Exit fullscreen mode

GenAI to the rescue

To solve this issue, we created a GenAIScript script that uses the OpenAI Vision model (gpt-4-turbo-v or gpt-4o) to analyze the documentation images and generate a description alt text.

We assume that the script is run on a single image file
and we use defImages (provided by GenAIScript) to add it to the prompt context.

const file = env.files[0] // selected in UI or cli
defImages(file) // encode image in openai json payload
Enter fullscreen mode Exit fullscreen mode

Then we give a task to the LLM to generate a good alt text.

$`You are an expert in assistive technology. 
Generate a description alt text for the image.`
Enter fullscreen mode Exit fullscreen mode

finally, we use defFileOutput to define a file output route.

defFileOutput(file.filename + ".txt", `Alt text for image ${file.filename}`)
Enter fullscreen mode Exit fullscreen mode

Usage in Astro

The GenAIScript documentation uses Astro, which allows to author pages in MDX.
The code below shows how the generate alt text, stored in a separate text file, is injected in the final HTML.

import { Image } from "astro:assets"
import src from "../../../assets/debugger.png"
import alt from "../../../assets/debugger.png.txt?raw"

<Image src={src} alt={alt} />
Enter fullscreen mode Exit fullscreen mode

The debugger.png image shows the screenshot of a debugging session and the generate alt text file contents.

Full source

script({ model: "openai:gpt-4-turbo-v", maxTokens: 4000, })
const file = env.files[0]
defImages(file)
$`You are an expert in assistive technology. 
Generate a description alt text for the image.
- Do not include Alt text in the description.`
defFileOutput(file.filename + ".txt", `Alt text for image ${file.filename}`)
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
greenersoft profile image
GreenerSoft

Funny, but is it really necessary to just describe an image?
What is the environmental cost of this?