Sitemap lastmod from git history in Astro, and the Vercel shallow clone trap

Google's sitemap docs say it uses <lastmod> "if it's consistently and verifiably (for example by comparing to the last modification of the page) accurate." Most static sites fail that test in one of two ways: they leave lastmod out, or they stamp every URL with the build time. A build-time date changes on every deploy, so it says nothing about the page.

On straffesites.com, a static Astro site with Content Collections (MDX and JSON in Git, deployed on Vercel), every sitemap entry now carries the date of the last commit that touched the content of that page. This article shows the code, the one Vercel setting that makes it work, and the trap I hit on the first deploy: a shallow clone that silently gives almost every file the same date.

Versions used: Astro 7.2.8, @astrojs/sitemap 3.7.3, Node 26, git 2.43.

The idea: one git log per build

Git already knows when each file last changed. You don't need a date registry in the repo, and file mtimes are useless in CI because a fresh checkout gives every file the same mtime.

One git log call over the content directory gives you all dates at once. It lists commits newest first, and the first time a path shows up is its most recent change:

// scripts/git-lastmod.mjs
import { execFileSync } from 'node:child_process';

export function gitDates(root = process.cwd(), dir = 'src') {
  const git = (...args) =>
    execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 }).trim();

  if (git('rev-parse', '--is-shallow-repository') === 'true') {
    throw new Error('sitemap lastmod needs full git history (shallow clone detected)');
  }

  // One git call: each commit date (newest first), followed by the files it touched.
  const log = git('log', '--format=%x00%cI%x00', '--name-only', '--', dir);
  const dates = new Map();
  let date;
  for (const line of log.split('\n')) {
    if (line.startsWith('\0')) date = new Date(line.slice(1, -1)).toISOString();
    else if (line && date && !dates.has(line)) dates.set(line, date);
  }
  return dates; // 'src/pages/blog/hello.md' -> '2026-03-02T09:00:00.000Z'
}

The %x00 bytes around the date make commit headers easy to tell apart from file paths. %cI is the committer date in strict ISO 8601, and toISOString() normalizes it to UTC.

Wiring it into @astrojs/sitemap

The sitemap integration has a serialize(item) callback that runs for every entry before the XML is written. Compute the map once in an astro:build:start hook, then look up each route in serialize:

// astro.config.mjs
import { defineConfig } from 'astro/config';
import sitemap from '@astrojs/sitemap';
import { gitDates } from './scripts/git-lastmod.mjs';

let dates;
// Map a route to the file that holds its main content.
const sourceFor = (route) =>
  route === '/' ? 'src/pages/index.md' : `src/pages${route.replace(/\/$/, '')}.md`;

export default defineConfig({
  site: 'https://example.com',
  trailingSlash: 'never',
  build: { format: 'file' },
  integrations: [
    {
      name: 'git-lastmod',
      hooks: {
        'astro:build:start': () => {
          dates = gitDates();
        },
      },
    },
    sitemap({
      serialize(item) {
        const route = new URL(item.url).pathname;
        const lastmod = dates.get(sourceFor(route));
        if (!lastmod) throw new Error(`No committed date for ${route}`);
        return { ...item, lastmod };
      },
    }),
  ],
});

I tested this in a throwaway repo with three Markdown pages and three commits with fixed dates: all pages on March 1, an edit to hello.md on March 2, and a README-only commit on March 3. The entries in the generated sitemap-0.xml, one per line:

<url><loc>https://example.com</loc><lastmod>2026-03-01T09:00:00.000Z</lastmod></url>
<url><loc>https://example.com/blog/hello</loc><lastmod>2026-03-02T09:00:00.000Z</lastmod></url>
<url><loc>https://example.com/blog/second</loc><lastmod>2026-03-01T09:00:00.000Z</lastmod></url>

The README commit doesn't move any date, because git log -- src only looks at commits that touched src. The integration also wrote a <lastmod> of March 2 on the single entry in sitemap-index.xml, the newest date in that file.

sourceFor is the part you'll have to adapt. On straffesites.com, routes come from a route: field in each MDX file's frontmatter, so the real script reads the frontmatter instead of deriving the path from the URL.

The trap: Vercel's shallow clone

Vercel's build docs say it clones your repo with git clone --depth=10. With only ten commits of history, the oldest commit in the clone is a boundary commit. Git has no parent to compare it with, so git log --name-only lists every file in the tree under that commit.

In the demo repo, a --depth=1 clone shows all three pages under the README-only commit (NUL bytes shown as |):

|2026-03-03T09:00:00+00:00|

src/pages/blog/hello.md
src/pages/blog/second.md
src/pages/index.md

I ran the same parsing on a --depth=10 clone of the straffesites.com repo (1,153 commits at the time). 272 of the 279 tracked content files got the date of the boundary commit, November 10, 2026. With full history, the same 279 files have dates spread from August 26 to November 10, 2026. Without a guard, the sitemap would have claimed that nearly the whole site changed on one day. That's exactly the kind of lastmod that Google learns to ignore.

That's why gitDates() checks git rev-parse --is-shallow-repository and throws. In the shallow demo clone, the build stops in the hook:

[ERROR] [git-lastmod] An unhandled error occurred while running the "astro:build:start" hook
sitemap lastmod needs full git history (shallow clone detected)

A failed build keeps the previous production deployment online. I'd rather have that than a sitemap full of wrong dates.

Why git fetch --unshallow wasn't enough

My first version tried to fix the history itself: if shallow, run git fetch --unshallow --no-tags origin. That works on my machine and in a fresh local clone. On Vercel it failed, because the build checkout doesn't keep credentials for origin, and the repository is private. Here's the same failure reproduced locally with credentials and prompts turned off:

$ GIT_TERMINAL_PROMPT=0 git -c credential.helper= fetch --unshallow --no-tags origin
fatal: could not read Username for 'https://github.com': terminal prompts disabled

On a public repository the fetch may succeed. On a private one, it fails every time.

The fix: VERCEL_DEEP_CLONE=true

Setting the environment variable VERCEL_DEEP_CLONE=true on the Vercel project makes Vercel clone the full history instead of ten commits. I couldn't find it in Vercel's own documentation. It shows up in a vercel/vercel GitHub discussion (#5737), a Docusaurus issue about "last updated" dates, and Zudoku's Vercel deploy guide. All of them need full history for the same reason. Because it's undocumented, treat it as something that can change, and keep the shallow check as a hard failure so you notice when it does. In that same Docusaurus issue, two people report that their deep clone failed on Vercel with a large repository, so check your build log after you turn it on.

On straffesites.com the variable is set for Production and Preview since September 13, 2026. Today (November 11, 2026) I compared the live sitemap-0.xml with the dates computed from a full local clone: 244 of 244 URLs match exactly.

The production script still keeps the fetch --unshallow path as a defensive fallback with a 60-second timeout and GIT_TERMINAL_PROMPT=0, so it can't hang on a prompt. If that fetch fails, the build fails with a clear message instead of guessing.

Which files count as "the content of a page"?

For a blog post, the answer is its own MDX file. For other pages it isn't that simple. A blog index changes when a post is added or removed. A page that renders a data table changes when the JSON behind it changes.

The production script builds a set of source files per route and takes the newest date. This is the dependency part, lightly trimmed:

const sources = new Set([file]);
const dataset = (name) => sources.add(`src/content/datasets/${name}.json`);
if (/<(?:WorkGrid|CaseGrid|CommercialPage)\b/u.test(body)) dataset(`cases${suffix}`);
if (/<BenchmarkTable\b/u.test(body)) dataset('agency-benchmark');
if (/<(?:Articles|InsightGrid)\b/u.test(body)) {
  // Include removals from the directory too: hub membership is content.
  for (const source of dates.keys()) {
    if (source.startsWith(`src/content/articles/${locale}/blog/`)) sources.add(source);
  }
}
const values = [...sources].map((source) => {
  if (!dates.has(source)) throw new Error(`Sitemap lastmod: no committed date for ${source}`);
  return dates.get(source);
});
result.set(route, values.sort().at(-1));

A nice side effect of reading paths from git log: deleted files are still in the map. So when a post is removed, the blog index gets the date of the deletion commit, which is correct because the list on that page changed.

What I deliberately left out: navigation, footer, CSS, related-post rails, and docs-only commits. Those change the HTML of every page, but not what the page is about. The date means "the main content changed," not "some byte in the output changed."

Trade-offs and edge cases

• Committer date, not author date. %cI is when the commit landed. A rebase, amend, or cherry-pick rewrites the committer date, so the touched files will look freshly modified. If you rebase published content often, consider %aI (author date), which survives a rebase.
• Merge commits. Plain git log shows no file list for merge commits. A file changed on a feature branch gets the date of the branch commit, not the merge. If you need "date it went live," use squash merges or add --first-parent. In git 2.43 that lists the merge commit with its changes against the first parent, so the file gets the merge date. I checked this in the demo repo; test it on your git version.
• Uncommitted edits get no new date. That's on purpose: a dirty working tree is not published content. It also means a local build right after an edit still shows the old date until you commit.
• No git, no build. CLI uploads without a .git directory can't produce these dates. The script fails instead of falling back to the build time or mtimes.
• Cost. One git log call over the content tree, regardless of how many routes you have. On the straffesites.com repo (about 1,150 commits) that call takes under 0.2 seconds.

Testing it

Two checks keep this honest.

First, a node:test test that builds a small git fixture with fixed GIT_COMMITTER_DATE values, including an unrelated commit and a dirty file, and then clones it with --depth=1. Trimmed, with comments added:

commit('2026-01-01T12:00:00Z');                       // page + dataset
put('src/content/datasets/cases.json', '{"updated":true}');
commit('2026-02-01T12:00:00Z');                       // dataset only
put('README.md', 'unrelated'); commit('2026-03-01T12:00:00Z');
put('src/content/articles/en/blog/example.mdx', '---\nroute: /en/blog/example\n---\nDirty draft');

const values = buildSitemapLastmod(root);
assert.equal(values.get('/'), '2026-02-01T12:00:00.000Z');             // follows the dataset
assert.equal(values.get('/en/blog/example'), '2026-01-01T12:00:00.000Z'); // dirty edit ignored

execFileSync('git', ['clone', '-q', '--depth=1', `file://${root}`, shallow]);
assert.throws(() => buildSitemapLastmod(shallow), /full git history/);

Second, a post-build check that parses the generated sitemap-0.xml, recomputes the expected value for every route, and fails if a single <lastmod> is missing or different. That check runs on every npm run check, so a change to the sitemap config can't quietly bring back build-time dates.

Summary

• Derive lastmod from git log --name-only once per build and pass it in through serialize().
• Fail the build on a shallow clone. On Vercel, a depth-10 clone gives most files the date of the boundary commit.
• git fetch --unshallow in the build fails on private repos because the checkout has no credentials. VERCEL_DEEP_CLONE=true fixes it, but it's undocumented, so keep the guard.
• Decide which files define the content of a page (datasets, hub listings) and leave layout changes out.

You can see the result in the live sitemap of the site: straffesites.com/sitemap-0.xml.