[ Feeds & publishing ]

@sdxc/sitemap

Read and write the sitemap protocol: collect URLs, or fetch a published sitemap

Installs with: @sdxc/result @sdxc/xml
Used by: uptime, blog
Source: packages/sitemap

View as Markdown Open in GitHub Open in ChatGPT Open in Claude

Read and write the sitemap protocol: collect URLs into a document, or fetch a published sitemap and read the pages it lists.

One Sitemap class covers both directions, so what append accepts is exactly what a parsed entry reports back, and a document that is read, filtered and serialized keeps its type.

Installation

npm add @sdxc/sitemap

Parsing and retrieval answer a Result from @sdxc/result, and Sitemap.parse takes a document from @sdxc/xml.

Usage

Build a sitemap

import { Sitemap } from "@sdxc/sitemap";

let sitemap = new Sitemap();

sitemap.append(new URL("https://example.com/"), { priority: 1, frequency: "weekly" });
sitemap.append(new URL("https://example.com/blog/hello"), { updatedAt: new Date("2026-09-11") });

sitemap.toString();
// <?xml version="1.0" encoding="UTF-8"?>
// <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">…</urlset>

updatedAt is written as <lastmod>, frequency as <changefreq> and priority as <priority>.

Read a published sitemap

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

let result = await Sitemap.fetch("https://example.com/sitemap.xml");
if (isFailure(result)) throw result.error;

result.data.kind; // "urlset" or "index"
result.data.size; // 128

for (let entry of result.data.entries) {
	entry.loc; // URL
	entry.updatedAt; // Date | undefined
	entry.frequency; // Sitemap.Frequency | undefined
	entry.priority; // number | undefined
}

Parse a document you already hold

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";
import { XML } from "@sdxc/xml";

let xml = XML.parse(source);
if (isFailure(xml)) throw xml.error;

let sitemap = Sitemap.parse(xml.data);
if (isFailure(sitemap)) throw sitemap.error; // SitemapParseError: received <html>

Reading follows these rules:

The root decides what arrived: <urlset> reads as kind "urlset", <sitemapindex> as "index", and any other root is a SitemapParseError naming it.
An entry needs a <loc> that is an absolute URL; a row without one is skipped and the rest of the document is kept.
A value the protocol refuses leaves only its own field undefined: an unreadable <lastmod>, a <changefreq> outside the enum, a <priority> outside 0.0–1.0.
<lastmod> reads as W3C Datetime, so 2026-09-11 and 2026-09-11T14:32:00+02:00 both arrive as a Date.
Extension children (image:, video:, news:, xhtml:link) are left out of the entry; read them from the XML document when you need them.

API

`new Sitemap()`

An empty document of kind "urlset".

`sitemap.append(loc: URL, options?: Sitemap.AppendOptions): void`

Adds one entry with optional updatedAt, frequency and priority (0.0–1.0).

`sitemap.toString(): string`

Serializes the entries under the root this instance carries. Index rows keep loc and updatedAt, the two fields the protocol allows in a <sitemap> element.

`sitemap.kind: Sitemap.Kind`

"urlset" for a built document or a parsed <urlset>, "index" for a parsed <sitemapindex>.

`sitemap.entries: Set<Sitemap.Entry>`

The live set of entries; deleting from it removes the entry from the serialized document.

`sitemap.size: number`

How many entries the document holds.

`Sitemap.parse(xml: XML): Result<Sitemap, SitemapParseError>`

Reads a parsed XML document as a sitemap, following the rules under Parse a document you already hold.

`Sitemap.fetch(input: URL | RequestInfo, init?: RequestInit): Promise<Result<Sitemap, SitemapFetchError | SitemapParseError | XMLParseError>>`

Retrieves and parses one sitemap, taking what fetch takes. Any ok response is parsed whatever its Content-Type; a network error or an error status is a SitemapFetchError.

`SitemapParseError`

The document's root is neither <urlset> nor <sitemapindex>; the message names the root.

`SitemapFetchError`

The request failed or answered an error status; the message carries the cause or status.

Types

namespace Sitemap {
	type Kind = "urlset" | "index";

	type Frequency = "always" | "hourly" | "daily" | "weekly" | "monthly" | "yearly" | "never";

	interface Entry {
		loc: URL;
		updatedAt?: Date;
		frequency?: Frequency;
		priority?: number;
	}

	interface AppendOptions {
		updatedAt?: Date;
		frequency?: Frequency;
		priority?: number;
	}
}

Pattern: Serving a Sitemap

Resolving every entry against the request URL lets one handler serve production and every preview deployment.

import { Sitemap } from "@sdxc/sitemap";

export async function handleSitemap(request: Request, posts: { slug: string; updatedAt: Date }[]) {
	let base = new URL(request.url);
	let sitemap = new Sitemap();

	sitemap.append(new URL("/", base), { priority: 1, frequency: "weekly" });

	for (let post of posts) {
		sitemap.append(new URL(`/blog/${post.slug}`, base), { updatedAt: post.updatedAt });
	}

	return new Response(sitemap.toString(), {
		headers: {
			"Content-Type": "application/xml; charset=utf-8",
			"Cache-Control": "public, max-age=3600",
		},
	});
}

Pattern: Writing a Sitemap Index

A built Sitemap is a <urlset>. Parse an empty <sitemapindex> to get an index, then append one entry per child sitemap.

import { unwrap } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";
import { XML } from "@sdxc/xml";

let empty = unwrap(
	XML.parse('<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"></sitemapindex>'),
);
let index = unwrap(Sitemap.parse(empty));

for (let section of ["pages", "blog", "products"]) {
	index.append(new URL(`https://example.com/sitemap-${section}.xml`), { updatedAt: new Date() });
}

index.toString(); // <sitemapindex …><sitemap><loc>…</loc><lastmod>…</lastmod></sitemap>…

Pattern: Walking an Index Into Its Pages

Sitemap.fetch retrieves one document, so following an index is a loop whose pace you choose.

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

async function listPages(url: string): Promise<URL[]> {
	let result = await Sitemap.fetch(url);
	if (isFailure(result)) return [];

	if (result.data.kind === "urlset") {
		return [...result.data.entries].map((entry) => entry.loc);
	}

	let pages: URL[] = [];
	for (let child of result.data.entries) {
		pages.push(...(await listPages(child.loc.href)));
	}
	return pages;
}

let pages = await listPages("https://example.com/sitemap.xml");

Pattern: Recrawling Only What Changed

updatedAt turns a sitemap into a change feed: check only the pages modified since the last run, and record the ones that stopped answering.

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

let lastRun = new Date("2026-09-01");

let result = await Sitemap.fetch("https://example.com/sitemap.xml", {
	headers: { "User-Agent": "example-crawler" },
});
if (isFailure(result)) throw result.error;

let broken: URL[] = [];

for (let entry of result.data.entries) {
	if (entry.updatedAt && entry.updatedAt < lastRun) continue;
	let response = await fetch(entry.loc, { method: "HEAD" });
	if (!response.ok) broken.push(entry.loc);
}

Pattern: Filtering and Republishing a Sitemap

entries is the live set, so deleting from it edits the document, and toString() writes it back under the root it was read from.

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";

let result = await Sitemap.fetch("https://example.com/sitemap.xml");
if (isFailure(result)) throw result.error;

for (let entry of result.data.entries) {
	if (entry.loc.pathname.startsWith("/drafts/")) result.data.entries.delete(entry);
}

let xml = result.data.toString();

Pattern: Reading a Gzipped Sitemap

A .xml.gz sitemap carries a gzip body. Decompress it with DecompressionStream and parse the text, reporting each failure by its type.

import { isFailure } from "@sdxc/result";
import { Sitemap } from "@sdxc/sitemap";
import { XML } from "@sdxc/xml";

let response = await fetch("https://example.com/sitemap.xml.gz");
if (!response.ok || !response.body) throw new Error(`Status ${response.status}`);

let text = await new Response(response.body.pipeThrough(new DecompressionStream("gzip"))).text();

let xml = XML.parse(text);
if (isFailure(xml)) throw xml.error; // XMLParseError: not well-formed XML

let sitemap = Sitemap.parse(xml.data);
if (isFailure(sitemap)) throw sitemap.error; // SitemapParseError: not a sitemap root