XML Sitemaps: What to Include and What to Leave Out

An XML sitemap should list only the URLs you want to appear in search: canonical pages that return a 200 status code and are not blocked or set to noindex. Leave out redirects, error pages, duplicates, parameter variations, and anything private or thin. A clean sitemap of real pages, with accurate lastmod dates, is worth far more than a big one that lists everything your CMS can generate.

Key takeaways

• Include only canonical, indexable URLs that return 200.
• A single sitemap file can hold up to 50,000 URLs or 50 MB uncompressed; use a sitemap index for more.
• Google ignores priority and changefreq, but uses lastmod when it is consistently accurate.
• Submit the sitemap in Search Console and list it in robots.txt.
• A sitemap helps discovery; it does not guarantee indexing.

What a sitemap does (and does not do)

A sitemap is a list of URLs you give search engines so they can find and recrawl your pages more efficiently. It is most useful when a site is new and has few external links, when it is large, when content changes often, or when some pages are not well linked internally.

It is a hint, not a command. Google decides for itself which URLs to crawl and index. A URL in your sitemap can still be left out of the index if Google judges it a duplicate or low quality, and a URL missing from your sitemap can still be indexed if Google finds it through links.

A minimal valid sitemap

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2026-10-12</lastmod>
  </url>
  <url>
    <loc>https://example.com/services/web-design/</loc>
    <lastmod>2026-09-30</lastmod>
  </url>
</urlset>

The file must be UTF-8 encoded, and URLs must be absolute (with https:// and the host). Characters like & inside a URL must be escaped as &.

What to include

Ask one question for each URL: "Do I want this exact address to show up in Google?" If yes, include it. In practice that usually means:

• Your home page and main section or category pages.
• Service, product and location pages.
• Published blog posts and guides.
• Useful hub pages, such as a resources index.

Every URL should be:

• The canonical version. If a page has rel="canonical" pointing elsewhere, list the target, not the page.
• The final URL. Use the exact protocol, host and trailing-slash form your site redirects to.
• Returning 200. No 301s, 302s, 404s or 5xx errors.
• Indexable. Not carrying a noindex tag or header, and not blocked in robots.txt.

What to leave out

A useful sanity check: compare the number of URLs in your sitemap with the number of pages you would honestly call "real content". If the sitemap is three times larger, something is generating junk URLs.

The optional tags: lastmod, changefreq and priority

lastmod

This is the one optional tag worth getting right. Google has said it uses lastmod when it is consistently and verifiably accurate. It should reflect the last significant change to the page content, such as updated text, new sections or changed prices, not a copyright year change or a sidebar update. Use W3C date format: 2026-10-12 or 2026-10-12T09:30:00+00:00.

The common mistake is setting every URL's lastmod to today's date on each sitemap rebuild. Once a search engine learns your dates are unreliable, it has little reason to trust them.

changefreq and priority

Google ignores both. You can leave them out entirely. Other search engines may read them, but there is little evidence that they make a practical difference.

Large sites: split and index

A single sitemap is limited to 50,000 URLs and 50 MB uncompressed. Larger sites split URLs across several files and list those files in a sitemap index:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemap-pages.xml</loc>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemap-posts.xml</loc>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemap-products.xml</loc>
  </sitemap>
</sitemapindex>

Splitting by content type is useful even on smaller sites. In Search Console you can then see indexing coverage per sitemap, which quickly shows whether, say, product pages are being indexed less than blog posts.

Specialized sitemaps

• Image sitemaps add image:image entries, useful when important images are loaded in ways crawlers might miss.
• Video sitemaps describe video content such as title, thumbnail and duration.
• News sitemaps are for publishers in Google News and should only contain recent articles.
• hreflang in sitemaps lets multilingual sites declare language alternates with xhtml:link entries instead of tags in each page.

Most small sites only need the standard sitemap.

How to create and submit a sitemap

1. Generate it. Most CMSs and SEO plugins create one automatically. For a small static site, the XML Sitemap Generator can crawl your pages and produce the file for you to upload.
2. Clean it. Remove anything from the "leave out" list above. Check that every URL resolves to 200 without redirects.
3. Upload it to your site root, for example https://example.com/sitemap.xml. A sitemap generally covers URLs on its own host and below its own folder.
4. Reference it in robots.txt: Sitemap: https://example.com/sitemap.xml
5. Submit it in Google Search Console under Sitemaps, and in Bing Webmaster Tools.
6. Monitor it. Check the Page indexing report filtered by sitemap for URLs that are submitted but not indexed.

Google retired its old sitemap "ping" URL in 2023, so Search Console, robots.txt and accurate lastmod values are the ways to tell Google about changes. See Google's guide to building a sitemap for the full specification of what it supports.

Check a sitemap by hand

Automatic generators are convenient, but they faithfully include whatever your CMS produces, including junk. Once in a while, open the sitemap in a browser and read it:

• Do the URLs use the right protocol and host, with no stray http:// or www versions?
• Are there test pages, "sample page" leftovers, or attachment pages for individual images?
• Do any URLs carry tracking parameters such as ?utm_source=?
• Pick five URLs at random and load them. Do they all return the real page without redirecting?

Ten minutes of reading usually reveals the settings or plugins that need adjusting.

Reading the "submitted but not indexed" gap

If Search Console shows many sitemap URLs as not indexed, look at the reasons it gives:

• Duplicate, Google chose different canonical: your sitemap lists a URL Google considers a duplicate. Fix canonicals or remove the URL.
• Crawled, currently not indexed: Google saw the page and chose not to index it. Usually a content quality or duplication question, not a sitemap problem.
• Page with redirect / Not found (404): your sitemap is out of date. Regenerate it.
• Excluded by noindex: you are listing pages you told Google to skip.

Quick checklist

• Only canonical, indexable, 200-status URLs.
• Absolute URLs that match your preferred protocol and host.
• Accurate lastmod; skip changefreq and priority.
• Under 50,000 URLs and 50 MB per file, with an index file if needed.
• Listed in robots.txt and submitted in Search Console.
• Regenerated automatically when content is added, changed or removed.