The sitemap is a file that lists the URLs of a website to facilitate their discovery by search engines. Google, Bing, and their crawlers rely on this file to methodically browse a site. However, the role of the sitemap has evolved in recent years, and some of its technical fields have become trust signals rather than mere structural indicators.
The lastmod field of the sitemap: a reliability signal that Google monitors
The lastmod tag in an XML sitemap indicates the date of the last modification of a URL. Long seen as a simple freshness marker, it now serves as a credibility indicator for the entire site.
When lastmod dates are systematically inaccurate (all identical, all equal to the site’s build date), Google tends to ignore this field for the entire domain. The site then loses a lever for prioritizing the recrawl of its pages that have actually been updated.
A sitemap whose dates accurately reflect actual changes directs crawlers to fresh content, which speeds up the consideration of updates. Conversely, a “deceptive” sitemap sends a muddled signal, and search engines eventually turn away from it.
This type of functioning can be observed by consulting the sitemap page of Le Comptoir de Violette, which illustrates the structure of a sitemap file accessible directly from a site.
Priority and changefreq tags of the XML sitemap: still useful or obsolete
The sitemaps.org protocol still documents the optional tags priority and changefreq. The first assigns a relative priority value among the site’s URLs. The second indicates the estimated frequency of a page’s modification.
In practice, major search engines ignore these two fields. Google has confirmed this several times: its algorithms determine the crawl frequency and indexing priority independently of what the sitemap file states.
Keeping these tags is not penalizing, but removing them reduces the size of the sitemap file and simplifies its maintenance. For large sites (several thousand pages), this reduction in size can have a marginal effect on the speed of processing by crawlers. Field reports vary on this point, with some webmasters noticing no measurable difference after removal.
XML Sitemap and AI Search: an emerging role in content freshness
AI-assisted search environments, such as Google’s generative responses, place increasing importance on content freshness. The XML sitemap, when its lastmod dates are reliable, becomes a signal that these systems can leverage to determine which pages deserve to be prioritized.
This phenomenon is still recent, and the available data does not allow for conclusions about its exact scope. However, the logic is coherent: an AI engine generating responses needs up-to-date sources, and the sitemap remains one of the few standardized files that explicitly states the modification date of a page.
Content of an XML sitemap: which URLs to include and which to exclude
An effective sitemap does not list all the URLs of a site. It only contains the pages intended to appear in search results. Including non-indexable, redirected, or duplicate URLs muddles the signal sent to engines.
The filtering criteria to apply before generating or updating a sitemap:
- Exclude URLs blocked by the robots.txt file or with a noindex tag, as their presence in the sitemap creates a technical contradiction
- Exclude pagination pages, date archives, and tag pages if they do not provide unique content to search engines
- Exclude redirected URLs (301, 302): the sitemap should point to the final destination, not the old address
- Include only pages whose content justifies indexing, such as articles, product sheets, and main category pages
A clean sitemap contains fewer URLs but sends a clearer signal to crawlers. On sites that exceed several hundred pages, this pruning makes a real difference in the speed of indexing recent content.
Submitting and verifying your sitemap: Google Search Console and robots.txt
Two complementary methods allow you to declare a sitemap to search engines. The first is through Google Search Console, where you directly submit the URL of the sitemap file. The tool then returns a status report indicating the number of discovered, indexed, or erroneous URLs.
The second method involves declaring the location of the sitemap in the robots.txt file, via a line like Sitemap: https://example.com/sitemap.xml. This declaration is read by all engines compatible with the protocol, not just Google.
The two approaches are not mutually exclusive. Combining submission via Search Console and declaration in robots.txt ensures maximum coverage. Bing Webmaster Tools offers a mechanism similar to Google’s, along with the IndexNow protocol that allows real-time notifications to engines when a page is published or modified.
- Search Console: manual submission, detailed coverage report, alerts in case of errors on the sitemap
- Robots.txt: passive declaration, automatically read by all compatible crawlers
- IndexNow: proactive notification compatible with Bing, which speeds up the consideration of updates without waiting for the next scheduled crawl
The sitemap remains a technical file whose effectiveness depends on the rigor with which it is maintained. An up-to-date, accurate, and properly submitted sitemap does not guarantee indexing, but it provides engines with the best conditions to explore a site coherently. Ignoring its maintenance means allowing crawlers to navigate without a reliable reference, which particularly penalizes large sites or those with frequently changing content.
