Bytelify

Sitemap Finder

Enter a domain to pull every URL listed in its sitemap.xml, falling back to the Sitemap: line in robots.txt if needed.

Domain is sent to our server to fetch its sitemap — nothing else is collected

Reads the domain's own sitemap.xml (falling back to the Sitemap: line in robots.txt), including each URL's own lastmod/priority/changefreq where the sitemap declares them — not every site sets these, so "—" means the sitemap simply didn't include that field, not that the tool failed to read it. Capped at 5,000 URLs and 20 sub-sitemaps per lookup.

Need the opposite? Use What Is My User Agent

What is Sitemap Finder?

A Sitemap Finder reads a website's own sitemap.xml file and lists every URL it declares, without you having to open the file yourself and dig through raw XML. Most sites publish a sitemap specifically so search engines (and tools like this one) can discover every page without crawling the whole site link by link. This tool fetches the domain's sitemap.xml directly. If a site doesn't have one at that path, it checks robots.txt for a Sitemap: line instead, since that's the other place sites are expected to point to it. If the sitemap is actually a sitemap index (a sitemap of sitemaps, common on larger sites), it follows those links too and combines every URL into one flat list. The result is only as complete as the site's own sitemap — a site with no sitemap, or an out-of-date one, won't return a full picture of what's actually live on the domain.

How to Use the Sitemap Finder

  1. 1

    Enter a Domain: Type a domain like example.com (with or without https://) and press Find URLs.

  2. 2

    Review the List: Every URL from the site's sitemap.xml (or robots.txt fallback) appears, with a live filter box to narrow it down.

  3. 3

    Export or Copy: Use the Copy all menu to copy the list as text, CSV, or JSON, or download it as a .txt, .csv, or .json file — all based on whatever filter, sort, or category you currently have applied.

Why a Browser Can't Do This Directly

Fetching another domain's sitemap.xml or robots.txt from JavaScript running in your browser is blocked by CORS (Cross-Origin Resource Sharing) unless that specific site opts in — most don't. This tool runs the actual fetch from a server-side Cloudflare Worker instead, then hands the resulting URL list back to your browser, the same way a backend service would.

What This Tool Doesn't Do

This isn't a full site crawler — it only reads what the site's sitemap.xml (or sitemap index) and robots.txt already declare. Pages that exist but aren't listed in the sitemap won't show up here. Lookups are also capped (up to 5,000 URLs and 20 sub-sitemaps per domain) to keep each request fast and bounded, so an unusually large site's sitemap may be returned partially rather than in full.

FAQs

Frequently Asked Questions

Common questions about Sitemap Finder.

The tool checks robots.txt for a Sitemap: line as a fallback. If neither exists, it reports that no sitemap was found rather than guessing at URLs.

No. It only reads the URLs the site's own sitemap.xml (or sitemap index) declares, plus following the robots.txt fallback. It does not follow links on the page itself.

Yes — up to 5,000 URLs and 20 sub-sitemaps per lookup, to keep each request fast and bounded. Very large sites may return a partial, capped list rather than every URL.

You can try any domain, but results depend entirely on what that site publishes — some sites block automated requests to their sitemap or robots.txt, in which case the lookup will fail for that domain.

The domain you enter is sent to Bytelify's own server-side lookup function so it can fetch that site's public sitemap — nothing else about you is collected or stored. No account or sign-up is required.

It depends which field. Google has stated it ignores <priority> and <changefreq> entirely, so missing those has no effect on Google indexing. <lastmod> is different — Google does use it as a freshness signal, but only when it's present and verifiably accurate (matches the page's real last significant update). A missing or untrustworthy lastmod doesn't get a page penalized, it just means that page isn't giving Google that re-crawl signal at all. This tool flags missing lastmod coverage for that reason, and shows the priority/changefreq gaps for completeness without implying they carry the same weight.

Yes. Bytelify's Sitemap Finder is free with no account or sign-up required.