SEO, sitemaps and product feeds

What every store publishes for search engines and ad platforms: titles, structured data, robots.txt, sitemaps that split themselves for big catalogs, and Google and Meta product feeds that stream from any catalog size.

Every store publishes what search engines and ad platforms need, on its own address and on a custom domain, with nothing to set up. Storefront → Domain & SEO holds the few settings there are and every link below.

On every page

• Titles and descriptions from the product, collection or page, with your home page title and description (Domain & SEO) as the defaults. The social image you set there is shared when the store is linked.
• Structured data: product pages carry Product JSON-LD with one offer per option (so 1 Day, 7 Days, 30 Days and Lifetime each have their own price and availability), plus reviews and ratings when you have verified reviews.
• Canonical URLs on your primary domain. A product whose handle changed answers its old URL with a 301 to the new one, so links and rankings carry over.
• Unlisted products are left out of collections, search, sitemaps and feeds, and say noindex.

robots.txt

/robots.txt points crawlers at the sitemap. Two switches in Domain & SEO change it:

• Let search engines index your store off: every page says noindex and robots.txt disallows everything (for a store that is not ready yet). A store behind its password is never indexed either.
• Ask AI crawlers to stay away: robots.txt asks AI training crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended and others) not to read the store. Regular search is not affected.

Sitemaps for any catalog size

/sitemap.xml is a sitemap index. It lists child sitemaps for products, collections and pages:

• Products are split across files by product id, so each file stays well under the protocol’s limits (at most 25,000 URLs or 25 MB when the index is built, never past 50,000 URLs or 50 MB). A 200,000-product store gets a few dozen files; a small store gets one.
• Collections and pages have their own files (sitemap-collections.xml, sitemap-pages.xml).
• A product change refreshes only the file that product is in, plus the index, so caches stay warm for the rest.
• Every file has an ETag. Crawlers that send If-None-Match get a 304 Not Modified without ShopFast reading a single product.

Submit https://<your domain>/sitemap.xml in Google Search Console once. Nothing else is needed as the catalog grows.

Product feeds

Feeds hold every listed product on sale that the platforms accept, one item each, with its price (and the sale price during a promotion), availability, image and your store’s currency. They are streamed a page at a time, so the first bytes arrive at once and a feed of a six-figure catalog works the same way as a small one. Like sitemaps, they carry an ETag and are cached for an hour at the edge.

Domain & SEO → Product feeds lists each product with whether it is in the feeds and what keeps it out (a free product, an unlisted one, a product type the platforms do not accept), so you can fix issues before the platforms report them. The same report is an API call:

curl https://api.shopfa.st/v1/store/storefront/feeds -H "Authorization: Bearer $SHOPFAST_KEY"

It answers with the three URLs (googleUrl, metaUrl, sitemapUrl) and, per product, listed, the reason when it is left out and any issues.