Domain explorer

Look up a single domain's robots.txt & llms.txt metrics, effective AI policy, inferred tech stack (labelled hypothesis), and crawl history. Read-only; one indexed lookup per query.

Try: openai.com nytimes.com github.com sheetz.com

lidl.com

Tranco rank

6,485

popularity rank

TLD

com

lidl.com

robots.txt records

across crawls

llms.txt records

across crawls

Live files on lidl.com

Open the file as it exists on the host right now — may differ from what we measured. We link out rather than mirror (we store derived metrics, not raw bodies).

robots.txt ↗ llms.txt ↗ llms-full.txt ↗ ai.txt ↗

Captured samples

Snapshots we captured for the showcase set (may differ from the live file above). Large files are truncated; the true size is noted.

/llms.txt · 5,804 bytes

<!doctype html><html lang="en-US"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width,initial-scale=1,viewport-fit=cover"/><title>Grocery Store | Low Prices | Lidl US</title><meta name="facebook-domain-verification" content="bd5srwxcisslwsqu6uzcr8tozddm6e"/><meta name="apple-itunes-app" content="app-id=1220932607"/><meta name="google-site-verification" content="DbAqZ-HZf5OhO1JrJLWoGwn08kYEwti6GbFLaLl4W1s"/><link rel="manifest" href="/assets/manifest.json"/><link rel="icon" href="/assets/favicon.ico"/><link rel="preload" href="/assets/fonts/LidlFontPro/LidlFontPro-SemiBold.woff2" as="font" type="font/woff" crossorigin/><link rel="preload" href="/assets/fonts/LidlFontPro/LidlFontPro-Regular.woff2" as="font" type="font/woff" crossorigin/><link rel="preload" href="/assets/fonts/LidlFontCondPro/LidlFontCondPro-Bold.woff2" as="font" type="font/woff" crossorigin/><script src="https://cdn.cookielaw.org/scripttemplates/otSDKStub.js" charset="UTF-8" data-domain-script="6dd4be68-ee21-4a16-8add-02a2616a1e11"></script><script>// Reload page after cookie consent change (https://community.cookiepro.com/s/article/UUID-534defd9-7ec8-fc0e-78bf-8dce8c6c5593?language=en_US&topicId=0TO1Q000000bChXWAU)
            function getCookie(name) {
                const value = '; ' + document.cookie
                const parts = value.split('; ' + name + '=')
                if (parts.length == 2) {
                    return parts
                        .pop()
                        .split(';')
                        .shift()
                }
            }

            function OptanonWrapper() {
                const OABCcookieName = 'OptanonAlertBoxClosed'
                const bannerAcceptBtn = document.getElementById('onetrust-accept-btn-handler')
                const pcAllowAllBtn = document.getElementById('accept-recommended-btn-handler')
                const pcSaveBtn = document.getElementsByClassName(
                    'save-preference-btn-handler onetrust-close-btn-handler'
                )[0]
                const OABCcookie = getCookie(OABCcookieName)
                const waitAndReload = () => {
                    setTimeout(() => {
                        location.reload()
                    }, 1000) // quick timeout so that the consent receipt can be sent and the cookie can be updated
                }

                // IF logic needed here because ot-banner-sdk DIV is not injected on page loads if banner is not exposed
                if (!OABCcookie && bannerAcceptBtn) {
                    bannerAcceptBtn.addEventListener('click', waitAndReload)
                }
                if (pcAllowAllBtn) {
                    pcAllowAllBtn.addEventListener('click', waitAndReload)
                }
                if (pcSaveBtn) {
                    pcSaveBtn.addEventListener('click', waitAndReload)
                }
            }</script><script>;(function(w, d, s, l, i) {
                w[l] = w[l] || []
                w[l].push({
                    'gtm.start': new Date().getTime(),
                    event: 'gtm.js',
                })
                var f = d.getElementsByTagName(s)[0],
                    j = d.createElement(s),
                    dl = l != 'dataLayer' ? '&l=' + l : ''
                j.async = true
                j.src =
                    'https://www.googletagmanager.com/gtm.js?id=' +
                    i +
                    dl +
                    '&gtm_auth=DaNW6qUshp6aGo0QVIW0ug&gtm_preview=env-6&gtm_cookies_win=x'
                f.parentNode.insertBefore(j, f)
            })(window, document, 'script', 'dataLayer', 'GTM-5D9NQR5')</script><script type="text/plain" class="optanon-category-C0004">;(function(w, d, t, r, u) {
                w[u] = w[u] || []
                w[u].push({
                    projectId: '10000',
                    properties: {
                        pixelId: '10141675',
                    },
                })
                var s = d.createElement(t)
                s.src = r
                s.async = true
                s.onload = s.onreadystatechange = function() {
                    var y,
                        rs = this.readyState,
                        c = w[u]
                    if (rs && rs != 'complete' && rs != 'loaded') {
                        return
                    }
                    try {
                        y = YAHOO.ywa.I13N.fireBeacon
                        w[u] = []
                        w[u].push = function(p) {
                            y([p])
                        }
                        y(c)
                    } catch (e) {}
                }
                var scr = d.getElementsByTagName(t)[0],
                    par = scr.parentNode
                par.insertBefore(s, scr)
            })(window, document, 'script', 'https://s.yimg.com/wi/ytc.js', 'dotq')</script><script src="https://assets.adobedtm.com/832a01ce1742/29d487a29220/launch-6c33e92d0b73.min.js" async></script><script type="module" crossorigin src="/static/index-BxJAslna.js"></script><link rel="stylesheet" crossorigin href="/static/index-4QQ7Sr-t.css"></head><body><noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-5D9NQR5&gtm_auth=DaNW6qUshp6aGo0QVIW0ug&gtm_preview=env-6&gtm_cookies_win=x" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript><div id="portals"></div><div id="app"></div><h1 class="visually-hidden">Grocery Store | Low Prices | Lidl US</h1><script src="https://www.google.com/recaptcha/api.js" async defer="defer"></script><script>/**
             * version: 2.80.0
             * build: 2.80.0-production-20260525.4
             * api: https://mobileapi.lidl.com
             * cms: production
             */</script></body></html>

Effective AI / crawl policy (Google posture, §3.3)

Crawl	Posture	Consec. 5xx	Reason	As of
CC-MAIN-2026-25	open	0	2xx: robots.txt present, no full disallow in effect	2026-06-12T16:03:39-07:00

robots.txt

Crawl	Status	Bytes	Groups	Allow	Disallow	Sitemaps	Wildcard *	Disallow-all	AI tokens	AI verdicts
CC-MAIN-2026-25	200	65	1	0	1	1	yes	no	—	—

llms.txt

Collected by our own polite crawler, not Common Crawl — Common Crawl doesn't capture /llms.txt. See Methodology.

Crawl	Status	Variant	Conformant	Sections	Links	Generator
CC-MAIN-2026-25	200	/llms.txt	no	0	0	—

llms.txt data reflects adoption + conformance only, never consumption — publishing a file does not mean any AI reads it.

Inferred tech stack hypothesis

No tech inference for this domain.