Source: https://dealerclaw.dev/insights/dealership-ai-access-study-2026

# 2026 Dealership AI Access Study: llms.txt, robots.txt and Inventory

An observational study of AI access and inventory data across 500 dealership websites.

DealerClaw Research Published September 8, 2026 Observed September 7, 2026

In this study

1.  [What we found at llms.txt](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#what-we-found-at-llms-txt)
2.  [Most retrieved files were lists of links](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#most-retrieved-files-were-lists-of-links)
3.  [Structured feeds supplied facts, with important gaps](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#structured-feeds-supplied-facts-with-important-gaps)
4.  [Price fields need labels and conditions](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#price-fields-need-labels-and-conditions)
5.  [Store, transit and inventory-date fields answer different questions](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#store-transit-and-inventory-date-fields-answer-different-questions)
6.  [A readable format still needs inspection](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#a-readable-format-still-needs-inspection)
7.  [robots.txt needs a bot and a specific URL](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#robots-txt-needs-a-bot-and-a-specific-url)
8.  [Five checks to request on your own inventory](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#five-checks-to-request-on-your-own-inventory)
9.  [Methodology and publication limits](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#methodology-and-publication-limits)
10.  [Data and citation](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/#data-and-citation)

DealerClaw inspected **500 dealership website domains** and retrieved a text index or guide at `/llms.txt` on **261: 52.2% of the sample**. We then followed selected links to examine what an agent could actually receive: inventory lists, structured vehicle records, ordinary vehicle pages and alternative text formats.

Those destinations produced the more useful findings. Inventory feeds supplied detailed vehicle facts, but reading one response did not always cover the available inventory. Prices carried different labels across representations. Incoming vehicles could retain a generic in-stock assertion. A website's identity did not necessarily identify the store named in every vehicle record.

For dealership owners, general managers and marketing leaders, these are practical checks to request when evaluating an agent or website provider. A file that points toward inventory is an entry point. The information reached through it still needs inspection.

This public report contains aggregate results and generalized examples. Dealership, website-provider and vehicle identities are withheld. [Download aggregate results as CSV](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/aggregate-results.csv) or [JSON](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/aggregate-results.json) .

## What we found at llms.txt

Table 1
| Outcome at `/llms.txt` | Domains | Share of 500 |
| --- | --- | --- |
| Text index or guide retrieved | 261 | 52.2% |
| HTTP response indicating the file was missing | 130 | 26.0% |
| Request deferred after robots.txt was unavailable | 106 | 21.2% |
| Successful response with an empty body | 1 | 0.2% |
| HTML returned at the text-file path | 2 | 0.4% |

![Results at llms.txt: 261 text indexes or guides, 130 missing, 106 deferred, one empty and two HTML responses across 500 sampled domains.](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/llms-results.svg)

All 500 domains remain in the denominator. The **106 deferred requests are unknown outcomes**, not missing files. Our collector conservatively deferred further requests when it could not obtain usable robots guidance because of access, server or transport problems.

A successful HTTP response did not automatically count as a text guide. We separately recorded empty bodies and HTML returned at the expected text-file path.

The 52.2% figure describes retrieved text resources in this sample. It does not measure the share of American dealerships using AI, complying with the llms.txt proposal or appearing in AI answers.

## Most retrieved files were lists of links

Of the 261 text resources, **180—69.0%—were unsectioned link-and-description lists**, without a Markdown H1 or section headings. Typical destinations included inventory, vehicle details, model information and service pages. Other resources included sectioned guides and explicit instructions for AI systems.

The [llms.txt proposal](https://llmstxt.org/) describes a Markdown guide with a required top-level title; its other sections are optional. It provides conventions for useful context, links and Markdown alternatives to conventional pages. Our count of retrieved text resources is broader than strict conformance to that proposal.

A list can help an agent discover a page. Its presence does not establish that the destination supplies vehicle facts, loads without a browser, remains current or is used by a commercial AI service. Shared templates also produced similar files across websites, so these observations should not be treated as independent decisions by every store.

## Structured feeds supplied facts, with important gaps

Twenty-one domains with explicit AI instructions and structured-inventory links supplied **24 selected JSON responses**. JSON organizes information into named fields that software can read directly.

Twenty-one responses contained vehicle records. Three responses had empty vehicle arrays and declared a total of zero for the selected inventory categories. Those were empty results for the selected category; they did not establish that the dealerships had no other inventory.

Across the 24 responses, we counted **887 vehicle-record rows**, containing **887 distinct VIN strings**. Every row supplied a nonempty VIN, stock number, year, make, model, dealer identity/location and vehicle-page URL. A separate recount agreed with the original extraction for every response.

The collection followed 18 sampled vehicle links from these feeds. All 18 returned HTML whose initial body text contained the matching VIN. This verified those sampled connections, not every link or the accuracy of every vehicle fact. Distinct VIN strings also do not establish VIN authenticity or complete inventory coverage.

All 887 rows had `imageUrls: null`, and the inspected vehicle objects lacked an explicit currency key. These were omissions in the feed representation; they did not mean the ordinary vehicle pages lacked photographs or currency information.

All 24 JSON responses also carried `X-Robots-Tag: noindex,nofollow`. Their exact feed URLs were permitted by the applicable robots rules for the bot names evaluated. Fetch permission and response indexing directives are separate observations. Neither proves how every AI service will use the content.

### The first response was not always the full inventory

**Fourteen of the 24 JSON responses declared another page.** A separate pagination check retrieved additional VINs that were absent from the first response.

An agent should distinguish records it actually read from the total claimed by the publisher. Stopping at the first successful response can leave part of the declared inventory uninspected. A working feed URL therefore does not, by itself, establish complete coverage for a stock comparison or campaign selection.

## Price fields need labels and conditions

The feeds separated `msrp`, `sellingPrice`, `dealerDiscount` and `cashRebates`. These are publisher-defined fields. Their names alone do not establish an unconditional transaction price or tell an agent to subtract rebates again.

Every one of the **887 records had `dealerDiscount: 0`**, yet **463 had different values in `msrp` and `sellingPrice`**. Of the 112 records with positive `cashRebates`, 51 still had equal MSRP and selling-price values. These comparisons describe the recorded fields; they do not assign every difference to a discount, fee or available offer.

Three rows contained a zero selling-price value. Follow-up checks found values that changed, persisted or differed from the ordinary vehicle page. We did not establish the cause of those zeros, and they did not establish free-car offers.

### The ordinary vehicle page could add facts and disagree

Each of the 18 matched vehicle pages contained a VIN-bound structured vehicle object with USD currency, an image URL and an `InStock` assertion. These supplied information absent from the feeds. An image URL is evidence of a reference, not a verified photograph; a stock assertion does not establish immediate delivery.

In **16 of the 18 pairs**, the vehicle page's structured `Offer.price` differed numerically from the feed's `sellingPrice`, despite source captures less than three seconds apart. Five of those structured prices matched the feed's separate MSRP field. Another difference reconciled with a disclosed fee.

This is **not a count of 16 incorrect offers**. Different price roles and fee treatment explained some differences; others remained unresolved. The audit kept those distinctions and later browser observations separate.

Mileage differed in four pairs: the feed reported a small nonzero value while the structured vehicle object reported zero. These were representation differences, not independent odometer checks.

Displayed finance and lease disclosures added another consideration: their amounts and eligibility conditions could differ. An agent answering a price question needs to preserve the label, payment context and applicable terms with the amount it reports.

## Store, transit and inventory-date fields answer different questions

Two captured feeds contained records naming multiple dealerships, while the website-domain field consistently identified the host website. Shared inventory may be intentional. An agent preparing a location-specific report should read the record's store information rather than assign every VIN to the website's store. Neither field independently verifies where a vehicle is parked.

For incoming inventory, we froze the complete **22-record cohort marked `inTransit: true`** before a separate follow-up. All 22 remained in the refreshed feed responses with that flag. All 22 matching vehicle pages contained a VIN-bound structured offer asserting `InStock`.

Two pages selected before those requests also received browser checks. Both visibly indicated that the vehicle was in transit while retaining `InStock` in their structured data. These were two displayed-page checks, not rendered verification of every cohort member.

Twenty-one of the 22 transit-flagged records had a non-null `dateInStock`. A checked page could show an arrival estimate still ahead while its feed carried an earlier inventory date. Subtracting an inventory date from the inspection date does not establish time physically spent on the lot.

[Schema.org defines InStock as an in-stock assertion](https://schema.org/InStock) . In this cohort it coexisted with more specific incoming-vehicle information. These website observations did not verify physical stock, actual arrival or delivery readiness.

Some records categorized as new also carried a loaner flag. Those publisher labels did not verify title status, prior use or warranty commencement. They did show why selecting records by the word “new” alone can omit a relevant distinction already present in the data.

## A readable format still needs inspection

Selected resources explicitly presented for AI returned Markdown inventory and vehicle information. Matched Markdown and ordinary HTML representations could agree on vehicle identity and total price while describing a price component differently. Media information also varied in completeness. Agreement on a total did not establish agreement on its components.

Other ordinary website routes supplied HTML or Markdown under different Accept-header requests using the same research-client identity. This demonstrated alternative representations under our test conditions, not what every commercial crawler receives. Some Markdown responses also required additional parsing to recover their embedded structured content.

Browser checks showed a related distinction: initial body text, structured data embedded in HTML and the rendered page could expose different inventory information. Missing vehicle cards in a text extraction did not establish that the response lacked machine-readable records.

An agent's retrieval and parsing capabilities therefore matter alongside the publisher's chosen format. JSON, Markdown and an AI-oriented label are useful clues; none independently verifies accuracy or completeness.

## robots.txt needs a bot and a specific URL

We retrieved **385 parseable robots.txt files** across the 500 domains. This measures readable directives, not sites allowing AI access.

Table 2
| Applicable rule group | GPTBot | OAI-SearchBot |
| --- | --- | --- |
| Matching named group | 203 | 221 |
| Wildcard group | 182 | 164 |
| Total parseable files | 385 | 385 |

These are counts of applicable groups, not permitted or blocked websites. A named group can produce the same permission outcome as a wildcard group.

We evaluated the final destinations, including query strings, of **725 selected inventory-purpose nominations**. Saved origin policies permitted both bots in **694 cases**, excluded both in **three**, and were unassessable in **28**. There was no permission difference between the two agents within the 697 assessable cases.

Published permission did not guarantee retrieval. Twenty-one of the 694 permitted destinations returned HTTP 403 to our research client, whose own requests were also permitted by the saved rules. This records what happened to our client; it does not establish what a commercial crawler received or why the server rejected the request.

The [Robots Exclusion Protocol](https://datatracker.ietf.org/doc/html/rfc9309) describes agent and path matching. It is not access security. OpenAI also documents different purposes for GPTBot, OAI-SearchBot and ChatGPT-User: potential model-training collection, search and certain user-triggered visits respectively. [Read the crawler documentation](https://developers.openai.com/api/docs/bots) .

An access review should retain the agent, requested URL, applicable rule and observed response. “AI allowed” is too broad to capture those distinctions.

## Five checks to request on your own inventory

1.  **Follow a vehicle from guide to detail page.** Identify the facts available in text, embedded structured data and the rendered page.
2.  **Check inventory coverage and store scope.** Follow pagination and distinguish inspected records, declared totals and each record's named store.
3.  **Inspect an incoming vehicle.** Explain the transit flag, arrival estimate and inventory-date definition before producing an on-lot or aged-stock report.
4.  **Reconcile a price.** Preserve MSRP, selling-price labels, fees, rebates and eligibility conditions. Identify anything the evidence cannot settle.
5.  **Test the intended access path.** Keep published crawler rules, indexing directives and observed retrieval outcomes separate.

## Methodology and publication limits

This was a directory-derived, stratified convenience sample of **500 distinct requested domains**, with 125 source-associated locations in each Census region and coverage of 40 states. It is not nationally representative or a 500-rooftop census. Multi-location businesses count once by domain.

The final cohort included 458 selections from an existing directory/group-roster frame, two same-region reserve replacements and 40 purposefully selected used-vehicle retailers. The replacements addressed observed placeholder websites; that eligibility amendment occurred during collection. Missing files, access blocks and network failures remained outcomes. We do not report national margins of error.

Across 390 domains, the protocol selected **2,304 linked URLs**. The other 110 had no selected destinations. Each domain had a ceiling of six initial linked resources; up to three selected HTML resources received paired browser observations. Destination selection prioritized relevant resources and was not random.

Separate purposive follow-ups examined saved and freshly captured representation pairs, pagination, zero prices and the complete incoming-vehicle cohort. They did not enlarge the 500-domain sample or change its baseline percentages. Source and later browser observations remained separate.

Requests identified the client as `DealerClawResearchBot` and followed its recorded robots permissions. Browser visits used bounded waits and scrolling, with some requests and frames restricted. Those restrictions limit negative conclusions about content that did not appear. Automated extraction and AI-assisted review supported the study; there was no independent human review panel.

DealerClaw retains the identified responses and audit trail privately. This aggregate-only release withholds dealership, provider and vehicle identifiers and generalizes identifying examples. Its downloads support checks of reported calculations, **not independent re-crawling or re-extraction of the sample**. Generalization reduces traceability but cannot guarantee that every observation is unrecognizable.

This study did not measure AI rankings, citations, traffic, crawler compliance or actual provider-bot visits. It does not establish that adding llms.txt improves any of those outcomes. Google's [AI search guidance](https://developers.google.com/search/docs/appearance/ai-features) likewise describes no special AI-file requirement beyond the relevant Search requirements and does not guarantee inclusion.

## Data and citation

[Aggregate results: CSV](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/aggregate-results.csv) · [Aggregate results: JSON](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/aggregate-results.json)

**Suggested citation:** DealerClaw Research. (2026, September 8). 2026 Dealership AI Access Study: llms.txt, robots.txt and Inventory. Observations collected September 7, 2026. [Public study](https://dealerclaw.dev/insights/dealership-ai-access-study-2026) .

## Data downloads

[Aggregate results (CSV)](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/aggregate-results.csv) · [Aggregate results (JSON)](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/aggregate-results.json) · [Article and methodology (Markdown)](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/study.md) · [Citation (BibTeX)](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/citation.bib) · [Check aggregate calculations (Python)](https://dealerclaw.dev/insights/dealership-ai-access-study-2026/assets/check_aggregates.py)

These files contain aggregate observations only. The identified study evidence is not distributed.