Research / 2026-11-09
Technical audit of US
small-business sites: 19 scanned
Key findings
- ·13 of 19 samples were reachable (4 timeouts, 1 × 403, 1 DNS failure)
- ·Mean score 73.0, median 75.0, three sites below 60
- ·sitemap coverage 13/13 — the basics are solved
- ·12 of 13 robots.txt files never mention a single AI crawler
- ·Only 7 of 13 have exactly one H1; six are missing or duplicated
Method
Sampling. Search engines were queried for service-provider sites in four industries (dental, plumbing, roofing, legal) across seven cities: Austin, Denver, Chicago, Phoenix, Dallas, Atlanta, Seattle. Directories, review platforms, social profiles and international firms were excluded, leaving 19 independently operated local business sites.
Collection. Only the homepage, /sitemap.xml and /robots.txt were requested. No JavaScript was executed; raw HTML was parsed. Requests were spaced 1.5 seconds apart with a 20-second timeout and a 3 MB per-site cap. All pages are public, the process is read-only, and nothing was written to any target.
Scoring.Six weighted criteria out of 100: body length 25 (≥800 chars full marks, ≥400 gets 15), H1 count 15 (exactly one full marks, multiple gets 8), JSON-LD 20 (any block scores full), image alt coverage 15 (≥90% full, ≥60% gets 8), sitemap 15 (HTTP 200 full marks), AI crawler policy 10 (robots.txt names at least one AI crawler and doesn't block everything).
Timing. 2026-11-09, single snapshot.
Per-site results (13 reachable)
Domain shown without the https://www. prefix.
| Industry | Domain | Score | Body | H1 | JSON-LD | alt | AI bots |
|---|---|---|---|---|---|---|---|
| Dental | austinadvanceddentistry.com | 75 | 7,285 | 0 | 1 | 100.0% | None |
| Dental | austindentalco.com | 68 | 12,464 | 2 | 2 | 48.4% | None |
| Dental | austindentalcare.com | 90 | 7,417 | 1 | 2 | 96.8% | None |
| Dental | thedenverdentists.com | 75 | 11,290 | 1 | 1 | 8.1% | None |
| Plumbing | consumersplumbing.com | 48 | 1,023 | 0 | 0 | 73.9% | None |
| Plumbing | suburbanplumbingexperts.com | 83 | 28,011 | 1 | 4 | 81.8% | None |
| Plumbing | parkerandsons.com | 100 | 4,379 | 1 | 1 | 100.0% | Named |
| Roofing | dfwroofingpro.com | 90 | 5,760 | 1 | 1 | 100.0% | None |
| Roofing | roofingsvc.com | 55 | 2,214 | 1 | 0 | 53.8% | None |
| Roofing | garoofingandrepair.com | 75 | 14,067 | 1 | 3 | 52.3% | None |
| Legal | elmlaw.com | 55 | 4,883 | 0 | 0 | 100.0% | None |
| Legal | gth-law.com | 60 | 9,582 | 0 | 2 | 20.0% | None |
| Legal | sociuslaw.com | 75 | 1,279 | 0 | 2 | 100.0% | None |
Three findings
1. The basics are solved — they no longer differentiate
All 13 reachable sites return a valid sitemap. "Do you have a sitemap" is no longer a useful question. What separates sites now is structured data, H1 discipline and AI crawler policy.
2. AI crawler policy is effectively absent
Twelve of thirteen robots.txt files never mention GPTBot, ClaudeBot, PerplexityBot or CCBot. The only one that does — parkerandsons.com — is also the only perfect score. In 2026, with ChatGPT and Copilot driving real traffic, this is not a mistake so much as an omission nobody noticed.
3. H1 structure is worse than expected
Only 7 of 13 have exactly one H1. Three sites have none at all, one has two, and several rely on styled divs or images where a heading belongs. That's not aesthetics — it's the primary signal search engines use to determine page topic.
Limitations
- ·Thirteen samples across four industries and seven cities. This does not generalise to the US small-business market.
- ·Six sites failed collection (4 timeouts, 1 × 403, 1 DNS failure). Failures may correlate with better configuration or stronger bot protection, so selection bias is possible.
- ·Homepages only. Inner-page quality, content depth, backlinks and actual rankings were not assessed.
- ·The scoring rubric is specific to this study and is not comparable to Lighthouse or PageSpeed scores.
- ·Single snapshot from 2026-11-09. Site redesigns invalidate the results.
Raw data
Per-site records, including status codes for the six failed sites, are stored as JSON with the measured values rather than scores alone. A CSV export is available on request.
How to cite
Not an industry report — a sample with a stated method and stated limits. The numbers can be re-measured, the method can be repeated, and the conclusions hold only for this sample.