Measured Twice

The typical dealer website in this panel changed its mobile Largest Contentful Paint by 71 milliseconds between its first and second measurement. The 95% confidence interval on that change runs from minus 1 to plus 152 milliseconds.

That interval is the whole finding. It barely includes zero and it is asymmetric. Improvement beyond about a millisecond at the median sits outside it, and worsening is bounded at about a seventh of a second. The reading is that this layer did not get faster, may have slipped slightly, and did not move far in either direction.

Every figure in sections 2 through 5 is produced by analysis/panel_study.py in the Showroom Standard repository, which is read-only against the audit tables and re-runnable. The cohort and panel-composition figures in section 1, and the boundary-distance counts in section 3, are direct read-only queries against the same tables and are not in the script yet. They are named here rather than left inside a guarantee that does not cover them.

1. What this panel is, and what it is not

The verified dealer cohort holds 7,700+ verified dealer sites, assembled from search discovery and from one manufacturer’s dealer-locator list. 140 of the 1,091 panel sites come from the locator list or from a competitor-tracking set rather than from search. The cohort is measured across the flooring, tile, cabinet, and countertop trades, and 98 of the 1,091 panel sites carry industry labels outside those four, mostly kitchen and bath. 1,332 of them carry two or more usable mobile audits. 1,091 of those have a first audit before June and a last audit in July or later, which is the condition for forming a before-and-after pair. In this corpus the later audits all fall in July or August. Those 1,091 are the panel. The condition excludes 241 sites, 18% of the sites with repeat audits, whose measurements sit on one side of the period.

“Usable” means the audit recorded a mobile LCP value, and no validity screen was applied beyond that. Baseline values in the panel run from 787ms to 322 seconds, and 12% of sites moved by more than 10 seconds between measurements. No screen was added, because choosing one after examining the data is the defect this paper is otherwise built to avoid. The median is reported throughout because it is insensitive to those values.

139 panel sites carry three or more usable audits. For those the first and last are used and every intermediate measurement is discarded, so “measured twice” describes the analysis rather than the audit history.

First measurements span 2026-01-17 to 2026-05-12 and last measurements span 2026-07-02 to 2026-08-02. The gap between a site’s two measurements has a median of 149 days, a minimum of 52, and a maximum of 196. The panel’s median mobile LCP is 8.77s at the first measurement and 9.11s at the second. Those are population medians of two different measurement rounds, and they are not the same quantity as the within-site median change in section 2, which is what this paper bounds.

This is a convenience panel drawn from an unrandomized re-audit history, with no pre-specification. No site was selected for re-audit by any rule this analysis can reconstruct, the records do not show what drove the selection, and no hypothesis, metric, or test was fixed before the data was examined. That is the central limitation and it is stated first because everything below inherits it.

Showroom Standard measures websites. It does not measure rankings, impressions, clicks, visibility, traffic, or algorithmic impact, and it holds no Google Search Console data for any site in the corpus. Nothing here describes what Google did to these businesses.

Every measurement here is a lab measurement. Each figure is a single PageSpeed Insights run under simulated mobile throttling, read from the Lighthouse result. Google’s Core Web Vitals assessment is computed on the 75th percentile of real-user field data over 28 days, which this instrument does not collect. Where this paper compares a site against Google’s 2.5 second threshold, it is comparing a lab measurement against a published number, and it is not reporting a Google verdict on that site.

2. The bound

The interval is an exact distribution-free confidence interval taken from the order statistics of the 1,091 within-site changes, at x(513) and x(579). Its nominal coverage under the exact binomial is 0.954, and the true coverage is a little higher and not exactly known, because nine sites changed by exactly zero and the values are rounded. It involves no resampling and no random seed, so it reproduces exactly on every run. The interval uses all 1,091 changes; the sign test below drops the nine ties and runs on 1,082.

Sign tests here are exact two-sided binomial tests with ties dropped. An earlier version of this analysis used a normal approximation, which is anti-conservative at this scale.

3. The share barely moved and the membership did not hold

Measured in a standardized lab test against Google’s 2.5 second threshold, the share of panel sites meeting it moved from 2.66% to 2.93%, on 22 sites crossing from pass to fail and 25 crossing the other way, at McNemar p = 0.77, using the continuity-corrected form. An exact binomial on the same 47 discordant pairs gives 0.771.

Underneath that quarter-point move, 29 sites met the threshold at the first measurement and 22 of those 29 did not at the second, a 76% turnover, while 25 different sites crossed in.

Against the instrument’s own noise, the crossing cannot be separated from it. Cohort sites re-audited twice inside a single month cross this threshold 3.1% of the time with no elapsed time at all, on a Wilson interval of [1.7%, 5.8%] resting on nine crossing events across 288 sites. The panel’s crossing rate is 4.3%, which sits inside that interval. The two cannot be told apart, and no ratio between them is stated here. So the rate at which sites crossed is not separable from what this instrument does on a site that has not changed. What the floor does not account for is the size of the moves behind those crossings, which the next paragraph gives.

Both facts are direct counts on the panel. The 29 sites meeting the threshold at baseline are not clustered at its edge: sixteen of them are below 2,000ms and four below 1,300ms. The 22 that stopped meeting it did not drift marginally past 2,500ms either; fifteen landed above 5,000ms and eight above 10,000ms. So the turnover is not a boundary effect, and this analysis does not explain what produced it. What the counts show is that the share meeting the threshold barely moved while most of the specific sites behind it changed, and nobody should read a site meeting this threshold once as a durable property of that site.

4. The environment

This section is context. It is not evidence of causation, and no test here attributes any measured value to any event in it. The events below are sourced under the registry’s Rule 11, from Google’s Search Status Dashboard, and the committed script verifies measurements only. It does not verify these dates and does not attempt to.

Google confirmed, from the Search Status Dashboard: a Discover update beginning 2026-02-05, a spam update beginning 2026-03-24, a core update beginning 2026-03-27, a core update beginning 2026-05-21, a spam update beginning 2026-06-24, and a spam update beginning 2026-08-18. Five of those six are ranking events; the February entry is a Discover update.

Those events do not fall inside every site’s measurement gap. Because first measurements span nearly four months, each site has its own window, and a site’s gap contains between two and five of the six events, with a median of four. The August spam update began after every last measurement in the panel and falls inside no site’s gap at all.

Google documented, and did not change, the Core Web Vitals thresholds during this period, including the 2.5 second boundary section 3 compares against. That matters for section 3 and not for the bound in section 2, which is a change in milliseconds and depends on no threshold at all.

5. What else was examined

Four metrics were tested on this panel and the panel was split by audit target into 838 sites whose audit targets a homepage and 253 whose audit targets an interior page. Mobile Cumulative Layout Shift produced a within-site median change of 0.0000 with a zero-width interval and moved only as a crossing result, desktop LCP moved slightly, and the interior-page split moved substantially. Mobile performance score is bounded at [0, +1] on the full panel, [0, 0] on the homepage split and [-1, +2] on the interior split, at sign test p = 0.55, 0.91 and 0.34 respectively. None of them carries a finding here, because each was found by examining several cuts of one panel with no hypothesis fixed in advance, which is a procedure that produces movement whether or not any exists. The full output is in the committed script.

Desktop LCP is the useful illustration of why that matters. On the 838-site homepage split it shows a median improvement of 35 milliseconds with an interval of [-61, -1] that excludes zero, at an exact sign test of p = 0.020. On the full panel the same metric gives a median improvement of 21 milliseconds with an interval of [-50, +2] that includes zero, at p = 0.095. The sign and the magnitude survive; the p-value moves from under 0.05 to well above it. A result that clears an uncorrected threshold under one post-hoc restriction of the panel does not clear it under the panel itself, and the split rule behind that restriction is a test on the audited URL’s path. That is the reason this paper reports one number and keeps its methods short.

6. What this does not establish

It does not establish that Google’s updates had no effect on these businesses. It measures page delivery and nothing else.

It does not establish that dealer websites do not change. It bounds what this panel did across this window.

It attempts no event-level attribution, and the reason is not only that the gaps are heterogeneous. Core and spam updates change ranking systems. They have no mechanism by which they would change how a dealer’s server or content delivery network renders a page, which is what this paper measures. The timeline in section 4 establishes when the measurements were taken relative to a changing search environment. It is not a set of candidate causes.

It bounds the median. It does not bound the mean. A change concentrated in a small number of sites making large moves would not be detected.

It says nothing about any individual site. The measurement is noisy per site and precise only across hundreds of them.

It measures a cohort discovered through search, in four trades, in one country, and nothing here generalizes to the web at large.

Most of all, it is a pilot. The limitation in section 1 is not repairable by analysis, because the selection has already happened and the data has already been examined. This one is the measurement that made pre-specification a rule here.

This panel is Pilot 001. The specification for the next one is written and dated ahead of any collection, as prespec/WAVE-2-PRESPEC.md in the Showroom Standard repository, and it fixes the re-audit cadence, the primary metric, the test, the noise floor, the crossing floor and the permitted subgroup cuts. Whether that holds is checkable against that file rather than against this paper.