Research pre-released by IQRush this week shows that AI visibility rankings remain unstable until platforms receive between 33 and 94 repeat queries on the same topic, with three out of 30 platform-topic combinations never achieving stable rankings even after 125 questions, according to a paper detailed by Search Engine Journal on July 11.
TL;DR: AI visibility tracking dashboards showing single-sample rankings cannot distinguish genuine competitive advantages from statistical noise until repeat measurements confirm the order stabilizes and top-ranked sites separate beyond margin of error.
The paper, written by Ron Sielinski, co-founder of IQRush, establishes that no fixed sample size works across all platforms and topics. The research examined 30 platform-topic combinations and found the number of citation-bearing answers needed for reliable rankings ranged from 33 to 94. Three tests on SearchGPT never reached stability within the 125-question budget because top sites remained too close to separate statistically.
IQRush sells software that measures AI visibility using the repeated-sampling method the paper advocates. A separate team published similar findings in April, indicating the instability problem extends beyond a single vendor’s observations.
Single Samples Mask Overlapping Confidence Intervals
Generative AI platforms introduce randomness into each response, causing the same query to cite different sources on successive runs. A prior paper by Sielinski showed Tom’s Guide captured 9.5 percent of SearchGPT citations for running-gear queries while Runner’s World captured 6.0 percent. The 3.5-point lead appeared meaningful on a dashboard, but overlapping margins of error meant the difference could have been statistical noise rather than genuine competitive advantage.
The new research addresses how many samples are required before rankings reflect real performance differences rather than measurement fluctuation. The answer depends on two conditions that must both hold: the ranking order must stop changing as new samples accumulate, and the gap between top-ranked sites must exceed the margin of error for each site’s citation share.
Platform Architecture Determines Sample Requirements
The platform being measured changes how much independent information each answer provides. Gemini concentrates multiple citations on the same handful of domains within a single response, reducing the independent information content of each answer. SearchGPT distributes fewer citations per answer but spreads them across more domains, making each answer carry more statistical weight.
A 50-answer sample on Gemini and a 50-answer sample on SearchGPT do not deliver equivalent confidence levels. Rankings that stabilize on Gemini within a given sample budget may remain unresolved on SearchGPT at the same sample size.
The research found SearchGPT accounted for all three tests that failed to stabilize even after 125 questions. In those cases, top-ranked sites remained statistically indistinguishable throughout the measurement period.
Before-and-After Measurements Require Multiple Samples
A three-point increase in citation share after publishing new content can fall within normal run-to-run variation, according to the data presented in Sielinski’s earlier work. Single before-and-after readings cannot separate a genuine content effect from ordinary measurement noise.
To claim a content change drove a visibility increase, the research indicates marketers need multiple pre-change samples and multiple post-change samples. A dashboard showing a clean percentage gain after one measurement in each period cannot support the conclusion that the content change caused the shift.
Rand Fishkin, who led a January 2026 SparkToro study showing AI tools deliver different brand recommendations on more than 99 percent of repeat queries, said in the Search Engine Journal article that businesses should verify their tracking provider “shows their math.” The IQRush paper provides a stopping rule—wait until both stability conditions hold—that removes reliance on intuition about adequate sample sizes.
Australian businesses evaluating AI visibility tracking frameworks face the same measurement challenge documented in the research: single-sample dashboards can show rankings that look definitive but remain within the noise band. The study’s findings align with broader patterns in SEO measurement practices where variance in the underlying data requires multiple observations before trends become actionable.
Trackers That Report “Not Enough Data” Signal Better Methodology
A tracker that occasionally reports insufficient data demonstrates more rigorous methodology than one that produces confident rankings on demand, according to the research. Three of the 30 tests examined in the paper could not cleanly separate top sites within the measurement budget. The appropriate output in those cases is to withhold a ranking rather than publish an order the data cannot statistically defend.
Dashboards showing clean percentages without confidence intervals or sample counts may signal single-sample measurements. The IQRush paper argues that citations shares presented as fixed numbers are “merely snapshots of a continuously changing target, not fixed facts.”
The research indicates the top portion of a ranking table becomes defensible first. With sufficient samples, leading sites pull away from the field and establish separations larger than their individual margins of error. Mid-table and lower-ranked positions require more data to resolve because closer citation shares demand narrower confidence intervals to separate.
What Happens Next
Australian SMEs paying for AI visibility tracking should request from their providers the number of samples collected per query, the confidence intervals around each citation share, and evidence that rankings stabilize before being reported. Single-sample dashboards deliver numbers that may change substantially on the next measurement, making month-over-month comparisons unreliable.
Content teams interpreting visibility changes after publishing new material should establish pre-change baselines with multiple samples and measure post-change performance the same way. A visibility increase documented with one sample before and one after cannot distinguish a content effect from ordinary platform variability. The research shows that declaring success from a single before-and-after reading risks attributing random fluctuation to strategy.
The findings suggest budgets for AI visibility tracking should account for platform-specific sample requirements. A monitoring plan adequate for Gemini may underfund SearchGPT measurements, leaving rankings unresolved even as the dashboard prints confident numbers. Trackers that acknowledge when data remains insufficient provide more actionable intelligence than those producing rankings regardless of statistical support.