The fast growth of AI search has led to new tools that promise to track brand visibility in ChatGPT, Perplexity, Gemini, and other large language model (LLM) platforms. These tools are being marketed as the next step in search engine optimization, often under the term “AI Engine Optimization” or AEO.

A recent discussion on LinkedIn led by Ahrefs CMO Tim Soulo, referencing Nathan Gotch’s YouTube video “AI Search Rank Trackers Are Lying to You,” sheds light on what is fact versus hype in this AI chatbot space. Many of these products are creating misconceptions about how much visibility data they can really provide.

But what are AI search rank trackers not saying? Suolo provides answers to this.

No company has access to real user prompts

Soulo explains that one of the most important facts is that no tool has direct access to the actual prompts people type into ChatGPT, Perplexity, Gemini, or other LLMs. These companies do not share their user prompt data.

According to him, marketers can only use “clickstream data” to get some insight into what people are searching for, but even the largest clickstream panels only capture a small slice of user behavior.

This means the visibility reports that advertisers see are not based on true user prompts. Instead, they are stitched together from assumptions. Any platform claiming to show “real” AI search visibility is likely overstating its data access.

Most tools depend on synthetic prompts

Another issue is that most AI visibility trackers use “synthetic” prompts. This means they rely on AI to generate lists of possible questions customers might ask. For example, a platform might scan a website and then create potential prompts such as “What is the best running shoe for flat feet?” But these are generated predictions, not verified customer inputs.

Why reliable AI visibility data is hard to get

 Even when tools try to simulate AI search visibility, the process is filled with challenges that make consistency difficult. According to Soulo, several factors create major inconsistencies:

  • People rarely use short keyword-style prompts. Instead, they type longer, more conversational sentences that differ from one another.
  • Personalization is turned on by default in tools like ChatGPT, meaning two users can see different answers for the same query.
  • Different AI models, such as GPT-3.5, GPT-4, or Gemini, return different results for the same input.
  • Even repeating the same prompt can produce slightly different answers, shifting brand positions in the response.
  • LLMs run internal “query fan-outs” to fetch supporting information, and these internal calls vary constantly.

What advertisers and marketers should take away

The push for AI search visibility tracking is still in its early stages. The industry does not yet have the kind of reliable data sources or standardized measurement that traditional SEO is built on. If advertisers treat these tools as definitive rankings, they risk making decisions based on flawed inputs.

The most important step right now is for advertisers and brands to look closely at how each tool collects its data and understand what its reports are really showing. Knowing where the data comes from, how prompts are generated, and what limitations exist is key before using AI visibility reports to guide ad spend or content strategies.

‍

Get Smarter at Marketing
The Keyword weekly update → your inbox
Writer, The Keyword
Nelson Oboh
HOME PAGE
Ahrefs
01
How can your services help improve my website's performance?
Our services enhance your website's performance by optimizing load times, improving user experience, and ensuring seamless functionality across all devices. We specialize in custom Webflow builds that are tailored to your specific needs, ensuring your site is both visually appealing and highly efficient.
02
What is the process for starting a project with your team?
Our process begins with an initial consultation to understand your goals and requirements. We then conduct a thorough analysis of your current setup, followed by a collaborative planning phase where we outline the project scope and timeline. Once the plan is approved, our team gets to work on designing and developing your custom Webflow solution.
03
How do you ensure the security of the websites you develop?
We ensure the security of the websites we develop by implementing robust security protocols, including SSL encryption, regular security audits, and secure coding practices. Our team stays updated with the latest security trends and threats to proactively protect your site from vulnerabilities.
04
What support do you offer after a project is completed?
After a project is completed, we offer comprehensive support to ensure your website continues to perform optimally. This includes regular maintenance checks, updates to keep your site secure and up-to-date, and troubleshooting any issues that may arise. Our team is always available to provide guidance and assistance as needed.
05
What is your approach to handling website redesigns?
Our approach to website redesigns involves a comprehensive analysis of your current site to identify areas for improvement. We prioritize user experience and performance, ensuring that the redesign aligns with your brand's goals and enhances functionality. By leveraging the capabilities of Webflow, we create visually stunning and efficient websites tailored to your specific needs.