How do AI visibility tools measure whether ChatGPT mentions a brand?
EasyFound · · 7 min read
AI visibility tools ask ChatGPT a fixed set of customer questions, repeat each one, and count the share of answers that name a business. How many runs, which languages and, research suggests, location and engine all change that number. In EasyFound's Malaysian test through OpenAI's API, 64% of the business mentions in one answer weren't repeated in either of the other two answers. Ask any tool how it measures first.
A sales rep shows you a slide: "ChatGPT mentions your renovation company 40% of the time." It looks precise, and the monthly fee rests on it.
AI visibility tools measure ChatGPT mentions by asking it a fixed set of customer questions, repeating each one several times, and counting the share of answers that name you. The number is only as good as the questions, repeats, languages and location behind it.
In our Malaysian test, nearly two in three businesses ChatGPT named in one answer were missing from both other answers to the same question.
How do AI visibility tools measure whether ChatGPT mentions a brand, step by step?
AI visibility tools measure ChatGPT mentions in five steps, each a choice that shapes the result:
- Pick the questions customers type, such as "Can you recommend a renovation contractor in KL for a terrace house? My budget is tight."
- Send them to ChatGPT, through OpenAI's API or through the app itself.
- Repeat each question, because ChatGPT gives a different answer every time.
- Read the names each answer gives, and the pages it links to.
- Count. The mention rate is the share of answers that name you. Named in one answer out of four is a 25% mention rate.
A mention is your name in the answer; a citation is a link to your page. They are worth tracking separately.
How many runs does an AI visibility tool need before a ChatGPT mention rate means anything?
An AI visibility tool needs more than one run per question, and ideally more questions, languages and AI tools, before a ChatGPT mention rate means much; one run tells you very little. In our test, 275 of the 431 business mentions in single answers didn't come back in either of the other two runs of that question. Hotels were steadier; outside hotels, about three in four mentions didn't come back.
Even with three runs, a business named once could really be named anywhere from about 6% to 79% of the time.
Researchers disagree on how to get larger samples. SparkToro's Rand Fishkin wrote that to know an AI's set of recommendations "you need to ask over and over again; usually at least 60-100X, then average these out"1. A July 2026 preprint on AI sentiment about brands found repeats past the fifth added almost nothing. Its conclusion: "reliability is bought by spreading across languages and models, not by repeating one prompt"2.
Which measurement choices change the ChatGPT mention rate a tool reports?
Five measurement choices change the ChatGPT mention rate a tool reports, and each has a sturdier setup.
| Choice | Setup that can mislead | Setup that holds up better |
|---|---|---|
| Runs per question | One run, reported as a fact | Many runs, reported as a share with a margin |
| What is counted | Your place in one list | How often you're named across runs |
| API or app | Either, without saying which | Named clearly, with the app's limits stated |
| Location | No location set | Malaysia set, and your city where it matters |
| Language | English only | English, Malay and Chinese, reported separately |
Rank position. Of the 69 businesses named in more than one run of a question, 44 appeared at a different place in the answer. Fishkin put it bluntly: any tool that gives a "ranking position in AI" is "full of baloney"1.
API or app. A small August 2026 study found the top pick changed as often on the logged-out ChatGPT website as through OpenAI's API, calling the instability "a property of the system and not of the surface"3. That doesn't make the answers identical, so ask which one a tool uses. The app adds personal factors: OpenAI says that with Memory on, ChatGPT "may use relevant details to formulate a more useful search query"4. It may also use "an approximate location based on your IP address"5.
Language. Of the businesses named for a Malay or Chinese question, 178 of 242 never appeared in the three English answers to the same question. Part of that is ordinary run-to-run change: answers in different languages overlapped only a little less than repeat runs in one language. Either way, three English runs didn't show them.
What should I ask before I buy an AI visibility tool in Malaysia?
Before you buy an AI visibility tool in Malaysia, ask the vendor to answer these in writing:
- Which AI tools do you check? A ChatGPT number doesn't cover Google's AI or Perplexity.
- Do you use the API or the app? If the API, does it set the location to Malaysia?
- Can I see and edit the questions? They should read like your customers, for example "Syarikat solar mana yang bagus untuk rumah teres di Shah Alam?"
- Do you test in Malay and Chinese, and report each language separately?
- How many times is each question run, and how often each month?
- Do you report a margin, or only a single percentage?
- Can I see the full answers and the source links, not just a score?
Fishkin urged buyers to stop paying for tracking without "stats-backed, publicly-reviewable research"1.
What can't a ChatGPT mention rate from any AI visibility tool tell me?
A ChatGPT mention rate can't tell you three things, however carefully a tool measures it:
- What real customers typed. Tools test chosen questions, not your customers' actual chats.
- Whether it brought sales. Being named isn't the same as being chosen.
- Why you were named. A tool sees what ChatGPT said and cited, not how it decided.
Where do I start measuring how often ChatGPT mentions my business?
Start measuring with a small baseline: your customers' questions, in their languages, run more than once. In our test, few names held steady between answers, so few competitors had a fixed place.
- Request EasyFound's free check. See what ChatGPT says about your business, then go through it with us on a free call.
- Write down ten questions your customers would ask, in English, Malay or Chinese.
- Send the seven questions above to any tool or agency before you sign.
- Re-measure the same way each month, so a change reflects your business rather than a new setup.
EasyFound, an AI SEO service for Malaysian businesses, follows what six AI tools say about you in English, Malay and Chinese over time, then improves the pages those answers rely on. It starts with the free check below.
How often does ChatGPT actually name you?
Before you pay for a tracking number, see what ChatGPT says about your business, then go through it with us on a free call.
Frequently asked questions
Do AI visibility tools check the real ChatGPT app or the API?
Some tools send questions through an API and others read answers from the ChatGPT app, so ask each vendor which one it uses. The app can add things the API doesn't: OpenAI says that with Memory on, ChatGPT may use details such as location to shape its searches, and it can estimate your location from your IP address. A small 2026 preprint found the top recommendation changed just as often on the logged-out ChatGPT website as through OpenAI's API, though that doesn't mean the two give the same answers.
Is a ranking position in ChatGPT worth paying to track?
Not on its own. In EasyFound's Malaysian test through OpenAI's API, 44 of the 69 businesses named in more than one run of the same question appeared at a different place in the answer. SparkToro's Rand Fishkin wrote that any tool giving a 'ranking position in AI' is 'full of baloney'. The share of answers that name you, across many runs, is the steadier number.
Is three runs per question enough to measure my ChatGPT mentions?
Three runs gives a rough first look, not a precise figure. A business named once in three runs could really be named anywhere from about 6% to 79% of the time. SparkToro suggests asking the same question 'usually at least 60-100X' to see an AI's full set of recommendations, and a 2026 preprint on AI sentiment about brands found spreading checks across languages and AI tools cut the noise more than repeating one question.
Should an AI visibility tool test my business in Malay and Chinese?
Yes, if your customers ask in Malay or Chinese. In EasyFound's September 2026 test of Malaysian buying questions through OpenAI's API, 178 of 242 businesses ChatGPT named for a Malay or Chinese question never appeared in the English answers to the same question. Part of that gap is ordinary run-to-run change, but the English runs alone didn't show those names.
Sources
Our data: EasyFound, How repeat runs, rank and language change a ChatGPT mention count, a re-analysis of 135 answers to 45 Malaysian buying questions (15 questions in English, Bahasa Malaysia and Chinese, in hotels, renovation, solar, company secretary and accounting), each asked three times on 24 September 2026 through OpenAI's Responses API with web search available, location Malaysia, model gpt-5.4-mini, default settings, no chat history. This is not the ChatGPT app. The test questions were short keyword forms (for example "best hotels in Penang"); the example questions above are written the way people type into ChatGPT. A model listed the businesses in each answer, and a name was kept only if it appears word for word in the answer. Place in the list is the order in which each business first appears in the answer. Each question's list is counted separately. The 6%–79% range is a 95% Wilson interval for 1 of 3. Download the per-question results (CSV). We don't name the businesses, because one test isn't a ranking.
Other sources:
- SparkToro (Rand Fishkin and Patrick O'Donnell), NEW Research: AIs are highly inconsistent when recommending brands or products, 27 January 2026; 2,961 responses from 600 volunteers, US prompts. SparkToro discloses that O'Donnell works at an AI tracking startup (link)
- Dmitrij Żatuchin, Where Does the Noise Come From?, arXiv preprint, 14 July 2026; 12,933 responses on 20 Central and Eastern European brands, 8 languages, 3 models, measuring sentiment rather than mentions; not peer reviewed (link)
- Dmitrij Żatuchin, The Language of the Question Selects the Market, arXiv preprint, 30 August 2026; 234 runs against the logged-out ChatGPT website and OpenAI's API, four exit countries, six languages; not peer reviewed (link)
- OpenAI, Memory in ChatGPT, Help Center, read 30 September 2026 (link)
- OpenAI, Searching the web with ChatGPT, Help Center, read 30 September 2026 (link)
The results are shared under CC BY 4.0: you're free to reuse them, with credit to EasyFound.
ChatGPT's features change often; the memory and location details above were checked on 30 September 2026.
Research by Janice Pang, EasyFound.