Why does ChatGPT give a different answer every time?

You ask ChatGPT for the best hotel in Penang or a renovation contractor in KL. You get a list. You ask again, and most of the names have changed. Nothing is broken: ChatGPT writes each answer fresh, and for buying questions it usually runs a new web search, so the list changes almost every time.

In our test, two answers to the same question had only about one business in seven in common. That changes how you should check whether ChatGPT recommends you.

How different are ChatGPT's answers when you ask the same question twice?

ChatGPT's answers to the same question changed a lot from run to run in our test. We asked Malaysian buying questions in English, Malay and Chinese, three times each. Two answers to the same question shared only 15% of the businesses they named (per-question data, CSV).

Some trades were steadier than others:

Hotels were steadiest because two questions drew well-known hotels and resorts. Company secretary questions mostly got "how to choose" advice, with a firm named in only 6 of 27 answers. Every question is in the CSV.

Why does ChatGPT give a different answer every time?

ChatGPT gives a different answer every time for four reasons, and they stack up:

1. Chance is built into how it writes. OpenAI's API documentation says its outputs are "non-deterministic by default (which means model outputs may differ from request to request)"¹. Our reading is that the chance that swaps one word can also swap which business comes next.
2. It doesn't always search. Asked in Chinese about solar panels to cut a TNB bill, ChatGPT searched in only 1 of 3 runs. Without a search, it can only name businesses it learned about in training.
3. Each search brings back different pages. OpenAI says ChatGPT "typically rewrites your query into one or more targeted queries"². Two runs of the same question shared only 9% of the pages they cited, and the businesses changed with the pages.
4. In the app, it remembers you. With Memory on, ChatGPT can draw on past chats, saved memories and custom instructions³. It may also estimate your location from your IP address².

Here's the third reason in action. Asked for the best hotels in Penang, ChatGPT wrote these two searches itself, in two different runs:

• "best hotels in Penang 2026 luxury family boutique official site"
• "best hotels in Penang 2026 luxury beach heritage Georgetown official awards reviews"

Different searches, different pages, different hotels.

Do other AI tools also give a different answer every time?

Other AI tools change their answers too, not only ChatGPT. In SparkToro's study, 600 volunteers asked ChatGPT, Claude and Google's AI the same prompts again and again. SparkToro found "there's a <1 in 100 chance that ChatGPT or Google's AI, if asked 100X, will give you the same list of brands in any two responses"⁴.

Their prompts were about US brands, ours about Malaysian businesses. Both point the same way: one list is one draw, not a ranking.

What can't our test tell me about why ChatGPT's answers change?

Our test measured how much the answers changed, not everything behind it:

• The app. We tested through OpenAI's API, which may vary more or less than the ChatGPT app.
• Which cause mattered most. Chance in the wording and changes in search results happened together.
• Change over weeks. All the answers came from one evening.

How do I check if ChatGPT recommends my business when the answer keeps changing?

Check ChatGPT by counting how often you're named across several runs, not where you appear in one list. SparkToro reached the same view: how often a brand appears across many runs is "(probably) a reasonable way" to measure visibility⁴.

The good news: few names are locked in. A competitor named today may be missing tomorrow, and you can be in the next draw if pages about you turn up in its searches.

1. Request EasyFound's free check. See what ChatGPT says about your business across repeated questions, then go through it with us on a free call.
2. Log out of ChatGPT. A Temporary Chat can still use your memories unless you pick "Unpersonalized"³, and OpenAI says "People who are not signed in can also use web search"².
3. Ask the questions your customers ask, in their words and languages, three times each.
4. Count your appearances. Named in 2 of 3 runs is a stronger sign than 1 of 3, though three runs is only a rough check.
5. Re-check every month the same way, and open the source links to see which pages ChatGPT used.

Because one answer is only a draw, EasyFound, an AI SEO service for Malaysian businesses, looks at how often you're named across six AI tools in English, Malay and Chinese, and works on the pages they draw on. Start with the free check below.