Find out why your AI gives the wrong policy even though the right document is in its files, fix the files and instructions, and re-test with known answers.
Who it is for
Owners using a Claude or ChatGPT project, or a support bot, that answers from their own documents and sometimes gets a policy or price wrong.
Tools
Your Claude or ChatGPT project
The files it uses
5 questions you already know the answers to
Steps
Ask the wrong question again and require it to quote the exact file and passage it used.
If it quotes an old or different file, delete or rename the outdated one. Two price lists in one project is the most common cause.
Anthropic says Claude switches to a retrieval mode when project knowledge gets large, so it pulls the passages that look most relevant. Put your key rules (refunds, prices, service area) in one short cheat sheet at the top.
Add the instruction from the prompt below so it quotes its source and says when files disagree.
Re-test with your 5 known questions, including the one it got wrong.
Only consider a different model after the files and instructions are clean.
The prompt
Add to the project instructions:
When you answer:
1. Quote the exact sentence you used and name its file.
2. If two files disagree, show both quotes, say which file is newer, and do not pick one silently.
3. If no file answers the question, say: Not in our documents.
4. For prices, refunds and deadlines, use the cheat sheet file first.
Now answer this test question using those rules: [THE QUESTION IT GOT WRONG]
Tips
Date your file names so old versions are easy to spot.
Short cheat sheets beat long manuals for rules that must be exact.
Keep your 5 test questions and re-run them after every file change.