Last updated:
Quick answer
Before switching models, check the files: old versions, two documents that disagree, or a long file where the rule is buried. Remove outdated files, put the key rules in one short cheat sheet, tell it to quote the file it used, and re-test with questions you know the answers to.
Free PDF
Get this setup as a PDF
Confirm your email and we'll open a print-ready copy of this setup you can save as a PDF. You also get the free starter pack and one setup a week. Unsubscribe anytime.
Already a subscriber? Unlock it on this device
At a glance
All outcome figures are estimates from cited sources, not independent verification by us.
- What it is allowed to do
- Read-only
Only reads, researches, or summarizes. It never drafts outbound content or changes anything.
- Time saved (estimate)
- About 30 minutes to diagnose and fix.
- Difficulty
- Intermediate
- Tools needed
- Claude, ChatGPT
- Typical cost
- Projects availability varies by plan. Prices: Not stated in sources.
Reviewed Nov 2, 2026
Basis (estimate): Estimate only: based on the steps above, not timed.
What does this setup do?
Find out why your AI gives the wrong policy even though the right document is in its files, fix the files and instructions, and re-test with known answers.
Who is it for?
Owners using a Claude or ChatGPT project, or a support bot, that answers from their own documents and sometimes gets a policy or price wrong.
How do you set it up?
Tools
- Your Claude or ChatGPT project
- The files it uses
- 5 questions you already know the answers to
Steps
- Ask the wrong question again and require it to quote the exact file and passage it used.
- If it quotes an old or different file, delete or rename the outdated one. Two price lists in one project is the most common cause.
- Anthropic says Claude switches to a retrieval mode when project knowledge gets large, so it pulls the passages that look most relevant. Put your key rules (refunds, prices, service area) in one short cheat sheet at the top.
- Add the instruction from the prompt below so it quotes its source and says when files disagree.
- Re-test with your 5 known questions, including the one it got wrong.
- Only consider a different model after the files and instructions are clean.
Example output
Illustrative. Your results will differ.
- Quoted: 'Refunds within 14 days' from returns-policy-2025.pdf (outdated)
- Newer file returns-policy-2026.pdf says 30 days
- Fix: delete the 2025 file, add the 30-day rule to the cheat sheet, re-test: correct
Where to paste this prompt
Claude → Project knowledge & custom instructions
Copy-paste prompt
Adapt bracketed fields to your situation.
Add to the project instructions:
When you answer:
1. Quote the exact sentence you used and name its file.
2. If two files disagree, show both quotes, say which file is newer, and do not pick one silently.
3. If no file answers the question, say: Not in our documents.
4. For prices, refunds and deadlines, use the cheat sheet file first.
Now answer this test question using those rules: [THE QUESTION IT GOT WRONG]Newsletter
Get the free starter pack
Free starter pack with five beginner copy-paste prompts after you confirm, plus one researched setup in your inbox each week. Unsubscribe anytime.
What usually goes wrong?
Where agents tend to stall on this job. More patterns in Where AI gets stuck.
- Swapping models when the real problem is an outdated file.
- Burying the refund rule on page 30 of a manual.
- Fixing the answer in one chat instead of in the project files.
Which chat tools work with this prompt?
Paste the prompt into any of these assistants (free or paid tiers).
- Claude
- ChatGPT
Common questions
- Why does my AI give the wrong answer when the right document is uploaded?
- Usually an outdated or conflicting file, or a rule buried in a long document. Make it quote its source to find out which.
- Should I switch AI models?
- Clean up the files and instructions first. Most wrong answers come from the documents, not the model.
- How do I know it is fixed?
- Re-run a set of questions you already know the answers to, including the one it got wrong.
What tips make it smoother?
- Date your file names so old versions are easy to spot.
- Short cheat sheets beat long manuals for rules that must be exact.
- Keep your 5 test questions and re-run them after every file change.
Where do the claims come from?
Summaries use public sources linked below, not independent verification. How setups are picked.
- Read Anthropic’s write-up (support.claude.com)
Anthropic: What are projects: project knowledge and instructions; RAG mode when project knowledge approaches context limits (read 2026-11-02 from a search result excerpt of the page)
- Read OpenAI’s write-up (help.openai.com)
OpenAI: Projects in ChatGPT: chats, files and instructions kept together (read 2026-11-02 from a search result excerpt of the page)
Related setups
- Build a knowledge base your AI answers from
Load your prices, policies and how-tos into one Claude or ChatGPT project so you and your team can ask it questions and get answers from your own documents.
- Ask your job files questions with NotebookLM
Put a job's specs, contract, change orders and notes into one NotebookLM notebook and ask plain questions, like what the contract says about change orders, with answers from your own files.