Writing · AI / Automation / Tech

2026-02-02
Why AI Guesses Instead of Saying 'I Don't Know AI gave me a confident answer last week. Completely made up. This keeps happening when you ask AI to pull data from documents. It reads your file, can't find what you're looking for, and instead of saying "I don't know," it guesses. Why? Because these tools are trained to be helpful. Helpful means giving you an answer. Even a wrong one. Try this next time... Plan before you execute. Don't ask "write me an analysis of this document." Instead: "Before you analyze this, outline your approach. What will you look for? How will you structure it?" Review the plan. Adjust it. Then let it run. Why? Because alignment on the front end prevents garbage on the back end. This alone doubles your hit rate on complex tasks. Use the smartest model available. Default models are lazy. Pick Claude's extended thinking, ChatGPT's o3, or Gemini Pro. Speed doesn't matter if the answer is wrong. Ground it with clear rules. Tell the AI exactly how to behave: "Only use information from this document. If you can't find something, say 'not found.' Don't guess. Cite page numbers and quote the source text. If you're not 100% confident, mark it as 'unverified.'" That last part helps you prioritize. You'll know exactly where to double-check first. Keep it short. Long AI responses create room for fabrication. Ask specific questions. Get specific answers. Tell it what NOT to do. If AI keeps making the same mistake with you, say so upfront. "When you analyze this T12, don't assume rent growth. Use actual numbers only." You're training it for this conversation. Know when to start fresh. Long chats degrade performance. The model forgets earlier details, prioritizes recent messages, and compresses older context. Errors stack up. When answers get sloppy, copy your key info into a new conversation and start clean. (MOST IMPORTANT IDEA)Verify with a skeptic. After complex analysis, I paste everything back and say: "Assume the role of a forensic auditor. Review these results for errors, contradictions, or unsupported claims. It's okay to find nothing wrong." That last line matters. Without it, the AI invents problems. I could be wrong, but the people complaining that AI is useless are treating it like a search engine instead of a junior analyst. Junior analysts need supervision. So does AI. The tool works. But only if you work the tool. I recently did a podcast on prompting with Paul Marks. We cover many of these ideas and more.
AI / Automation / TechMarketing / Copy / BrandHiring / People / LeadershipReal Estate (general)

View original on LinkedIn

← Back to writing