How do I know if AI's answer is good or garbage?

by Stephen Wu, a retired engineering manager, spent several years building AI workflows for his teams and now trying to make AI something everyone can understand and use successfully

AI can sound confident and authoritative even when it's completely wrong, so learning to evaluate its responses is just as important as learning to ask good questions. The best way to judge an answer is to check if it makes logical sense, aligns with what you already know, and can be verified through other sources when facts matter. Treat AI like a helpful but fallible assistant—trust, but verify.

The confidence problem

AI doesn't say "I'm not sure" or "this might be wrong." It presents everything with the same level of certainty, whether it's explaining basic math (where it's almost always right) or discussing obscure historical events (where it might be making things up).

What this means for you: Don't judge an answer's quality by how confident it sounds. A detailed, well-written response could be completely fabricated. A hedged, uncertain answer might be more honest about the limits of what AI actually knows.

Quick checks you can do immediately

Does it make logical sense? Read the answer and ask yourself: does this actually make sense, or does something feel off? If AI tells you to clean your laptop screen with ammonia-based cleaner (don't), your common sense should raise a flag.

Does it align with what you already know? This is why starting with familiar topics is helpful. If you ask for recipe suggestions and AI tells you to bake cookies at 800 degrees, you know something's wrong even if you're not a professional chef.

Are there internal contradictions? Does the answer contradict itself? If AI says "always do X" in one paragraph and then "never do X" two paragraphs later, that's a red flag.

Is it oddly generic? If you asked a specific question but got a vague, general answer that could apply to almost anything, AI might not actually know and is giving you filler content.

When you absolutely need to verify

Facts, numbers, and dates AI frequently gets these wrong. If the specific detail matters—"What year did this law pass?" or "How much does this medication cost?"—look it up independently.

Current events and recent information Most AI systems have knowledge cutoffs, meaning they don't know about anything that happened after a certain date. Even if they try to answer, they're guessing.

Health, legal, or financial advice AI isn't qualified to diagnose medical conditions, provide legal counsel, or offer financial planning. Treat these answers as starting points for questions to ask real professionals, not as actual advice.

Anything with serious consequences If you're making a decision that affects your career, health, relationships, or money, verify AI's answer through authoritative sources or real experts.

How to verify when it matters

Cross-reference with authoritative sources If AI says a fact, Google it or check Wikipedia. If AI summarizes a process, look at official documentation. For health information, check sites like Mayo Clinic or CDC.

Ask AI for sources Try following up with: "Where did this information come from?" or "Can you cite sources for this?" AI might provide references—but be aware it sometimes invents fake sources that sound real. Always check if the source actually exists and says what AI claims.

Test it with something you know If you're asking AI about Topic X which you don't know well, first ask it a question about Topic Y which you do know well. How accurate was it there? That gives you a baseline for its reliability.

Get a second opinion Ask a different AI tool the same question. If ChatGPT and Claude give wildly different answers, neither should be fully trusted without verification.

Red flags that an answer might be wrong

Overly specific details that seem random If you ask "How many people visited Yellowstone in 2023?" and AI gives you an oddly specific number like "4,287,619 visitors," be skeptical—especially if it's a number that would be hard to know precisely.

Confident answers to questions with no clear answer "What's the best programming language?" has no objective answer, but AI might present one option as definitively correct.

Historical claims that sound too neat Real history is messy. If AI tells you a historical story with perfect narrative arc and suspiciously convenient details, it might be embellishing or inventing.

Medical or legal advice that's too specific General information ("most antibiotics should be taken with food") is more likely accurate than specific medical advice ("you should take 500mg of amoxicillin twice daily for your infection").

What "good enough" looks like

Not every answer needs perfect accuracy. Sometimes "good enough" is fine.

Good enough for brainstorming: "Give me ideas for a team-building activity." The suggestions don't need to be revolutionary or perfectly tailored—they're just starting points.

Good enough for learning: "Explain how a car engine works." Minor inaccuracies in a simplified explanation are acceptable if the overall concept is correct.

Good enough for drafting: "Help me write an email declining an invitation." The draft doesn't need to be perfect—you'll edit it anyway.

Not good enough for facts: "What's the exact tax rate for my situation?" Wrong here could cost you money.

Not good enough for decisions: "Should I take this job offer?" AI doesn't know your life context well enough for this.

How to handle answers you're not sure about

Ask AI to explain its reasoning "How did you arrive at that conclusion?" or "What's your thought process here?" Sometimes this reveals gaps in AI's logic.

Ask AI to steelman the opposite view "What's the strongest argument against what you just said?" This can reveal whether AI is considering the full picture.

Request simpler language If an answer is confusing, ask: "Explain that like I'm 10 years old." If AI can't simplify it clearly, it might not actually understand the topic.

Challenge it "Are you sure about that?" or "I read that the opposite is true—can you clarify?" AI will often revise its answer, which tells you it wasn't certain to begin with.

Building your own judgment over time

The more you use AI, the better you'll get at recognizing when it's on solid ground versus when it's confabulating. Pay attention to patterns:

What types of questions does it consistently handle well?
Where have you caught it being wrong before?
What topics does it tend to be generic or vague about?

Your goal isn't to become a fact-checking expert. It's to develop a healthy skepticism—trust AI as a useful tool, but verify anything that matters.

For more on what AI is reliably good versus bad at, see What tasks is AI good at vs. not good at?. And for tips on asking better questions that get better answers, check out What if I don't know how to phrase what I'm asking?.