AI is a stranger on the internet.
I mean that literally. It was trained on the internet, which means it is a statistical average of every voice on every forum, blog, Q&A site, comment thread, and Reddit subforum from the last twenty years. The thing on the other side of the chat box has no name, no domain, no date, no comment section, and nothing on the line. Anonymous and confident is the worst possible combination of traits on the old internet, and we used to know that.
Once you hold that frame, the trust calibration mostly handles itself. You already know how to handle a confident-sounding stranger on the internet. You learned it in forums and chat rooms by getting fooled enough times to develop a permanent eyebrow-raise. On topics where the average internet voice is a meaningful upgrade over the uninformed friend you’d otherwise have asked (and that covers most of what we Google in a day), AI is useful. Summaries, drafts, explanations, brainstorms. The average voice is good enough.
There is exactly one place the frame stops working. The place where, on the old internet, you wouldn’t have asked strangers at all. You’d have gone and found a licensed professional. A doctor. A lawyer. A CPA. Someone with a name and a license number and a malpractice policy. AI doesn’t get to follow you there. The interesting question is why.
The answer most people give is something like “because the stakes are higher, so the answer needs to be more accurate.” That isn’t quite it. AI is often accurate. But accuracy was never the thing doing the work.
What “licensed” actually means is closer to four things.
There is a written standard of care. GAAP for accountants. ABA model rules for lawyers. Generally accepted standards of medical practice for doctors. The standard pre-exists the question. A licensed professional applies a body of rules that has been hammered out, often over centuries, by a community that earned the authority to define what “right” means in that domain. The average internet voice has nothing like that behind it. It is just opinion.
There is a specific human who is accountable for the answer. With a name, an address, a license number, and a body that can take the license away. The accountability is enforceable. You can sue them. You can file a complaint. The licensing board can pull their ticket and end their career. That enforceable exposure is what keeps them careful, even on the days they can’t be sure they’re right.
That human went through apprenticeship under consequences. Years of supervised practice on real client work, with real outcomes attached. A CPA’s ten thousand hours are spent doing the work in front of someone who can fail them, for clients whose money is real, with auditors who will eventually look at the file. That is the apprenticeship that produces judgment, and judgment is what gets exercised when the standard of care isn’t a clean fit for the specific question on the table.
And there is an insurance market that has priced their competence. You can buy a policy that pays out if your accountant misses something material. You can buy a policy that pays out if your surgeon nicks the wrong artery. You cannot buy a policy that pays out if your favorite LinkedIn AI evangelist was confidently wrong about your tax position. The insurance market is, structurally, a second opinion, one that has put its own money behind the answer.
Together those four things (written standard, named accountability, supervised apprenticeship, priced insurance) are the architecture of trust for decisions where being wrong has asymmetric downside. You can’t bolt them onto a model. They exist only because a specific human exists who can be held to them, and a better LLM does nothing to supply them, because what they turn on is who is on the line when the answer is wrong.
AI has none of those four things, and not for lack of effort. There is no licensing board for an LLM. There is no insurance policy that pays out when an LLM hallucinates a tax position. Its training resembles apprenticeship statistically but carries none of the consequences. An LLM can’t lose a license it never had, and there is nothing there to sue. No model release closes that. The absence is categorical.
Some forms of trust can be replaced by raw capability. Reading comprehension, summarization, basic legal drafting, the median doctor’s bedside Google. Those were always about access to information and pattern matching, and AI is at parity or better on most of them already. Other forms of trust run on accountability, which has nothing to do with how capable the responder is. Capability and accountability are different categories, and one does not bootstrap into the other no matter how good the model gets.
This is the calibration rule the stranger-on-the-internet frame implies, once you sit with it long enough. AI is a real upgrade on every question you used to ask the internet. On every question you used to find a licensed person for, you still find a licensed person, even when the AI’s answer might be perfectly correct. Because the thing you actually need was never just the right answer in the abstract. You need a right answer that a specific human is accountable for, and that human has to exist before the question gets asked.
That part doesn’t ship in a chat box. It probably never will.