Trust in AI is not a binary switch you flip to “on” or “off.” Research over the past three decades shows it depends on a tangle of factors: who the user is, what the AI system looks like and does, and the situation in which the two meet. That interaction produces outcomes ranging from blind over-reliance to stubborn refusal to use a perfectly competent tool. Understanding what nudges people in each direction has become one of the most active areas in human-computer interaction research, and the findings are often counterintuitive.
What Actually Shapes Your Trust in an AI System
A large review tracing the evolution of trust research from early factory automation to modern AI in healthcare distilled the drivers into three buckets: user characteristics, AI system attributes, and contextual factors.1PubMed Central. From Trust in Automation to Trust in AI in Healthcare: A 30-Year Longitudinal Review and an Interdisciplinary Framework User characteristics include things like your personality, your prior experience with technology, your domain expertise, and even your general attitude toward machines. System attributes cover accuracy, reliability, the way an AI communicates its reasoning, and how human-like it feels. Context includes the stakes involved, the time pressure you’re under, and whether you have an alternative to the AI’s recommendation.
Earlier foundational work on trust in automation found that these characteristics don’t operate through a single mental channel. People form trust through analytic processes (evaluating evidence about how well the system performs), analogical processes (comparing the AI to past experiences with similar tools or people), and affective processes (gut feelings shaped by how the system looks, sounds, or makes them feel).2PubMed. Trust in automation: designing for appropriate reliance In practice, all three run simultaneously. You might know a chatbot’s accuracy statistics, compare it mentally to a bad experience with automated phone menus, and still feel reassured because it has a friendly avatar. That messy overlap is why designing for “appropriate trust” is so hard.
Automation Bias and Over-Reliance
The most studied failure mode in AI trust isn’t too little trust. It’s too much. Automation bias is the tendency to accept an AI’s output without adequate scrutiny, even when you have the skills to catch an error. A review of the research found that the traditional explanation, simple over-trust, tells only part of the story. AI literacy, professional expertise, cognitive style, and how much effort verification demands all interact to determine whether someone will just go along with a machine’s answer.3AI & SOCIETY. Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
An empirical study of clinicians using AI-assisted wound diagnostics showed this in action. Having certified wound-care training and stronger diagnostic skills reduced the rate at which clinicians falsely agreed with an incorrect AI recommendation. But clinicians who perceived the system as highly beneficial were more likely to go along with wrong answers.4PubMed. Automation Bias in AI-Decision Support: Results from an Empirical Study In other words, liking the tool and believing in its value can make you worse at catching its mistakes. That finding has uncomfortable implications for any workplace where enthusiasm for a new AI system is treated as a positive sign.
A separate study on distinguishing real faces from AI-generated ones found a related pattern: people who held more positive attitudes toward AI were actually worse at telling real from synthetic faces when given AI guidance.5Scientific Reports. Examining human reliance on artificial intelligence in decision making Positive attitudes didn’t just make people more willing to rely on the AI; they seemed to dull the ability to spot when the AI was wrong.
How Explanations Affect Trust
One of the most intuitive ideas in AI design is that explaining how a system reaches its conclusions should help people trust it appropriately. The reality is messier. An experimental study with 215 participants found that explanations did increase trust, but only under certain conditions. When the AI showed which features mattered most for a prediction, trust went up. Counterfactual explanations (“if this variable had been different, the outcome would have changed”) didn’t have the same effect. And when the system’s actual performance wasn’t guaranteed, providing explanations seemed to push people toward over-reliance rather than calibrated trust.6Human Behavior and Emerging Technologies. AI Trust: Can Explainable AI Enhance Warranted Trust?
This points to a paradox. Explanations can make you feel more confident without actually making you more accurate. If the explanation sounds plausible, you may stop questioning the output, which is the opposite of what transparent AI design is supposed to accomplish. Research on health-monitoring AI found that visual explanations of how individual features contribute to a prediction can help clinicians interpret and validate results, giving them genuine insight into the model’s reasoning.7Scientific Reports. Personalized health monitoring using explainable AI: bridging trust in predictive healthcare But even well-designed visualizations assume the user has enough domain knowledge to evaluate whether the explanation makes sense.
Domain expertise turns out to matter a great deal. In a peer-to-peer lending experiment, participants who received AI explanations made significantly fewer risky investments than those who didn’t, and they were better at overriding the AI when it made incorrect predictions.8International Journal of Human-Computer Studies. The effects of domain knowledge on trust in explainable AI and task performance: A case of peer-to-peer lending The key difference: people with more domain knowledge used the explanations as tools for independent judgment, while those with less knowledge tended to treat the explanation itself as evidence that the AI was right.
The Confidence Display Problem
Many AI systems now show confidence scores alongside their recommendations: “I’m 87% confident this is a benign mole,” or “there is a 72% chance of rain.” When those confidence displays are well calibrated, meaning the AI really is right about 87% of the time when it says 87%, they help users make better decisions. But when they’re off, users struggle badly. Research on miscalibrated AI confidence found that it impairs appropriate reliance and reduces the quality of AI-assisted decisions. Worse, users have a hard time detecting when the confidence numbers are wrong.9arXiv. Understanding the Effects of Miscalibrated AI Confidence on User Trust, Reliance, and Decision Efficacy
This is a design problem, not just a technical one. A system that confidently displays “94% sure” when its actual track record at that confidence level is closer to 60% doesn’t just produce bad individual predictions. It trains the user to rely on the confidence score as a mental shortcut, and then that shortcut leads them astray. The finding that miscalibration is hard for users to detect means people won’t naturally self-correct over time; they’ll just keep trusting the numbers.
What Happens When AI Makes a Mistake
Errors are inevitable, and how they affect trust is sharply asymmetric. A study of AI-assisted financial advising found that participants initially preferred algorithmic advisors over human ones, showing what researchers call “algorithmic appreciation.” But a single advisory error caused a substantial drop in trust. The good news: post-error explanations significantly helped trust recover, sometimes even beyond the baseline level.10PubMed Central. Trust Formation, Error Impact, and Repair in Human–AI Financial Advisory: A Dynamic Behavioral Analysis
That pattern, quick to trust initially, fragile after an error, responsive to explanation, shows up in many contexts. People seem to hold AI to a different standard than they hold humans. A human financial advisor who makes one bad call might keep your benefit of the doubt. An AI that makes one bad call can trigger an abrupt switch from “I trust this completely” to “I don’t trust this at all.” Providing a clear reason for the error (“the market conditions were unprecedented” or “the data was incomplete”) doesn’t just manage the immediate damage; it can actually strengthen the relationship if the explanation feels honest.
Trust in the Doctor’s Office
Healthcare is one of the highest-stakes environments for AI trust, and the dynamics are more complex than in most other domains because there’s a triangle: the patient, the physician, and the AI system. A vignette-based study explored what happens to patient trust in their doctor when patients learn the doctor used an AI clinical decision-support tool. In high-risk scenarios, women rated their physician as less competent and less trustworthy when AI was involved, while men showed no significant change.11PubMed Central. The Effect of Artificial Intelligence on Patient-Physician Trust: Cross-Sectional Vignette Study In low-risk scenarios, the AI tool didn’t affect trust either way, but patients who already had high trust in technology rated the physician more favorably.
The gendered response is worth pausing on. It suggests that for some patients, learning a doctor relied on AI feels like the doctor is outsourcing their judgment, and that perception affects some demographics more than others. Physicians who want to use AI tools may need to frame them carefully: “I used this tool to double-check my assessment” lands differently than “the AI recommended this treatment.”
Trust Behind the Wheel
Autonomous vehicles offer a particularly visceral test of AI trust, because the consequences of misplaced trust are immediate and physical. Research on Level 3 automated driving found that collisions during a takeover event undermined trust, which is unsurprising. More useful was the finding that giving drivers more lead time before a takeover request improved trust, without causing the confusion designers had feared.12PubMed. Effect of Takeover Request Time and Warning Modality on Trust in L3 Automated Driving The modality of the warning, whether it was visual, auditory, or both, didn’t make a significant difference to trust levels.
Drivers with low trust in autonomous systems spent more time manually controlling the vehicle and delayed handing control back after taking over.13Proceedings of the Human Factors and Ergonomics Society Annual Meeting. To Takeover or Not to Takeover? The Role of Drivers’ Trust in Influencing Time Spent in Manual Versus Automated Driving That behavior creates its own risks: a driver who is constantly toggling between manual and automated mode is paying a mental switching cost, and the extra manual driving might not actually be safer. Higher vehicle reliability and repeated positive interactions with the autonomous system were the strongest trust builders.14Safety Science. The effects of automated vehicle reliability, self-estimated confidence, and repeated interactions on drivers’ trust and takeover decisions In short, the vehicle has to earn trust through consistent performance, and there’s no design shortcut that replaces that track record.
Hallucinations and the Trust Crisis in Generative AI
Large language models introduced a trust problem that older, narrower AI systems didn’t have: hallucination, where the system produces confident-sounding statements that are simply wrong. Research on human-AI collaboration found that encountering hallucinations damages both cognitive trust (your rational assessment that the AI is competent) and emotional trust (your gut feeling that it’s reliable). That dual erosion, in turn, impairs the quality of collaboration itself.15Technology in Society. The trust crisis in artificial intelligence: AI hallucinations and human-AI collaboration
What makes hallucinations especially damaging is that they’re indistinguishable from correct outputs unless you already know the answer. A financial model that gives a wrong stock prediction is wrong in a way you can eventually verify. A chatbot that fabricates a legal citation or invents a medical study is wrong in a way that requires independent research to catch. For users who lack the domain knowledge to fact-check, a single hallucination they later discover can poison their willingness to use the tool at all, even for tasks where it performs well.
Does Labeling AI-Generated Content Change Anything?
Governments and platforms have increasingly pushed for disclosing when content is AI-generated, under the reasonable assumption that transparency should help users calibrate their trust. The experimental evidence so far is surprisingly underwhelming. A randomized controlled experiment found that AI-generated content labels did not significantly affect perceived accuracy, message credibility, or sharing intentions for misinformation.16PubMed Central. Impact of Artificial Intelligence–Generated Content Labels On Perceived Accuracy, Message Credibility, and Sharing Intentions for Misinformation A separate study specifically examining how labels affect perceived credibility found the same thing: the main effect of the label on credibility was not significant.17Emerging Media. Nudging Perceived Credibility: The Impact of AIGC Labeling on User Distinction of AI-Generated Content
This doesn’t mean disclosure is useless, but it does suggest that slapping a label on AI content isn’t the trust-calibration lever that policymakers hoped. People seem to evaluate content based on its surface plausibility, the source that shared it, and their own prior beliefs rather than on whether a label tells them a machine produced it. Making disclosure effective may require pairing the label with other interventions, like media literacy training or contextual warnings that explain why the AI-generated distinction matters for a specific type of content.
Fairness and Bias as Trust Destroyers
Trust erodes fast when people believe an AI system treats them unfairly. A perspective paper examining algorithmic bias in Africa documented how biased AI applications can worsen social disparities and diminish confidence in public institutions.18PubMed Central. Navigating algorithm bias in AI: ensuring fairness and trust in Africa When AI-driven lending tools, hiring platforms, or health triage systems produce systematically different outcomes for different groups, the damage extends beyond individual decisions. It undermines trust not just in the specific tool but in the institutions deploying it.
Algorithmic lending provides a concrete example. The growing reliance on AI for credit decisions raises concerns about fairness, discrimination, and transparency, and proposed frameworks suggest that consumer trust depends on data quality, human oversight, and organizational accountability supporting those systems.19Proceedings of the Association for Information Science and Technology. “Saying is Believing”: Exploring the Importance of AI‐Generated Content Disclosure and User Trust A system that denies loans at disproportionate rates to certain communities won’t build trust by adding an explanation module. The explanation has to accompany genuinely fair outcomes.
How Trust Evolves Through Interaction
Trust in AI is not static. Research tracking trust dynamics within human-AI teams found that people form expectations about the AI’s ability, integrity, and adaptability before they ever start working together. Those expectations set a baseline. But through collaboration, people develop “situational trust” that can either confirm or overwrite that initial impression, depending on how the AI communicates and performs in specific moments.20Proceedings of the ACM on Human-Computer Interaction. Understanding the Evolvement of Trust Over Time within Human-AI Teams Crucially, the trust that develops through collaboration shapes future expectations, creating a feedback loop.
This means early interactions carry disproportionate weight. An AI system that performs well in the first few encounters builds a trust reservoir that can absorb a later mistake. One that stumbles early may never recover, even if its long-term accuracy is high. Designers sometimes call this the “cold start” problem: the period before the user has enough experience to calibrate their trust is the most dangerous window for both over-trust and under-trust.
When Numbers Create a False Sense of Intelligence
One of the more surprising findings in AI trust research involves the persuasive power of numbers, even meaningless ones. A study examining how people react to different styles of AI explanation found that both AI-literate and AI-illiterate participants rated a robot that used numerical representations as more intelligent than one that explained itself in plain language, even when the numbers were incomprehensible. Participants in the AI-literate group associated math with logical algorithmic thinking, even while admitting they couldn’t understand the reasoning. In the AI-illiterate group, the inability to understand the numbers actually reinforced the perception of intelligence: the “cryptic incomprehensibility” of numerical output signaled that the system was smarter than them.21Cell Press (Patterns). Explainability pitfalls: Unintended negative downstream effects of AI explanations
This is a genuinely alarming finding for anyone designing or regulating AI explanations. It suggests that making a system look more trustworthy can be as simple as displaying numbers alongside its output, regardless of whether those numbers mean anything useful. Regulations that mandate “explainability” without specifying what counts as a good explanation could inadvertently encourage this kind of trust theater, where the form of an explanation creates trust that its content does not warrant.
Cultural and Appearance Effects
How an AI system looks, and who is looking at it, also matters. Research examining cultural cognition and AI trust found that the visual appearance of the AI had a significant effect on trust levels across different cultural groups. The type of explanation the AI provided also influenced how users judged explanation quality, though appearance itself didn’t change quality perceptions.22ScienceDirect. How cultural cognition affects trust and perceived quality of AI explanations Separately, research on customer-service chatbots found that giving a chatbot more human-like design features, a name, an avatar, conversational language, served as an “effective trust shield” when the chatbot made errors.23ECIS 2021 Proceedings. Chatbots often Fail! Can Anthropomorphic Design Mitigate Trust Loss in Conversational Agents for Customer Service?
That anthropomorphism effect cuts both ways. On one hand, it can help users maintain a productive working relationship with a system that occasionally makes mistakes, much as you’d forgive a human colleague for an off day. On the other hand, it can lead to inappropriate levels of trust in a system that has fundamental accuracy problems. A chatbot that feels like a person is still software, and the social instincts it activates, give it the benefit of the doubt, assume good intentions, are imported from a context where they made more sense.
Measuring Trust and Why It’s Harder Than It Sounds
Researchers studying all these phenomena need tools to measure trust itself, and even that isn’t straightforward. A validation study found that the Trust in Automation Scale, one of the most widely used instruments, is a reliable measure of AI trust, but at 12 items it’s often too cumbersome for real-time or repeated measurement. A three-item short form demonstrated similar predictive power for whether people would follow an AI’s recommendation.24PubMed Central. Measuring trust in artificial intelligence: validation of an established scale and its short form A separate effort to develop a trust scale specifically for AI in online education found that trust has at least five distinct dimensions, and the instrument explained about 63% of the variance in how students felt about AI learning tools.25PubMed Central. Trust in AI: scale development and validation for online distance learners
The measurement challenge matters because “trust” in casual conversation is treated as one thing, but in practice it splinters into components: do you think the system is competent, do you think it’s honest, do you think it has your interests in mind, do you think it will adapt to new situations? Two people with the same overall trust score can differ wildly across those dimensions, and those differences predict different behaviors. Someone who trusts AI’s competence but doubts its benevolence will use it as a second opinion but never delegate fully. Someone who trusts its benevolence but doubts its competence might share personal data willingly but double-check every output. Collapsing those into a single “trust” score hides the details that matter most for designing better systems.