The ChatGPT source cue shapes answer acceptance in a reasoning task

As generative artificial intelligence systems weave themselves deeper into the fabric of daily academic, professional, and personal decision-making, individuals are increasingly forced to judge externally provided solutions rather than derive answers independently. A recent empirical study led by researchers Rosa Angela Fabio, Emanuela Rantuccio, and Federica Bruno sheds light on this psychological phenomenon, investigating how the mere attribution of an answer to ChatGPT versus a generic computer influences human evaluative judgment. Published in Frontiers in Psychology, the research reveals that people are significantly more likely to accept mathematical and logical answers when they believe the output originated from ChatGPT, particularly when dealing with straightforward arithmetic.

Experimental Design and Methodology

To explore how source attribution alters cognitive evaluation under conflict, the researchers implemented a rigorous within-subjects design. A sample of 103 participants—comprising university students, doctoral candidates, and professionals across education, healthcare, administration, and public services—evaluated a series of 30 pre-generated problems. These items were adapted from the classic Cognitive Reflection Test (CRT), originally developed by Shane Frederick to measure a person’s ability to override an initial, intuitive incorrect response in favor of a reflective, analytically correct solution.

Participants were not asked to solve the math problems from scratch. Instead, they were shown logical-mathematical questions paired with pre-generated answers. Every trial displayed a visually identical interface, differing only in the accompanying source label and icon: either "ChatGPT" or a generic "Computer." The test battery consisted of 15 pairs of parallel problems sharing identical logical structures but varied numerical values. An independent pretest involving 35 participants confirmed the strict comparability of these problem batteries prior to the main experiment.

The trials were categorized into three distinct groups: reflective-correct answers (where a deliberate solution replaces an intuitive trap), intuitive-incorrect answers (compelling traps like the classic bat-and-ball problem), and control-correct answers (straightforward arithmetic computations). Throughout the online administration hosted via PsyToolkit, researchers recorded binary correct-or-incorrect judgments along with precise response latencies.

Key Findings and Statistical Insights

When analyzing the behavioral data, the research team uncovered notable discrepancies in how participants endorsed answers based solely on labels. Overall acceptance rates were higher under the ChatGPT attribution banner, standing at 68.0% compared to 63.4% for the generic computer label.

A granular trial-level logistic mixed model further revealed a striking source-by-category interaction. While the source contrast proved statistically insignificant for the cognitively demanding CRT categories, it was exceptionally pronounced for simple arithmetic control questions. Answers to basic math problems were more than six times as likely to be accepted when attributed to ChatGPT than when attributed to a generic computer (Odds Ratio = 6.08).

Signal-detection theory (SDT) was applied to separate actual sensitivity from response bias within the CRT problem sets. Interestingly, the analysis detected no significant difference in perceptual sensitivity ($d’$) or response criterion ($c$) between the two source labels. Participants struggled equally to discriminate between correct and incorrect answers regardless of whether the system was labeled as ChatGPT or a computer, maintaining near-zero sensitivity in the conflict-heavy CRT categories. Furthermore, while raw response times indicated that evaluations took roughly one second longer under the ChatGPT label (averaging 12,557 milliseconds versus 11,514 milliseconds), this temporal discrepancy disappeared after natural-log transformation, suggesting cautious interpretation is required regarding evaluation latency.

Contextualizing Human-AI Interaction

The psychological implications of these findings intersect with broader frameworks of automation bias, algorithm appreciation, and dual-process theories of reasoning. Previous literature has documented polarized reactions to artificial intelligence. On one hand, studies on algorithm appreciation have shown that individuals often favor algorithmic advice over human judgment. On the other hand, recent work examining algorithm aversion demonstrates that observing systemic errors can severely dampen a user’s trust and willingness to rely on automated forecasts.

However, the new study isolates a specific nuance: when presented with fixed, non-iterative outputs, users tend to grant a higher baseline of endorsement to a branded generative AI conversational tool compared to a nondescript computational source. This uncritical acceptance of routine outputs points toward potential risks in professional and educational workflows where individuals outsource verification to large language models.

To explore whether stable individual differences drove these effects, the researchers administered several self-report instruments: the Human-Computer Trust Measure (HCTM), the General Attitudes toward Artificial Intelligence Scale (GAAIS), and the Critical Thinking Attitude Scale (CTAS). Exploratory correlational analyses revealed no significant linear associations between source-related acceptance differences and broad constructs such as trust in AI, general positive or negative attitudes toward technology, or dispositional critical thinking.

Broader Implications and Future Research Directions

As generative AI tools become ubiquitous assistants in classrooms, boardrooms, and research laboratories, the tendency to defer to branded conversational systems carries profound societal consequences. The research underscores a critical vulnerability in human-computer interaction: individuals may readily endorse correct or plausible outputs due to heuristic reliance on prestigious or conversational brand cues, while failing to effectively filter out sophisticated errors embedded in complex reasoning tasks.

The authors note several limitations inherent in their work, including a convenience sample, a modest trial count per condition, and the lack of a fully crossed design separating answer validity from problem categories. Future inquiries are encouraged to investigate how live interactive dialogues, feedback loops, and varying levels of user expertise modulate these source attribution effects.

Ultimately, the study serves as a timely reminder that human evaluative judgment is heavily colored by superficial source cues. As society navigates an era defined by synthetic intelligence, cultivating robust verification habits remains essential to ensuring that convenience does not override analytical rigor.