Superficial Reflection or Genuine Thought? A Fine-Grained Cognitive Analysis of Large Reasoning Models

arXiv:2512.00729v2 Announce Type: replace Abstract: Motivated by the observed human-like behaviours in Large Reasoning Models (LRMs), this paper introduces a comprehensive taxonomy to characterise atomic reasoning steps and analyse the reasoning behaviours of LRMs. Grounded in human cognitive processes, we propose a taxonomy comprising five groups and seventeen categories. Through this taxonomy, we conduct an in-depth analysis of contemporary LRMs and distil four actionable takeaways for model optimisation. Most notably, we reveal that prevailing post-answer ``doublechecks'' are largely superficial and rarely yield substantive revisions. A targeted intervention further shows that explicitly eliciting richer reflection processes can substantially improve failed self-correction. To support this largescale study, we propose CAPO, an automated annotation method used to construct a dataset of 277,534 reasoning steps with strong agreement with human expert annotations. We further validate the main behavioural patterns on a newer reasoning model and a coding domain, demonstrating the broader applicability of the proposed taxonomy. All source code and data are available at https://github.com/hehepig4/psyche.

Related

• ReflCtrl: Controlling LLM Reflection Efficiently via Representation Engineering
• Cognitive Profiling of LRMs' Reasoning Traces Using Bloom's Taxonomy
• Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
• PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoning

Source: arXiv cs.AI | 2026-10-01