Causal Evidence for Dual Mechanisms in LLM In-Context Graph Learning

A new research paper titled 'Belief or Circuitry? Causal Evidence for In-Context Graph Learning' investigates how Large Language Models (LLMs) perform in-context learning. The study addresses whether models rely on simple pattern-matching of recent tokens or infer latent structural information. Using a toy graph random-walk task with two competing structures, the authors provide evidence that neither explanation is sufficient alone. Principal Component Analysis (PCA) reveals that both graph topologies are encoded simultaneously in orthogonal subspaces, contradicting pure local transition copying. Furthermore causal interventions, including residual-stream activation patching and linear steering, demonstrate that late-layer patching transfers graph preferences while steering moves predictions as intended. These findings support a dual-mechanism account where genuine structure inference and induction circuits operate in parallel. This research contributes to the understanding of internal representation structures in artificial intelligence, offering insights into the cognitive-like capabilities of modern language models.