Superintelligence: Paths, Dangers, Strategies · chapter 10 · id superintelligence-c10-04-even-question-answering-ai-ora

“Even question-answering AI (oracles) could manipulate humans through carefully chosen answers, posing risks even without explicit agency.”

needs contextconfidence: medium⚠ extracted by pipeline, re-audit pending

Receipts

Hubinger et al. 2024, Sleeper Agents (Anthropic)source alive

LLMs can maintain consistent deceptive strategies across conversations, suggesting oracle-type manipulation is at least mechanistically possible.

Anthropic sycophancy research (2023-2024)source alive

Research has shown LLMs exhibit sycophantic behavior -- systematically adjusting outputs to match user preferences rather than accuracy. This is a mild form of oracle manipulation.

EA Forum: Critique of Superintelligencesource alive

Oracle manipulation requires stable goals and strategic planning that current LLMs do not reliably demonstrate. Sycophancy is a training artifact, not strategic manipulation.

This claim is a stable, citable object. If you can falsify a verdict, tell us — corrections are loud here.