Sakana AI SSoT Fixes LLM Sampling Bias with Prompt-Only Entropy Injection
Sakana AI's String Seed of Thought (SSoT) technique, accepted at ICLR 2026, addresses a fundamental LLM failure mode: models asked for diverse outputs generate variations of the same answer rather than genuinely exploring the distribution. SSoT works by prompting the model to generate an internal random-looking string before the task and use it as a diversity seed—requiring no fine-tuning or external randomness. It reduces sampling bias across open and closed models; on some reasoning models, output distributions approach true random behavior. The technique significantly increases creative-writing diversity without quality loss and can replace retry logic or external samplers in production pipelines.
Why It Matters
A prompt-only fix for LLM sampling bias removes a significant friction point for agentic systems that rely on diverse candidate generation—brainstorming, A/B content, multi-path planning—without requiring any model modification or additional infrastructure.