Chain-of-Thought Deep Dive

When explicit reasoning helps, when it hurts, and how to structure it well

Chain-of-thought isn’t always the answer

“Think step by step” became a famous prompting trick because for certain tasks it measurably improves accuracy. But treating it as a default you bolt onto every prompt is a mistake advanced users grow out of. Chain-of-thought (CoT) helps some tasks a lot, does nothing for others, and can actively hurt a few. Knowing which is which is the actual skill.

Where CoT genuinely helps

CoT reliably improves performance on tasks with multiple dependent steps, where an error early in the reasoning would otherwise silently propagate: multi-step math, logic puzzles, multi-clause contract analysis, debugging a chain of cause and effect, or any task where the answer depends on getting an intermediate step right before moving to the next.

Without CoT:
"Is this pricing plan profitable at 10,000 users?"

With CoT:
"Walk through this step by step:
1. Calculate total revenue at 10,000 users
2. Calculate total infrastructure cost at that scale
3. Calculate total support cost at that scale
4. Sum total costs and compare to revenue
5. State whether it's profitable and by how much
Show your work for each step before the final answer."

The second version catches errors you can inspect — if step 2’s cost estimate looks wrong, you know exactly where the final answer went astray.

Where CoT does nothing or actively hurts

For tasks that are more about judgment, taste, or fluency than sequential logic — writing a tagline, picking a tone, summarizing a well-understood document — forcing step-by-step reasoning can make output more mechanical and less natural. Explicit reasoning is built for problems with a right answer reached through steps; it’s a poor fit for problems where the “right answer” is a matter of feel.

Extended thinking vs. prompted reasoning

It’s worth distinguishing two different things that both get called “chain-of-thought”: asking Claude to show its reasoning in the visible response text (prompted CoT, what this lesson mostly covers), versus Claude’s extended thinking mode, which reasons in a separate space before producing the final answer. Extended thinking is suited to genuinely hard problems where you want maximum reasoning depth without cluttering the final answer with the scratch work. Prompted CoT in the response itself is what you reach for when you specifically want to see and audit the intermediate steps — like the pricing example above, where you want to inspect step 2’s assumption before trusting the conclusion.

Structured reasoning beats generic reasoning

As covered in Lesson 1, “think step by step” is weaker than a specific structure. For CoT specifically, the highest-leverage move is defining the actual steps rather than leaving them open-ended — it constrains where Claude’s reasoning can drift and makes runs more consistent with each other.

A decision rule

Before adding CoT to a prompt, ask: does this task have dependent steps where an early error would propagate? If yes, use structured CoT. If the task is closer to “produce something that sounds right” than “arrive at a provably correct answer,” skip it — or worse, test both and see which output you actually prefer.