Interpretability

Do Large Language Models Already Know the Answer Before They Finish Thinking?

Probing hidden states during reasoning reveals that LLMs already know the answer before finishing thinking. We detect overthinking via ‘jumps’ and intervene during inference to improve reasoning.

Yuyao Ge 葛钰峣, Shenghua Liu, Yiwei Wang, Tianyu Liu, Lingrui Mei, Baolong Bi, Jiayuan Guo, Jiayu Yao, Jiafeng Guo, Xueqi Cheng