LLM

We are open sourcing our latest model, Kimi K2.6, featuring state-of-the-art coding, long-horizon execution, and agent swarm …

An agentic RL method that evolves the skill library through a fitness-driven lifecycle, enabling skills and the model to co-evolve throughout training.

We propose PRISM-Δ, a differential subspace steering method for prompt highlighting that matches or exceeds the best existing method on 19 of 20 configurations with relative gains up to +10.6%, while halving the fluency cost.

Probing hidden states during reasoning reveals that LLMs already know the answer before finishing thinking. We detect overthinking via ‘jumps’ and intervene during inference to improve reasoning.

Abstract: We present PromptCD, a test-time method for controlling LLM behavior without additional training. The approach creates paired positive and negative guiding prompts for a target behavior and contrasts model responses at the token-probability level for LLMs and through visual attention patterns for VLMs.

We introduce Kimi K2.5, an open-source multimodal agentic model designed to advance general agentic intelligence. The model focuses on …

Abstract: Long contexts challenge transformers as attention scores dilute across thousands of tokens and critical information is often lost in the middle. We reframe test-time adaptation as a budget-constrained memory consolidation problem and propose Gdwm (Gated Differentiable Working Memory), which introduces a write controller that estimates Contextual Utility, an information-theoretic measure of long-range contextual dependence, to allocate gradient steps efficiently.

Abstract: Reinforcement learning (RL) has shown great promise in enhancing LLM reasoning, but current approaches mainly focus on single domains with verifiable rewards. We propose RGR-GRPO, a rubric-driven RL framework for multi-domain reasoning that uses rubrics to provide fine-grained reward signals and offline guidance.

We present the first comprehensive analysis of how the order of graph descriptions impacts LLM performance, evaluating four graph description orders across six graph problems using six mainstream LLMs.

We introduce Kimi K2, a Mixture-of-Experts large language model with 32 billion activated parameters and 1 trillion total parameters. …

Abstract: Large language models (LLMs) have achieved remarkable progress, demonstrating unprecedented capabilities across various natural language processing tasks. However, the high costs associated with such exceptional performance limit the widespread adoption of LLMs, highlighting the need for prompt compression.

Large Language Models (LLMs) have made remarkable breakthroughs in reasoning, yet continue to struggle with hallucinations, logical …

We present the first comprehensive analysis of the impacts of CoT prompting on Reasoning LLMs, finding that one-shot CoT consistently enhances performance and reduces excessive reflections by approximately 90%.

作者提出CoT-Influx方法，一种对CoT的示例和内容进行优化从而提高LLMs推理能力的方法，其核心思想是通过剪枝最大化有效信息的输入。

Abstract: The current research on text-guided 3D synthesis predominantly utilizes complex diffusion models, posing significant challenges in tasks like terrain generation. This study ventures into the direct synthesis of text-to-3D terrain in a zero-shot fashion, circumventing the need for diffusion models.

在本文，作者提出了一个新的数据集MaliciousInstruct，一种模型回答毒性评估方式，一种通过操纵解码超参数的攻击手段——generation exploitation，一种对齐策略——generation-aware alignment

本文提出了一种新的方法，全面评估大模型多个候选答案的可信度，以减轻大模型对于错误答案的过度自信。

本文所属系列为笔者学习陈天奇和J.Zico Kolter在CMU开设的Deep Learning Systems的课程笔记。

3月最新用于游戏的大模型Agent综述

Word2Vec作者Tomas Mikolov二月最新关于大模型综述

这篇文章将用编码复现Transformer架构中使用的自注意力机制。

在过去CoT有两种范式，一种是Zero-shot，在问题末尾添加"Let’s think step by step"。另一种Manual CoT(Few-shot CoT)，每个例子由问题和推理链组成。第二种方法表现是否好取决于CoT写的好不好，不过这需要人手工来写。本文通过提出Auto-CoT这一方法使得Few-shot CoT可以自动生成，解放双手！

思维链(Chain of thought - CoT)在过去的实践中已经证明对提升大模型的推理能力有显著帮助。然而，目前还没有一项工作解释思维链长度与推理能力之间的关系。本文围绕这一核心问题，围绕CoT做了系统实验，并给出许多有意思和反直觉的结论。

本文揭示了大模型并不存在「涌现」，大模型的能力随着参数的增长是线性增长的。

如今AI的能力越来越强，接近甚至已经超越人类的能力，人类已经很难监督这类超人AI了。那么该如何监督比人类更强大的超人AI呢？

This paper explore the impact of encoding global and local graph structures using different modalities, particularly focusing on node classification tasks

提出了一种基于大模型的图引导的面向多步推理问题的推理方式。本文的主要贡献有两点：提出上述推理方式，提出允许变量定义的用于知识三元组提取的上下文学习方法

在本文,作者跟据图结构的特点提出了一种将图映射到文本空间的框架GRAPHTEXT

LLM ReAct范式，在大语言模型中结合推理和动作

过去的研究证明，人类拥有一个内部的世界模型，使人类能够模拟行动及其对世界状态的影响，以进行复杂任务的有意识的规划，包括运动控制、想象、推理和决策。而大模型只能通过自回归的方式进行推理，为此作者将强化学习和蒙特卡洛树搜索引入大模型推理

本文首次尝试使用大模型来解决图问题。在本文作者基于自然语言提出了含有千余个图基础问题的BenchMark NLGraph。作者在GPT3/4上进行实验获得了一些结论。在论文的最后，作者提出了两个可以提升大模型处理图问题的方法。

尽管在自动推理在自然文本方面取得了显著进展，但大型语言模型（LLMs）上的图推理效果依然不尽人意。在本文，作者团队进行了首次全面的研究，将图结构数据编码为文本，以供LLMs推理

大语言模型（LLM）的领军地位引发了大模型系统到底只是对海量数据进行肤浅地统计还是真正学习到了世界模型的争论。在本文，作者以llama2为实验对象，在三个空间数据集和三个时间数据集上进行了实验发现了后者存在的证据。

LLM.int8的进一步研究及Emergent feature现象

大模型量化基础知识及LLM.int8量化