LongCat-2.5Native Multimodality for Agentic Tasks2026-11-1 GitHub Coming soon HuggingFace Try it API AccessWe introduce LongCat-2.5, a native multimodal model for long-horizon agentic tasks. With 1.6 trillion total parameters and approximately 48 billion activated per token, it builds on LongCat-2.0 with stronger native visual understanding and agentic coding.We incorporate more long-horizon trajectories into long-sequence training and use large-scale long-horizon reinforcement learning to strengthen end-to-end engineering capabilities across complex codebase development and full-project builds. We continue to scale this training pipeline through data synthesis, environment construction, and mixed training across harnesses. We also convert real-world agent failure trajectories into reproducible, verifiable training tasks, continually adding targeted training data to address model capability gaps.LongCat-2.5 also represents visual signals as multi-level discrete tokens, enabling the backbone to process visual and textual inputs in a unified token-based format.
Combining native visual understanding with agentic coding delivers strong visual coding performance, while also supporting creative design and 3D tasks.These advances are supported by continued infrastructure improvements. We optimize long-context training and W4A4 inference on domestic AI accelerators, while numerical consistency between training and inference and long-context rollout optimizations enable efficient large-scale long-horizon RL.Metrics DashboardTo provide a clear view of LongCat-2.5’s current capabilities, the charts below compare it with GPT-5.5, GPT-5.6-Sol, Claude-Opus-4.8, and Claude-Opus-5. Detailed evaluation results covering a broader set of models are presented in the table later in this post.ShowcasesEvery case below was generated end-to-end by LongCat-2.5, spanning software development and professional office scenarios — from real-time fluid simulation, 3D digital exhibitions, and city dioramas to business analysis, feedback reports, and event operations. Click any case to watch the full demo.
Software Development · Web VisualDEMO 01 / 08Demo footage coming soonBuild a real-time fluid simulation and VFX tool with LongCat-2.5 that generates high-FPS, interactive webpages with live fluid dynamics, enabling developers, designers, and science communicators to visualize complex physical phenomena interactively and beautifully.Visual Primitive ModelingLongCat-2.5 represents visual signals as multi-level discrete tokens, enabling the backbone to process visual and textual inputs in a unified token-based format. This design combines native-resolution visual tokenization, offline visual preprocessing, and visual embeddings learned jointly with the backbone, supporting native multimodal integration while simplifying visual computation and workload management during training.Discrete Native Resolution Vision Transformer: To bring visual primitive modeling to a token-level abstraction analogous to language modeling, we represent visual signals as multi-level vocabulary tokens. We overcome the information bottleneck of visual discretization by applying Residual Vector Quantization (RVQ) to native-resolution features from a Semantic-and-Aligned Encoder (SAE), as in LongCat-Next.Figure 1. Native-resolution visual tokenization: images at different resolutions are encoded as discrete token sequences and reconstructed by a visual de-tokenizer.Offline Visual Preprocessing: To reduce online visual computation and workload heterogeneity, a fixed tokenizer allows visual token IDs to be precomputed and cached offline.
The backbone therefore processes pretokenized sequences, simplifying long-sequence packing and partitioning and improving load balancing during distributed training.Native Visual Representation: Visual codebook embeddings are learned jointly with the backbone. Preliminary validation on a smaller MoE model with approximately 3B activated parameters on average shows interwoven visual and textual embeddings in t-SNE visualizations, suggesting integration within a shared semantic space.Figure 2. t-SNE comparison of visual and textual embeddings. The right panel shows validation on a MoE model with approximately 3B activated parameters on average.Scaling Agentic Reinforcement LearningRL scaling is a key lever for pushing model intelligence further. For LongCat-2.5, we scale an end-to-end agentic RL pipeline across data, environments, algorithms, and harnesses for long-horizon tasks.Figure 3.
The agentic RL pipeline for long-horizon tasks, spanning data and environments, asynchronous training, reward design, and mixed training.Large-Scale Data Synthesis and Environment Construction: For agentic tasks such as coding, the model learns from real environment feedback. A scalable synthesis pipeline automatically generates training tasks, interactive environments, and verifiable tests. We filter GitHub pull requests and linked issues, then use agents to annotate tasks and synthesize environments, validated with fail-to-pass (F2P) and pass-to-pass (P2P) tests. Real internal coding tasks and evaluation criteria broaden coverage of everyday engineering needs.
Live rollout pass rates guide re-labeling and filtering, while offline analysis of false-positive and false-negative trajectories continually corrects and extends F2P tests to cover task specifications more fully and improve verification accuracy.Asynchronous RL and Stability: To improve training efficiency, we adopt asynchronous partial rollouts: each RL step maintains an oversampling ratio, and once a full batch is collected, in-flight trajectories are interrupted, saved, and resumed in the next sampling phase. We increase oversampling based on precomputed pass rates and filter out zero-gradient groups whose rollouts all succeed or all fail, ensuring sufficient effective gradient signal. Tokens within one trajectory may be generated by different historical checkpoints, introducing staleness. We clip the discrepancy between sampling-policy and current-policy log-probs and rescale each token’s gradient contribution using accumulated historical discrepancy, bounding the gradient bias from staleness.
Same-weight numerical differences are handled by the train–inference alignment described in the infrastructure section.Reward Hacking and Output Behavior: We disable network access in task environments, remove git history and cached files, and use an LLM judge to audit trajectories for exploits, curbing opportunistic behavior. Additional penalties suppress repetitive reasoning and repeated tool calls during multi-turn interaction. To counter overthinking and overly rapid growth in response length, we redistribute reward among positive samples.Mixed Training and Generalization: The model readily adapts to similar tasks and harnesses during RL. Only a small number of scaffold adaptation examples are needed in earlier training stages.
During RL, we mix training tasks aligned with different benchmarks and scale their corresponding scaffolds in varying proportions. Mixed training across tasks and scaffolds improves generalization, with performance gains across tasks reinforcing one another.Learning from Multiple TeachersTo integrate specialized capabilities into the main model, we retain LongCat-2.0’s expert groups for agent capabilities, reasoning, and interaction, covering multi-turn task execution, single-turn reasoning, fine-grained instruction following, and safety without compromising usefulness. During MOPD, the student samples online across task domains, and domain experts train it through full-vocabulary KL divergence on the sampled outputs.Samples must reflect the student’s actual task behavior while allowing teachers to provide reliable signals. To limit distribution shift, prior work uses per-turn prefixes from trajectories seen by teachers during SFT or RL as contexts for student sampling.
We further split each teacher response into fixed-length segments to form per-cut prefixes and cap student self-sampling at the same length to address distribution shift within long single-turn responses.Figure 4. Per-cut prefixes for online student sampling.To control computation and storage costs, prefixes from the same teacher trajectory are grouped on one inference engine, sharing prefill computation and KV cache. For training, we assemble a tree with the teacher trajectory as the trunk and student samples as branches. Tree Attention completes OPD training for all samples in the group in one forward and backward pass, avoiding redundant computation and storage of the teacher trajectory.Long-Context Training on Domestic AI AcceleratorsLongCat-2.5 uses long-sequence training during both mid-training and SFT.
With less than 80 GB of memory per domestic AI accelerator, the training system prioritizes memory efficiency and aligns its parallelism with the supernode architecture.Supernode-Aware ParallelismLongCat-2.5 aligns its parallel groups with the supernode topology, with FSDP and EP degrees equal to the number of accelerators per supernode and a CP degree no greater than that number. This keeps parameter all-gather, expert token all-to-all, and KV all-gather within the high-bandwidth supernode interconnect.Hybrid vocabulary parallelism further reduces memory consumption by sharding the base embedding layer, LM head, and N-gram embeddings across the DP × CP domain.To address load imbalance in large-scale expert parallelism, LongCat-2.5 introduces threshold-based dynamic expert replication built on MoonEP. Replication is only enabled when EP load imbalance reaches a configured threshold, avoiding communication overhead that outweighs the load-balancing gains, destination-side OOM caused by routed tokens, or sender-side OOM caused by communications.The discrete image-token design simplifies parallelization by removing the vision encoder from the training graph. With image token counts known during data packing and shuffling, modality load ratios are predictable; this allows us to keep modality proportions approximately uniform across data-parallel ranks.FSDP and System OptimizationsLongCat-2.5 introduces dual-shard FSDP, which keeps model-parameter FSDP sharding within each supernode while further sharding FP32 master weights and optimizer states across FSDP domains through HSDP replicas, preserving efficient parameter communication and reducing persistent memory consumption.Parameters, gradients, and optimizer states are organized into contiguous flattened buffers to support uneven sharding and Muon optimizer sharding, while direct weight-gradient writes and temporary-buffer reuse reduce intermediate copies and peak memory usage.Zero-copy AllGather/ReduceScatter operations are involved to remove intermediate copies caused by default communication algorithms, while EP packs tokens and routing probabilities together for transmission and calls for AIV-based communication to alleviate AI-CPU task-expansion bottlenecks.
Parameter prefetching overlaps all-gathers with computation while limiting contention with CP, and ScMoE overlaps EP communication with dense-branch computation.Batched gradient operations, coalesced metric reductions, CPU core binding, and NUMA-aware scheduling reduce host overhead. Overall, the FSDP configuration achieves 86% higher per-device throughput than the DDP baseline in long-sequence SFT tasks.Figure 5. Long-context training on domestic AI accelerators.Infrastructure for Agentic RLNumerical Consistency Between Training and InferenceBuilding on IsoExec, GLM-5, and XoRL, we decouple the parallelism configurations of training, prefill, and decode while preserving bitwise-identical per-token log probabilities with near-zero performance overhead. Both engines use batch-invariant kernels and aligned framework logic, with backward support for training.
Enabled by default in experimental and production jobs, the system provides a numerical regression basel
Texto de longcat.ai tal como lo publicó, leído de su página y mostrado aquí con el formato de El Autómata. El original, en longcat.ai · Volver a la noticia de El Autómata