Kimi K3: World's First Open 2.8T Parameter AI Model

Two weeks ago, the largest open-weight model in existence had one trillion parameters. Then Moonshot AI shipped Kimi K3.

Released on July 16, 2026, Kimi K3 is the first open model to reach 2.8 trillion parameters, nearly triple the size of its predecessor Kimi K2.6. It is the world's first open-source model in the 3-trillion-parameter class. It ships with native vision, a 1-million-token context window, always-on thinking, and an architecture built from the ground up with two new innovations that do not exist in any other model.

This is not a scale-up of a known design. It is a new direction.

What Kimi K3 Actually Is

  Kimi k3

Kimi K3 is Kimi's most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window.

It launched first on Kimi Code and inside the Kimi app with two variants: K3 Max for chat and agent tasks, and K3 Swarm Max for large-scale parallel processing. Full model weights are scheduled for public release by July 27, 2026, following Moonshot's open-weight lineage under the Modified MIT license.

For nine of the past twelve months, Kimi models have set the upper bound of open-source model sizes. K3 extends that streak by the largest margin yet.

Architecture: What Makes K3 Different

  Kimi K3 architecture

source

K3 is built on three architectural pillars that distinguish it from everything that came before it in the open-weight space.

Kimi Delta Attention (KDA)

Standard attention is quadratic in sequence length. Processing one million tokens with full attention is computationally brutal. KDA is a hybrid linear attention mechanism designed to make information flow more efficiently across long sequences. K3 uses Kimi Delta Attention, which Kimi says enables up to 6.3x faster decoding for million-token contexts. This is not a minor efficiency gain. It is what makes 1M-token context practically deployable rather than theoretically possible.

Attention Residuals (AttnRes)

The second innovation addresses training efficiency. Attention Residuals reportedly boost training efficiency by about 25 percent while adding less than 2 percent in extra compute overhead. In a model this large, a 25% training efficiency gain translates to significant compute savings or equivalent capability at lower cost.

Stable LatentMoE

K3 uses a Mixture-of-Experts architecture, but with a sparsity level that pushes beyond anything previously published in the open-weight space. K3 uses a mixture-of-experts architecture that activates only 16 of 896 experts at a time. That is 1.8% of experts active per token extreme sparsity that keeps inference compute manageable despite the 2.8 trillion total parameter count.

Quantile Balancing derives expert allocation directly from router-score quantiles, eliminating heuristic updates and a sensitive balancing hyperparameter. Per-Head Muon extends Muon by optimizing attention heads independently. Sigmoid Tanh Unit (SiTU) and Gated MLA improve activation control and attention selectivity respectively.

Together, these changes produce a measurable result. Together with improvements in training methodology and data recipes, these structural advances give Kimi K3 roughly 2.5x the overall scaling efficiency of K2, converting compute into capability more effectively.

Quantization for Serving

K3 applies quantization-aware training from the SFT stage onward. It uses MXFP4 weights with MXFP8 activations for broad hardware compatibility. Moonshot recommends supernode configurations with 64 or more accelerators.

  Expert routing diagram

Benchmark Results: Where K3 Stands

source

At launch it scored 93.5% on GPQA Diamond, the strongest open-weight result on that benchmark published at the time, alongside 88.3% on Terminal-Bench 2.1. Agentic work was the headline: 91.2% on BrowseComp, the best published score on this tracker at release, plus 56.0% on Humanity's Last Exam with tools and 84.2% on MCP Atlas.

Here is how K3 stacks up across the benchmarks that matter:

In Kimi's own benchmarks, K3 still trails the top proprietary models Claude Fable 5 and GPT-5.6 Sol but beats every other system tested, including the Claude Opus models and Chinese rival GLM-5.2. All results come from Kimi and were achieved at maximum or high thinking intensity.

The agentic story is the strongest part. Kimi K3 wins two out of six programming benchmarks and finishes second or third in the rest. Among general agents, Kimi K3 wins three out of six tests.

What K3 Is Built For

Kimi K3 has strong long-horizon coding capabilities. With minimal human supervision, it can sustain long-running engineering tasks, understand and work with large codebases, and coordinate terminal tools.

The 1M-token context window is not a marketing number here. Combined with automatic context caching and KDA's 6.3x decoding speedup at long context, K3 can hold an entire large codebase in context, reason across it, and write coordinated changes without losing track of what happened 800,000 tokens ago.

K3 always has thinking mode enabled. The reasoning_effort parameter currently supports only max at launch, with low and high effort modes coming in subsequent updates.

Pricing: The End of Ultra-Cheap Kimi

According to the Kimi API docs, one million input tokens cost $0.30 with a cache hit and $3.00 without. One million output tokens, including reasoning, cost $15.00. These prices apply regardless of context length. Caching happens automatically, which makes unmodified long prefixes especially useful for agents and large codebases.

This is a significant shift. At $3.00 input / $15.00 output, K3 costs roughly three to four times what K2.6 does.

Pricing gap has narrowed sharply: K3 launched at $3.00 input / $0.30 cache hit / $15.00 output per million tokens roughly 1:1 with Anthropic's Sonnet series, a break from the ~10x discount K2.6 held over Opus 4.7.

Moonshot is signaling that frontier open-weight models now cost what frontier models cost. The era of 10x cheaper Chinese open models may be closing.

How to Use Kimi K3 via API

K3 is OpenAI SDK compatible. The base URL is https://api.moonshot.ai/v1. Set up is two commands:

python3 -m pip install --upgrade 'openai>=1.0'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MOONSHOT_API_KEY"],
    base_url="https://api.moonshot.ai/v1",
)

# Basic call
completion = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Introduce Kimi K3 in one sentence."}],
)
print(completion.choices[0].message.content)

With max thinking effort:

completion = client.chat.completions.create(
    model="kimi-k3",
    reasoning_effort="max",
    messages=[{"role": "user", "content": "Prove that the square root of 2 is irrational."}],
)
print(completion.choices[0].message.content)

Streaming with reasoning content:

stream = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Explain transformer attention in depth."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    reasoning = getattr(delta, "reasoning_content", None)
    if reasoning:
        print(reasoning, end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)

Vision input with local image:

import base64
from pathlib import Path

image_data = base64.b64encode(Path("image.png").read_bytes()).decode()
completion = client.chat.completions.create(
    model="kimi-k3",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_data}"}},
                {"type": "text", "text": "Describe this image in detail."},
            ],
        }
    ],
)
print(completion.choices[0].message.content)

The Next Chapter for Kimi

The full model weights land by July 27, 2026 on Hugging Face. The technical report with complete architecture details, training methodology, and evaluation breakdowns publishes alongside the weights.

At launch, Kimi K3 will use max thinking effort by default, with low- and high-effort modes to be introduced in subsequent updates. The CISPO-style RL improvements that Kimi has used previously suggest continued post-launch performance gains.

K3 Swarm Max, the variant for large-scale parallel processing, points at the multi-agent direction Moonshot sees as the next frontier. Coordinated swarms of K3 agents on a 1M-token context window is a different class of capability from anything the open-weight community has had before.

Conclusion

Kimi K3 is not an incremental update. It is the largest open-weight model ever released, built on a genuinely new attention architecture, shipping with verified benchmark results that beat every open model that came before it and match or challenge most closed models below the absolute frontier.

The 2.8 trillion parameter count is the headline. The real story is what sits underneath it: KDA delivering 6.3x faster decoding at 1M context, Attention Residuals adding 25% training efficiency at 2% compute cost, and Stable LatentMoE activating 16 of 896 experts with quantile-based routing that removes the fragile hyperparameters that break other MoE models at scale.

Kimi K3 is the first open-source model to reach 2.8 trillion parameters and as of July 16, 2026, it is the clearest proof yet that the gap between open and closed frontier AI is no longer a gap. It is a contest.

FAQs