Thinking Machines Lab Releases Inkling: A 975B-Parameter Open-Weights Multimodal MoE With 41B Active Parameters And Controllable Thinking Effort

Thinking Machines Lab just released Inkling, their first model trained from scratch, weights are open, fine-tunable on Tinker. The lab pitches it as a base for customization.

What is Inkling?

Inkling is a Mixture-of-Experts transformer with 975B total parameters and 41B active. It supports a context window of up to 1M tokens. Pretraining covered 45 trillion tokens of text, images, audio, and video. Inputs accept text, images, and audio; output is UTF-8 text only.

The research team also previewed Inkling-Small, a 276B-parameter MoE with 12B active parameters. It matches or exceeds its larger sibling on many benchmarks, and its weights arrive once testing finishes. Because customization/finetuning is the key differentiator, the architecture matters here very much.

Inside The Architecture

The model architecture includes a 66-layer decoder-only transformer with a sparse MoE feed-forward backbone. Each MoE layer holds 256 routed experts plus 2 shared experts. Six routed experts activate per token, and both shared experts activate on every token. A sigmoid-based router handles selection, using an auxiliary-loss-free load-balancing bias. Routed and shared scores are normalized jointly, then used to weight combined outputs. The MoE design largely follows DeepSeek-V3.

Attention departs from convention. Sliding-window and global layers interleave at a 5:1 ratio with 8 KV heads. Position uses a relative positional embedding rather than RoPE, which the lab reports extrapolates better. Short convolutions are applied after key and value projections, and on residual branch outputs.

Multimodality is encoder-free. Audio enters as dMel spectrograms, and images become 40×40 pixel patches through a four-layer hMLP. A lightweight embedding layer projects both, then the decoder processes them jointly with text tokens.

Training used Muon for large matrix weights and Adam for other parameters, on NVIDIA GB300 NVL72 systems. Post-training bootstrapped from SFT on synthetic data, including data generated by Kimi K2.5. Most compute went to asynchronous RL, scaled past 30M rollouts, improving log-linearly throughout. That RL run also produced the model’s main control surface.

Controllable Thinking Effort

During RL, the research team set effort by changing the system message and adjusting per-token cost. The model consequently learned to spend different token budgets on different rollouts. The release post sweeps effort from 0.2 to 0.99, and harnesses can set it directly. In transformers, the same control is exposed as a reasoning_effort argument with named levels.

The efficiency data is quite specific. Inkling spends one third as many tokens as Nemotron 3 Ultra for equal Terminal Bench 2.1 performance. Cost and latency become tunable per call, not fixed per model.

Alongside effort, the research team targeted trustworthiness directly.

Performance

All Inkling evals run at effort=0.99 and temperature 1.0, with a 256K trajectory limit for coding. Several scores are externally reported by Artificial Analysis. Against open-weights peers, the picture is quite competitive.

Inkling leads this open-weights group on FORTRESS Adversarial at 78.0%. It trails GLM 5.2 on Terminal Bench 2.1 by 18.9 points. It reports 73.5% on MMMU Pro and 91.4% on VoiceBench. It posts 1257 on Design Arena’s Agentic Web Dev leaderboard, a blinded human evaluation.

With numbers established, deployment becomes the practical question.

Running and Fine-Tuning Inkling

Two checkpoints ship. BF16 needs at least 2 TB aggregated VRAM: 8x NVIDIA B300 or 16x H200. NVFP4 drops that to at least 600 GB, running W4A4 on 4x B300 or W4A16 on 8x H200. Runtimes include SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face transformers.

OpenAI-compatible serving takes one command:

For fine-tuning, Inkling is live on Tinker with 64K and 256K context options. The research team also released tml-renderers for post-training with tool calls and multimodal inputs. Hosted APIs exist via TogetherAI, Fireworks, Modal, Databricks, and Baseten.

Given those constraints, three deployment patterns follow.

Where Inkling Fits: Use Cases

• Voice-and-vision agents: A major design goal was backing the lab’s interaction models system. A support agent could ingest a 16kHz WAV call plus a screenshot, then emit a structured ticket.
• Cost-tiered agentic pipelines: Low effort handles routing and triage; max effort handles the hard repair step. One deployment, two budgets.
• Domain fine-tuning.:The lab cites financial-judgment work where fine-tuning closed the generalist gap. Chart-heavy analytics fits too, given 82.0% on CharXiv RQ with Python.

Taken together, the trade-offs are clear.

Strengths And Weaknesses

Strengths

• Apache 2.0 weights, 1M-token context, native text, image, and audio input.
• Controllable effort matches Nemotron 3 Ultra’s Terminal Bench score at a third of the tokens.
• Highest FORTRESS Adversarial score (78.0%) among the compared open-weights models.
• Day-0 support across transformers, SGLang, vLLM, llama.cpp, plus five hosted APIs.
• A multi-token-prediction drafter ships for speculative decoding.

Weaknesses

• Trails GLM 5.2 and Kimi K2.6 on HLE, Terminal Bench 2.1, and SWEBench Verified.
• BF16 needs 2 TB aggregated VRAM; NVFP4 W4A4 additionally requires SM100+ hardware.
• SimpleQA Verified at 43.9% sits well below DeepSeek V4 Pro’s 57.0%.
• Inkling-Small weights are unreleased, and there is no audio or image output.
• Terminal Bench 2.1 numbers use an internal harness, unlike self-reported competitor scores.
• The project page flags role-play and indirect prompts as residual safety risks.

Sources