Tencent Hy3 landed on July 6, 2026 as one of the largest open-weight language models any major lab has put under a permissive license this year, and its headline pitch is unusual: a 295-billion-parameter model that only lights up 21 billion parameters per token, yet claims to trade blows with rivals two to five times its active size. The release, reported by The Decoder and flagged by Simon Willison the same day, arrives under Apache 2.0, which means anyone can download the weights, run them locally, and ship products on top without asking Tencent for anything.
- Hy3 is a 295B-parameter mixture-of-experts model with 21B active parameters, a 256K context window, and an Apache 2.0 license.
- Tencent reports a blind evaluation by 270 experts in which Hy3 scored 2.67 out of 4 against GLM-5.1 at 2.51, and says it halved its hallucination rate to 5.4 percent.
- The weights are large, 598GB at full precision and 300GB in the FP8 build, so deployment realistically needs a multi-GPU box or a hosting provider.
What Tencent shipped in Tencent Hy3
The core spec sheet is easy to state and worth reading closely. Hy3 carries 295 billion total parameters but activates only 21 billion for any given token, a ratio that puts it firmly in the sparse mixture-of-experts camp. According to the model card on Hugging Face, the network stacks 80 transformer layers plus a single multi-token-prediction layer worth 3.8 billion parameters, routes through 192 experts with eight active at a time, and uses grouped-query attention with 64 query heads over 8 key-value heads. The hidden size sits at 4096, the vocabulary at 120,832 tokens, and the context window stretches to 256,000 tokens.
Those numbers matter because they set the economics. A dense 295B model would demand compute proportional to all 295 billion weights on every token. A sparse model of the same nominal size pays only for the experts it actually routes to, so the per-token cost tracks the 21 billion active parameters rather than the full count. That is the whole reason labs keep pushing MoE: you get the representational capacity of a huge model at close to the inference bill of a medium one. Tencent is leaning on exactly that trade, and the marketing line about matching models "two to five times its size" is a claim about that active-parameter comparison, not the total.
Willison, who tends to kick the tires on new releases with a fixed benchmark, ran his usual test and asked Hy3 to "generate an SVG of a pelican riding a bicycle." He got back a flat cartoon of a white pelican with an orange beak perched on a red bicycle against a pale blue background. It is a small, unscientific probe, but he has run it across dozens of models, and the fact that a fresh Chinese open model produces a coherent result on it says something about how far the floor has risen.
Inside the mixture-of-experts design
Not every mixture-of-experts model is built the same way, and Hy3's twist is where the engineering gets interesting. A companion write-up on Hugging Face describes what the author calls a differentiated expert size design: instead of 192 identical experts, Hy3 uses experts of varying capacities and routes tokens of different difficulty to experts sized to match. Easy tokens can go to small, cheap experts; harder tokens can be sent to larger ones. In a uniform MoE, every expert costs the same regardless of whether the token needs that horsepower, so a variable-size layout is a bet that you can spend capacity where it earns its keep.
There is a catch with any such scheme. A router left to its own devices tends to fall in love with the big experts, because sending everything to the highest-capacity option is a low-risk way to reduce training loss. To counter that, the Hugging Face write-up describes a P-penalty loss that penalizes the model's habit of favoring large experts and nudges it toward activating more of the small ones. Without that pressure, the differentiated design would quietly collapse back into an expensive uniform one during training.
The multi-token-prediction layer is the other piece worth understanding. Rather than predicting only the next token, an MTP layer is trained to look a step further ahead, and at inference time that extra head can drive speculative decoding, where the model drafts several tokens and verifies them in a batch instead of generating strictly one at a time. The model card notes that both vLLM and SGLang integrations support MTP speculative decoding for Hy3, which is the practical payoff: on suitable hardware, that can meaningfully lift tokens-per-second without changing the output distribution.
The attention setup is tuned for the same efficiency goal at long context. Hy3 uses grouped-query attention, sharing a small pool of eight key-value heads across all 64 query heads, which shrinks the key-value cache that a model has to keep in memory as a conversation grows. That choice is what makes a 256,000-token window plausible to serve rather than merely advertise, because the memory a transformer spends on attention scales with the number of key-value heads times the sequence length. A quarter-million-token context is enough to hold an entire codebase, a long legal filing, or a book-length document in a single prompt, and the GQA layout is the quiet reason the memory bill for doing so stays bounded.
Active-parameter counts describe compute per token, not memory. You still need enough VRAM to hold all 295 billion weights resident, which is why Hy3's file footprint dominates the deployment story even though only 21 billion parameters do work on each token.
The benchmark claims, and how to read Tencent Hy3 numbers
Tencent published a stack of self-reported benchmark figures on the model card, and they are strong on paper. The card lists a 90.4 on GPQA Diamond, a graduate-level science reasoning test; 78 on SWE-Bench Verified and 57.9 on the harder SWE-Bench Pro, both software-engineering suites that ask a model to resolve real GitHub issues; 53.2 on Humanity's Last Exam; 55.3 on SkillsBench; 53.6 on WildClawBench; and 25.6 on Apex Agents, an agentic task benchmark where scores across the field tend to run low. Read those as vendor numbers, run by the vendor, on evaluations the vendor selected. They are a useful signal of where a lab thinks its model is competitive, not an independent verdict.
The two SWE-Bench figures deserve a closer look because they are the ones most relevant to anyone using a model to write code. SWE-Bench Verified is a human-filtered set of real GitHub issues where the model has to produce a patch that makes a project's own test suite pass, and a 78 there puts Hy3 in the same neighborhood as recent frontier coding models. SWE-Bench Pro is deliberately harder, drawing on more complex repositories and longer task chains, so the drop to 57.9 is expected and still respectable. The gap between the two scores is itself informative: it tells you the model handles well-scoped bug fixes far better than sprawling, multi-file changes, which is exactly the pattern most coding assistants show. The 25.6 on Apex Agents fits the same shape, since agentic benchmarks that require long tool-use chains remain the hardest frontier for every model, open or closed, and low absolute scores are the norm rather than a red flag.
The more novel data point is the human study. The Decoder and the search record both describe a blind evaluation in which 270 domain experts scored model outputs on tasks drawn from their own daily work. Hy3 came out at 2.67 on a 4-point scale, ahead of GLM-5.1 at 2.51. Blind, task-grounded expert scoring is harder to game than a static benchmark because the graders bring real problems and do not know which model produced which answer, so a lead there carries more weight than a single leaderboard row. It is still Tencent's own study, and a 0.16 gap on a 4-point scale is a margin, not a rout, but it is the kind of evidence that ages better than a benchmark screenshot.
Hallucination is the claim most likely to matter to anyone building on top of a model. Tencent says Hy3 cut its hallucination rate roughly in half, from 12.5 percent to 5.4 percent, as reported by The Decoder. The company did not, in the coverage available, pin that figure to a named public benchmark, so treat it as a directional improvement Tencent measured internally rather than a portable score you can reproduce. Even so, a lab foregrounding a factuality number at all is a shift from the era when raw capability was the only headline anyone chased.
Apache 2.0 and what open weights actually unlock
The license is not a footnote. Apache 2.0 is one of the most permissive terms a lab can attach to model weights: it allows commercial use and redistribution, adds an explicit patent grant, and imposes none of the field-of-use restrictions or acceptable-use riders that some "open" model licenses bolt on. Willison called it out specifically in his note, and for good reason. A model you can legally run inside your own product, fine-tune on your own data, and redeploy without a per-seat agreement is a fundamentally different asset from one you rent through an API.
Tencent published the weights across several hosts, including Hugging Face and ModelScope with additional mirrors, and released an FP8-quantized build alongside the full-precision one. The company also opened a free route on OpenRouter that runs through July 21, 2026, which gives developers a two-week window to test the model at scale before committing hardware or budget. That distribution strategy, weights everywhere plus a time-boxed free hosted endpoint, is now the standard Chinese-lab playbook for driving fast adoption, and it works because it removes the two biggest barriers to trial at once: access and cost.
It is worth separating what open weights give you from what they do not. You get the parameters and the right to use them, which is enough to self-host the model and audit its behavior without vendor lock-in. You generally do not get the training data, nor the code and recipe you would need to reproduce the model from scratch. Hy3 is open-weight, in other words, not open-source in the sense a software engineer might mean the term. That distinction is the one that decides whether "open" means you can deploy freely or you can actually rebuild the thing, and for almost every commercial user the deployment freedom is what counts.
Running Hy3 without underestimating the hardware
The romance of an open model fades the moment you look at the disk. The full-precision Hy3 weights weigh in at 598GB on Hugging Face, and even the FP8 quantized build is 300GB, figures both Willison and The Decoder cite. That is not a laptop model. Tencent's own guidance, per the model card, recommends eight high-memory GPUs of the H20-3e class or similar to serve it, and points users at vLLM and SGLang for production inference with the MTP speculative-decoding path enabled.
For teams without a GPU cluster, the practical routes are the hosted endpoints and the community quantizations. The model card notes 17 community quantizations already exist across formats like llama.cpp and Ollama, which shrink the memory footprint further at some cost to quality and let smaller setups run the model, if slowly. The realistic mental model is a spectrum. At one end sits the free OpenRouter route for zero-setup evaluation and a hosted inference provider for production without owning hardware. At the other end sits self-hosting on a multi-GPU box, the option teams reach for when data has to stay in house or they want to fine-tune. Each rung trades convenience for control, and the right choice depends on whether you are testing an idea or shipping a product. Anyone weighing that build-versus-rent question for their own stack will recognize it as the same calculus behind every usage-based pricing decision in AI tooling.
Where Hy3 sits in the open-model race
Hy3 did not appear from nowhere. Tencent ran a Preview of the model in late April 2026, gathered feedback from more than 50 products, and scaled up post-training with higher-quality data before this full release, according to Willison's summary of the release notes. The Preview lineage is visible in the naming and in the shared 295B-active-21B architecture, and it explains why the July version arrives with product integrations already in place, spanning Tencent's WorkBuddy assistant, the Yuanbao app, WeChat, and even an in-game assistant for a Path of Exile title.
The broader context is a Chinese open-weight surge that has been reshaping the field all year. The Hugging Face write-up positions Hy3 against MiniMax-M2.7 and Kimi-K2.5 and singles out long-text understanding and summarization as a standout strength, alongside tool usage and code generation. The blind study pits it directly against Zhipu's GLM-5.1. Set alongside DeepSeek's earlier V3 and the steady cadence of Qwen and Kimi releases, the picture is of several well-funded labs shipping large MoE models under permissive licenses on a compressed timeline, each release nudging the open frontier closer to the closed one. Hy3's specific bet, differentiated expert sizing plus an emphasis on lower hallucination, is a claim that the next round of gains comes from architecture and training quality rather than from simply stacking more parameters.
For the rest of the ecosystem, the interesting question is durability. A model that tops a leaderboard for a week is a headline; a model that becomes a default others build on is an event. Hy3 has the license and distribution to reach that status, backed by a real architecture story, but which of the two it becomes will be decided by independent evaluations and third-party fine-tunes, and by whether the hallucination improvement holds up outside Tencent's own test set. The weights are public now, which means those answers will arrive quickly, and from people with no incentive to flatter the release.
The read on Tencent Hy3
Strip away the launch-day superlatives and Hy3 is a serious, permissively licensed model with a coherent engineering thesis. The mixture-of-experts design is not a copy of the standard recipe: variable-capacity experts and a routing penalty that fights the pull toward the big ones are real attempts to spend compute more intelligently, and the multi-token-prediction layer gives production teams a concrete speed lever through speculative decoding. The self-reported benchmarks are strong but should be read as vendor numbers, while the 270-expert blind study and the halved hallucination rate are the two claims most worth watching, precisely because they are the ones an outside party can eventually check.
What Hy3 mostly demonstrates is that the gap between "download the weights" and "run the weights" is now the real story in open AI. Apache 2.0 gives you the legal freedom to build; a 598GB footprint and an eight-GPU recommendation set the price of exercising it. For anyone deciding between hosting an open model and paying for a closed API, that tension is the whole decision, and Hy3 is a clean example of both sides of it in a single release. The frontier keeps moving toward open, and the friction that remains is increasingly about operations rather than access. That is a healthier place for the field to be arguing about than it was a year ago.