Open Weights, Open Source, and the Gap Between Them

The reason a small company ends up looking at downloadable models is almost never curiosity. It is a clause. A client contract that says personal data stays in the country. A data protection assessment that came back awkward. A finance spreadsheet where per token costs stopped being a rounding error somewhere around month four.

Then you open a model directory and find several hundred entries, all described as open, in wildly different senses of the word. This piece is about how to read that list: what the license really permits, what the size number really means for your hardware, and which of the promises are made by the architecture rather than by anything you will experience.

What does open source actually mean for a model?

Something much stricter than "you can download it". The Open Source Initiative published version 1.0 of its Open Source AI Definition and it asks for three components, not one.

The first is data information: enough detail about the training data that a skilled person could build a substantially equivalent system, including provenance, scope, labeling procedures and filtering. The second is the complete source code used to train and run the system. The third is the parameters themselves, the weights, including intermediate checkpoints.

Nearly every model people call open source today ships the third item and part of the second. That is why the more accurate phrase, and the one used through the rest of this article, is open weights. You get the trained artefact. You usually do not get the recipe, and you cannot rebuild it from scratch.

This distinction is not pedantry, and it is not only about ideology. If you cannot see the training data description, you cannot answer a customer asking whether their industry's regulated content was in it. If you cannot rebuild the model, then the day the publisher stops publishing, you have the copy you already downloaded and nothing else.

Where is the license trap?

In the gap between a permissive badge and a custom document nobody read. TechCrunch's survey of the restrictions carried by so called open model licenses is still the clearest write up of the pattern, and it names the two big ones.

Meta's Llama terms restrict using the model's outputs to improve any model other than Llama, and require a separate negotiated license from Meta for any product above 700 million monthly active users. The second clause is irrelevant to you and it is quoted constantly. The first is the one that matters commercially, because generating synthetic training data from a model is a normal thing for a small team to want to do.

Google's Gemma terms went further. The Gemma Terms of Use require you to pass the use restrictions down to anyone you distribute to, to ship a copy of the terms with any non hosted distribution, and they reserve Google the right to restrict usage, remotely or otherwise, of anything Google reasonably believes breaches the agreement. No OSI approved license contains a remote restriction clause, because the whole point of those licenses is that permission is granted once and cannot be withdrawn.

Here is the part most articles on this subject have not caught up with. The Gemma 4 12B model card lists Apache 2.0, a genuine OSI approved license, on a model released in July 2026. Google moved. The lesson is not that Google is untrustworthy or that it has been redeemed. It is that license status is a property of a specific release, it changes, and the only reliable source is the model card in front of you on the day you download it.

What the current families actually offer

The table below was assembled from the individual model cards, each of which publishes its numbers in isolation and none of which compares itself on license terms. Figures read on 4 August 2026.

One column is deliberately missing, and it is the one the search results promise: which closed model each of these is closest to. We are not printing that, because the honest version of that column is a set of vendor supplied benchmark numbers measured under conditions nobody else can reproduce. We looked at what those comparisons are worth in our piece on how far open weight models have closed the gap on frontier systems, and the summary is that the gap depends entirely on the task you measure.

Why does a 27B model need eight GPUs?

Because parameters and context compete for the same memory, and the second one is what surprises people. Weights are a fixed cost you pay once at load time. The key value cache, which is the running memory of the conversation, grows with every token in the window, and at a 262,144 token context that cache can dwarf the model itself.

The Qwen3.6 27B card is unusually direct about this. The model is 27 billion parameters, which fits comfortably on a single high memory card at reduced precision. To serve the full native context the card suggests tensor parallelism across 8 GPUs. Both statements are true at once, and which one applies to you depends on whether your workload is short prompts or long documents.

The mixture of experts families invert a different intuition. DeepSeek V4 Flash lists 284B total parameters with 13B active per token, so it computes like a small model and stores like a very large one. Compute cost drops. The storage bill does not: the quantized builds run from 82.5 GB at the most aggressive setting up to 162 GB, and that is disk before you consider the memory to hold it.

If you want the practical version of this arithmetic rather than the architectural one, our guide to what your hardware can honestly handle when running a model locally works through the sizing, and the comparison of vLLM against Ollama covers which server suits which situation.

What can you actually run on the hardware you own?

Three brackets cover almost every real situation, and knowing which one you are in saves a week of reading.

A laptop or a workstation with one consumer card. This is the Gemma 4 12B bracket. Its card describes it as optimized for on device use and suited to consumer GPUs and workstations, which is a rare piece of plain speaking from a model publisher. At this size the model is genuinely useful for classification, extraction, drafting and rewriting, and genuinely weak at long multi step reasoning. Expect to run it at reduced precision and expect the context you can hold to be a fraction of the number on the card.

One server with several cards. The Qwen3.6 27B bracket, and where most serious small deployments land. Dense models in the twenty to thirty billion range hit a sweet spot: one machine, predictable memory, no exotic serving setup. This is also where the vision capable models become interesting for anyone with a product catalogue, because captioning and attribute extraction from photographs stop being a separate service.

A rack, or somebody else's rack. DeepSeek V4 Pro at 1.6 trillion parameters and GLM 5.2 at 753 billion live here. These are not workstation propositions in any configuration. If you want them, you rent them from a provider that already owns the hardware, at which point you have reintroduced the third party you were trying to remove, unless that provider is one you have contractually pinned down on data handling. Worth saying plainly: renting open weights from a host is a different arrangement from running them, and it solves the vendor lock in problem while solving none of the data residency problem.

The middle bracket is where most of the value sits for a business, and it is the least written about, because it is neither the impressive demo nor the cheap trick.

What breaks after you deploy?

The failures are boringly consistent, which is good news, because it means you can plan for them.

Memory pressure arrives first, and it arrives under load rather than in testing. A model that serves one conversation happily will fall over on the fifth concurrent one, because each of those conversations is holding its own cache. The fix is capacity planning against concurrent sessions, not against model size, and it is the number people forget to measure.

Version drift arrives second. You built against one server release and a quantized file produced by one tool. Both move. Six months later the model file format has a new revision, the server has a breaking change in its API, and the specific quantized build you tested has been replaced by a better one you have not. Pinning versions is not optional here, and neither is writing down exactly which artefact you deployed.

Quality regression arrives last and quietest. Nobody is degrading your model, since a frozen local copy cannot change. What changes is your business: new products, new phrasing, customers asking things the prompt was never built for. A hosted model would have improved underneath you. A local one holds exactly the capability it had on the day you downloaded it, which is a real advantage for reproducibility and a real liability for anyone who expected free progress.

Does self hosting actually save money?

Sometimes, and the crossover point is further away than most people assume. A hosted API charges per token and nothing when idle. Self hosting charges for the machine whether or not anyone sends a request, and a GPU instance idles at the same price as a busy one.

The shape of your traffic decides it. Steady heavy volume favours owned hardware. Spiky low volume, which describes most small businesses, favours per token billing by a wide margin. We put real numbers against that curve in what a real feature costs once you price the API calls, and the pattern holds regardless of which model you pick.

There is also a cost that never appears in the comparison: someone has to own the deployment. Model servers get security patches. Drivers break. A new quantization comes out and the old one stops being supported by the tool you built around. None of that is difficult, and all of it is a recurring claim on the only scarce resource a small business has, which is attention.

How do you choose without running benchmarks?

Work through the constraints in the order that eliminates the most options first.

License first. If you will redistribute anything, or generate synthetic data, or your legal counsel wants a named OSI approved license, that single filter removes most of the list before you look at a single score. Apache 2.0 and MIT pass. Custom community terms need a lawyer, and for a company of five people the cost of that review usually exceeds the value of the model.

Hardware second. Take the memory you actually have, subtract what the operating system and your application need, and see which quantized build fits with room for the context you will really use. Be honest about the context: almost nobody needs a million tokens, and paying for that headroom on every request is a common and expensive mistake.

Task third. Only now does quality enter, and only on your own data. Ten representative prompts from your actual business, run against three candidates, graded by you, will tell you more than any public leaderboard. Our note on building an evaluation you can rerun describes how to set that up in an afternoon.

Maintenance last, and it is the tiebreaker. Between two models that both pass, take the one with the larger ecosystem of tooling and quantized builds. Not because it is better, but because in eight months you will want to move to a new server version and the popular model will already be supported.

What changes and what does not

The specific models in the table will be superseded. Something will publish larger context, cheaper inference and a better score, probably before the end of the quarter. Keeping a single vendor lineup straight is its own exercise, which is why we set out the full Mistral lineup and what each model is for. The structure underneath is much slower moving, and it is worth writing on a card and keeping.

Downloadable does not mean open source. License status belongs to a release, not to a company. Parameter count tells you about storage while context tells you about memory, and the two bills arrive separately. Quality is decided on your data, not on a leaderboard built from someone else's. The real cost of self hosting is a person, and that person is usually you.

Read the card. Read the license file next to it, which takes four minutes and is the single highest return task in this whole exercise. Then pick the boring option that fits your hardware, because the model you can actually run today beats the one you would need to buy a rack to try.

One last thing, aimed at anyone being pushed toward this by a compliance requirement rather than by preference. Self hosting is a defensible answer to a data question and a poor answer to a cost question at small volume. If the driver is a contract clause about where personal data may sit, owned weights on owned infrastructure answers it cleanly, and the extra work is worth it. If the driver is a monthly bill that felt high, measure first, because the machine you would need to replace that bill usually costs more than the bill did.