Some of the most capable AI models are only available through an API: you send data to the provider and receive answers. Others are released as open weights: anyone can download the trained model and run it on their own hardware. Choosing between them is one of the most important architecture decisions for any AI project.
Definitions first
- Closed (proprietary) models run only on the provider’s servers. You access them through an app or API.
- Open-weight models publish their trained parameters. Licenses vary from permissive (such as Apache 2.0 or MIT) to custom licenses with usage restrictions.
- Open-source AI, in the strict sense defined by the Open Source Initiative in 2024, also requires sufficient information about the training data and the code used to train the model. Many “open” models are open-weight but not open-source by that definition. See our glossary entry on open-weight models.
The comparison
| Factor | Closed API | Open-weight, self-hosted |
|---|---|---|
| Peak quality | Often the frontier | Usually close behind, sometimes level |
| Setup | Minutes | Days to weeks |
| Cost model | Pay per token | Pay for hardware and operations |
| Data control | Data goes to provider (with contractual protections) | Data can stay entirely in-house |
| Customization | Prompting, limited fine-tuning | Full fine-tuning and modification |
| Vendor risk | Price changes, deprecations | You control versions |
| Maintenance | Provider handles it | Your team handles it |
When closed APIs win
- You need the best possible quality on hard reasoning, coding or agentic tasks.
- Your volume is low or unpredictable, so pay-per-use is cheaper than idle hardware.
- You lack the team to run GPU infrastructure reliably.
- You want features such as built-in web search, tool use and multimodal input without building them.
When open weights win
- Data cannot leave your environment for regulatory, contractual or security reasons.
- You have high, steady volume, where owning or renting hardware becomes cheaper than per-token fees.
- You need deep customization: fine-tuning on proprietary data, modifying behavior, or running offline.
- You want stability: no surprise model changes or retirements.
- You need on-device or edge deployment, where small open models shine.
Hidden costs on both sides
Self-hosting requires GPUs (or rented cloud instances), inference software, monitoring, security patching, and evaluation every time you upgrade the model. Engineering time is often the largest cost.
APIs can become expensive at scale, and you depend on the provider’s roadmap, pricing and data terms.
Licenses matter
Open-weight licenses differ. Some allow any commercial use; others restrict use above a certain number of users, prohibit specific applications or require attribution. Read the license before building a product.
The common answer: both
Many teams use a hybrid: a closed frontier model for the hardest requests and an open model, often fine-tuned, for high-volume or sensitive tasks. Routing requests by difficulty keeps quality high and costs under control.
Common mistakes to avoid
- Comparing only per-token price with hardware price. Self-hosting costs also include engineering time, monitoring, upgrades and idle capacity. Compare total cost of ownership.
- Assuming open means safe from all risk. Self-hosted models still need security hardening, content safeguards and evaluation, and their licenses can restrict use.
- Choosing based on a single benchmark. Test both options on your own tasks; leaderboard positions change constantly.
- Underestimating serving complexity. High-traffic, low-latency serving of large models requires specialized inference software and expertise.
- Locking in too early. Design your application so the model can be swapped. The best option today may not be the best next year.
Key takeaways
- Closed models run only on the provider’s servers; open-weight models can be downloaded and run anywhere.
- Open-weight is not the same as open-source: training data and code are often not released.
- APIs win on peak quality, speed of setup and low or unpredictable volume.
- Open weights win on data control, customization, stability and high steady volume.
- Many production systems combine both and route requests by difficulty and sensitivity.
Frequently asked questions
Are open-weight models free?
The weights are usually free to download, but running them is not: you pay for hardware or cloud instances, energy and engineering time. Licenses may also limit commercial use.
Can I run an open model on my laptop?
Small models, typically up to a few billion to around 30 billion parameters with compression, can run on modern laptops using tools such as Ollama or LM Studio. Large models need server-grade GPUs.
Are closed APIs less private?
Not necessarily. Enterprise API agreements often guarantee that data is not used for training and is retained only briefly. But with self-hosting, data never leaves your control at all.
How close are open models to the frontier?
The gap has narrowed considerably. The best open-weight models often trail the leading closed models by months rather than years, and on some tasks they match them.