One machine runs the model. Everyone keeps their code.
Codendum turns a single NVIDIA GB10 into a coding-model service shared by a class or a team. Each person runs OpenCode on their own workstation, and prompts and source code never leave the organization’s network.
A request makes three stops
Scripts, example configuration and documentation to install the service, secure it, test it, measure it and operate it. Every service runs in Docker, and nothing is installed on the host system.
Workstation
OpenCode, Git, toolchains, IDEs, builds and tests
- Each person runs OpenCode on their own machine, with an explicit OpenAI-compatible provider.
- Code is edited, built and tested here, in any language and for any platform.
- One command configures OpenCode from the server’s own model name and context limit.
nginx container, port 8443
The only way in
- HTTPS from the organization’s LAN or VPN, checked against a network allowlist.
- One API key per user, with per-user and global limits.
- Only /v1/chat/completions and /v1/models reach vLLM. Everything else stays on the host.
vLLM container on the GB10
Qwen3-Coder-30B-A3B-Instruct-FP8, served as “coder”
- Listens on 127.0.0.1 only, with tool calling enabled.
- FP8 KV cache, chunked prefill and prefix caching.
- The GB10 only serves inference: CPU and GPU share the same memory, so builds run elsewhere.
Measured on a GB10, not promised
40 simulated students ran OpenCode-like agent sessions on a Java project against one Lenovo ThinkStation PGX, through a WireGuard VPN with about 100 ms of round trip. Each made four requests with 20–90 s pauses.
2K and V×48layers×4KV heads×128head size×1 byteFP8=48 KiBper token
After 31.2 GB of FP8 weights, vLLM reported 62.4 GiB of KV cache: 1,362,640 tokens, or 20.8 full 64K contexts at the same time.
| 40 simulated students | classroom-64k16 running | …-high-concurrency24 running |
|---|---|---|
| Server output, steady state | ~110 tok/s | ~140–150 tok/s |
| Time to first token, P50 / P95 | 17.5 / 60 s | 10.6 / 34 s |
| Output speed per user, P50 | 7.2 tok/s | 6.1 tok/s |
| Time to complete a coding request, P50 / P95 | 11 / 22 min | 8.3 / 18 min |
| Coding requests completed | 45 | 64 |
| Peak KV cache usage | 8.5 % | 11.5 % |
| Preemptions / server errors | 0 / 0 | 0 / 0 |
| Prefix-cache hit rate | 98.2 % | 98.2 % |
The limit is generation, not memory
Agent contexts stayed between 7K and 19K tokens, so the KV cache never went above 12 %. Admitting more requests at once raised total output.
Prefix caching does the heavy lifting
98 % of the 7 million prompt tokens were served from the cache.
Plan in coding requests per hour
About 2,700 output tokens over 15 model calls each: roughly 150–200 requests per hour for the whole class.
Full method and scripts in the benchmark documentation. Run bench.sh and bench-classroom.sh on your own host.
From a fresh GB10 to a working class
On the host, with Docker and the NVIDIA runtime that DGX OS provides. The installation guide and the proxy guide explain every step.
Check the host
Read-only checks: ARM64, Docker, NVIDIA runtime, memory, disk, port exposure, image pinning.
git clone https://github.com/stefanoferi/codendum.git cd codendum scripts/preflight.shStart the model
vLLM binds to loopback only and refuses unpinned images.
cp .env.example .env scripts/start-vllm.sh --waitOpen the gate
Set the host name and allowed networks, install a TLS certificate and generate per-user keys, then start the proxy.
scripts/start-proxy.sh --init scripts/gen-api-keys.sh --users-file users.txt scripts/start-proxy.shTest it
Health, model list, chat, streaming, tool calls and a tool-result round trip.
scripts/smoke-test.shConnect each workstation
Writes the OpenCode provider from the server’s limits, runs a test completion and backs up the previous file.
scripts/configure-opencode.sh --base-url https://llm.lab.example:8443
Preview in 0.2.0
A web panel for the class
People sign up and set up OpenCode from the browser, and administrators follow the service live. The panel has been used in one classroom on a GB10. Its options, routes and pages may still change in a minor release.
For the people using it
- Register with an e-mail address. An administrator approves it, or a whole e-mail domain is approved automatically.
- Install OpenCode, following the guide on the personal page.
- Run a one-time setup command. It is valid for 10 minutes, accepts only the proxy’s public key, and writes the OpenCode provider and the API key.
- Start OpenCode.
The setup has been tested on macOS and on a simulated Linux workstation. The Windows script exists but has not been tested on a real Windows PC yet.
For administrators
- A live dashboard: tokens per second, requests, latency, KV cache and prefix-cache hits; GPU utilisation, temperature and power; model and engine settings, totals since vLLM started, and who is using the API.
- Approve, block or delete people, renew keys, and close registration.
- An optional single login for Open WebUI: people log in once, and panel administrators are chat administrators. Web search in the chat can be switched on or off.
One Python file using the standard library only, plus one SQLite file. It runs in a stock Python container on the loopback interface, behind the same nginx proxy. It mounts the Docker socket, which is the main point to revisit before it leaves the preview.
What Codendum does not do
The limits are written down, so you can decide whether they matter for your class or your team before you install anything.
- One host, no high availability
- When the GB10 or vLLM is down, every user is affected.
- No per-user token quotas
- nginx caps requests, not tokens, and vLLM serves requests first come, first served.
- The prefix cache is shared
- Timing could in principle reveal that someone recently sent the same prefix. You can turn prefix caching off, at a throughput cost.
- Agents run on the workstations
- OpenCode executes tools with the user’s permissions. Sandboxing belongs on the workstation side.
- Policies are yours
- Prompts contain users’ code and the logs contain user ids. Data protection and log retention are the operator’s responsibility.
- Numbers are for this setup
- One GB10, one model, one client version. Measure again with the benchmark scripts after every upgrade.
Read the full threat model and report vulnerabilities privately as described in SECURITY.md.
Next: governing what goes through the model
Today Codendum controls who can use the model. A possible next step is governing what goes through it: personal-data redaction, a prompt-injection firewall, agent loop breaking and a tamper-evident audit log.
That could come from Admina, an open-source framework for governed AI, placed as a gateway between the proxy and vLLM. It is a working hypothesis: there is no code and no timeline yet. See Governance.
Disclaimer
Codendum is provided “as is”, without warranties or conditions of any kind, as stated in the Apache License 2.0. The measurements on this page come from one GB10, one model and one client version, and are not a promise of performance.
Container images, model weights and client software are downloaded separately and keep their own licenses; see the component inventory.
NVIDIA, DGX Spark and GB10 are trademarks of NVIDIA Corporation. Other product names are trademarks of their respective owners. Codendum is not affiliated with or endorsed by them.
