VectorGrid — run any open model on a grid you own

Two ways in

An agent that does the work. A server that speaks OpenAI.

Both live in the same binary. Both run entirely on your machines.

The local agent

Type vectorgrid and you're talking to an agent that can run commands, read and write files, search your code, browse the web, and call MCP servers — fully local, on the model you chose. First run, it detects your hardware and recommends a model that fits.

vectorgrid agent -p "summarize the last commit"

The OpenAI-compatible server

Point any OpenAI client at localhost:28100/v1 and it works — streaming, tools, embeddings, continuous batching, speculative decoding. Your existing SDKs and apps swap over by changing one base URL.

vectorgrid serve --model qwen2.5-7b-instruct

The grid

Your machines. One model.

A gaming PC, a MacBook, an old workstation — each too small for the model you want. Pooled over your LAN, they run it together, streaming layer-to-layer with a transport built for inference.

# on the machines with GPUs $ vectorgrid worker --listen vgt://0.0.0.0:19310 --announce # on your laptop — workers are auto-discovered $ vectorgrid run r1-distill-qwen-14b --pool discovered 2 workers · split 24/24 · streaming

Byte-identical, split or not. The same seed produces the same bytes on one machine or five — verified by SHA on every release.
Survives failures mid-generation. A worker dies at token 88? The driver reconnects — or finds a different worker — and the output is still identical.
Loss-engineered transport. UDP lanes with selective-ACK ARQ and Reed-Solomon parity: at 5% packet loss, decode holds clean-link speed.
Encrypted by default. AEAD on every session; a plaintext peer fails loud, never silently.

Deterministic by design

Same seed. Same bytes. Any machine.

Greedy decoding is bit-exact across CUDA, Metal, and CPU — alone or split across a pool, with or without speculative decoding. Every optimization ships only after it reproduces the golden hash. If your workload needs proof, VectorGrid can give you a hash, not a promise.

single · Metalsha256 44e72049…

split 2× · VGT · CUDA+Metalsha256 44e72049…

split + speculativesha256 44e72049…

≡ byte-identical

Field-measured

Numbers from real machines, not marketing.

Every figure below is reproduced by the test suite or logged from the hardware named. Nothing is projected.

0 tokens

One prompt, one $1/hr GPU. A needle buried 40,000 words deep — retrieved exactly.

llama-3.1-8B · single A10G · f16 KV

1.49× prefill

Pipelined prefill across two machines over plain WiFi — an 8K prompt, 25s → 17s.

14-layer split · CUDA + Metal · byte-identical

3× at 5% loss

Reed-Solomon parity holds decode at clean-link speed where unprotected UDP collapses.

27 vs 9 tok/s · 5% random loss · zero clean-link cost

+29%

int8 wire encoding on consumer uplinks — decode 61 → 78 tok/s on a 25 Mbit link.

opt-in --wire-dtype i8 · labeled, never silent

2× context

f16 KV cache doubles the context that fits — 131K tokens where f32 capped at 102K.

--kv-dtype f16 · byte-identical goldens held

0 tokens

One prompt across two pooled GPUs — 1.35× beyond what either could hold alone. The needle, 29K tokens deep, retrieved exactly.

14B · 2× A10G over vgt:// · single-GPU ceiling 51,959 · byte-exact f32

Distributed results measured over real consumer WiFi between a Mac mini (M4 Pro, Metal) and an RTX laptop (CUDA); long-context results on AWS g5.xlarge. Methodology and raw logs live in the engine's execution log.

Curated catalog

40 models, one command.

vectorgrid pull <name> — resolved, downloaded in parallel, SHA-verified, resumable. Or import any GGUF you already have.

qwen3 0.6b–14b qwen2.5 0.5b–72b llama 3.1 / 3.2 / 3.3 deepseek-r1 distills gemma 2 + 3 phi-3 / phi-4 mistral · nemo · small qwen2.5-coder 7b–32b devstral glm-4 granite 3.3 smollm2 / smollm3 llava vision embeddings · bge · nomic

Speculative decoding

Blessed draft pairs ship in the catalog — --draft-model auto picks a family-matched draft with a verified vocabulary. Byte-identical output, fewer forward passes.

Quantize & import

Bring any GGUF, or quantize your own — Q8_0 to Q4_K and beyond. The engine refuses combinations it can't run faithfully, out loud.

Doctor & analyse

vectorgrid doctor diagnoses your setup — firewall, discovery, GPU init, stale downloads — with the exact fix commands. analyse tells you which models fit your hardware, and your pool.

Telemetry, in the open

What VectorGrid reports. And what it never can.

Signed-in installs report anonymous usage metadata so we can see what breaks and what matters. The schema is content-free by construction — these fields don't exist in the code, so they can't leak:

Reported

Command used and duration · model alias and speed (tokens/sec, time-to-first-token) · context length · pool size and failover counts · crash location (file:line) with the panic message hashed · version, OS, backend.

Never reported

Prompts · outputs · file names or paths · hostnames or IP addresses · environment variables · panic message text. Batches upload every 15 minutes; offline they wait on disk, capped at 20 MB.

Inspect the queue any time: vectorgrid telemetry status · opt out permanently: vectorgrid telemetry off (the local queue is purged).

Get started

Thirty seconds to your first token.

Install, pull a model, talk to it. Everything after that is your business — literally.

curl -sSL https://raw.githubusercontent.com/VectorGrid/vectorgrid/main/install.sh | sh

Download v0.6.29 Read the docs