VectorGrid documentation

One binary, one GGUF file: a local agent, an OpenAI-compatible server, and a network layer that pools every machine you own into one inference grid.

Install

macOS and Linux:

$ curl -sSL https://raw.githubusercontent.com/VectorGrid/vectorgrid/main/install.sh | sh

Windows (PowerShell):

> irm https://raw.githubusercontent.com/VectorGrid/vectorgrid/main/install.ps1 | iex

The installer detects your platform, verifies SHA-256 checksums, and installs to ~/.vectorgrid/bin. On Linux it picks the CUDA build automatically when an NVIDIA driver is present — the CUDA runtime ships inside the tarball, so the driver is all you need. Update any time with vectorgrid update.

Your account

$ vectorgrid login

Product commands require a signed-in account — once per machine. If you don't have an account, the login prompt creates one right there (type new), including email verification. After that, scripts and services on the machine inherit the stored sign-in automatically, and a 72-hour offline grace window keeps a signed-in machine working without a network.

Diagnostics (doctor, analyse) and account commands always work without sign-in. Everything your account runs appears in your console.

First run

$ vectorgrid

The bare command opens the agent. With no model installed, it detects your hardware, recommends a model that fits, and offers to download and start it. Prefer a plain model chat? vectorgrid pull tinyllama && vectorgrid run tinyllama gets you a REPL with a ~640 MB starter model.

The agent

A Claude-Code-style assistant, fully local: it can run commands, read and write files, search your code, browse the web, and call MCP servers — powered by the model of your choice.

$ vectorgrid pull qwen2.5-7b-instruct   # agent-grade model
$ vectorgrid                            # interactive agent
$ vectorgrid agent -p "summarize what changed in the last commit"

Serve an API

$ vectorgrid serve --model qwen2.5-7b-instruct

An OpenAI-compatible API on port 28100 (deliberately uncommon — no collisions with 8080/8000/3000/11434; override with --port). Point any OpenAI client at http://localhost:28100/v1: chat completions with streaming, tools, embeddings, continuous batching, speculative decoding.

$ vectorgrid daemon start --model qwen2.5-7b-instruct   # run in the background
$ vectorgrid ps                                          # what's up
$ vectorgrid daemon stop

Operational niceties built in: /metrics (Prometheus), /health, API keys via --api-keys, request-duration limits that auto-scale to hardware, and an OOM admission breaker.

Pool your machines

A gaming PC, a MacBook, an old workstation — each too small for the model you want. Pooled over your LAN, they run it together: layers split across machines, activations streaming over a UDP transport built for inference.

$ # on the machines with GPUs:
$ vectorgrid worker --listen vgt://0.0.0.0:19310 --announce

$ # on your laptop — workers are auto-discovered:
$ vectorgrid run r1-distill-qwen-14b --pool

Byte-identical, split or not. The same seed produces the same bytes on one machine or five — greedy decoding is bit-exact.
Survives failures. A worker dying mid-generation reconnects — or fails over to a different worker — and the output is still identical.
Loss-engineered transport. Selective-ACK ARQ plus Reed-Solomon parity: at 5% packet loss, decode holds clean-link speed.
Encrypted by default. AEAD on every session.

Name machines explicitly with --nodes vgt://host:19310, control the split with --gpu-split 24,24. If discovery misbehaves, vectorgrid doctor diagnoses your LAN (firewalls, broadcast, WiFi power-save) with exact fix commands.

The VectorGrid network

No second machine? Pool with a peer on the grid instead. The model file downloads to your disk; the heavy compute runs on your peer over the same encrypted VGT session pooling uses on a LAN.

$ vectorgrid grid models        # what the network serves right now
$ vectorgrid run qwen2.5-14b-instruct --grid

Tiers compose. --pool --grid prefers your LAN and falls back to the network only when no local peer holds the model.
Honest by design. No capacity is a clear message and a local fallback — never a hang. The network is young; what it serves will grow.
Signed-in only. Grid sessions need a live sign-in; the offline grace window that keeps local commands working doesn't grant network capacity.
Same guarantees. AEAD-encrypted session, per-session key, and the engine's mid-generation failover.

Models & context

$ vectorgrid pull <name>      # 40+ curated models, parallel + resumable + verified
$ vectorgrid list             # what's installed
$ vectorgrid analyse          # which models fit this machine (and your pool)

The curated catalog spans Qwen 3 / 2.5, Llama 3.1–3.3, DeepSeek-R1 distills, Gemma 2/3, Phi, Mistral, GLM-4, Granite, coding models and embeddings. Any GGUF you already have works via vectorgrid import; make your own quantizations with vectorgrid quantize.

Long context

--ctx-size N requests context; a memory preflight grants what actually fits and says so honestly.
--kv-dtype f16 halves KV memory — roughly double the context. Labeled non-exact: outputs can differ from f32 at the last bit. Keep f32 for attestation.
--prompt-file path for prompts beyond the OS argument limit (long-context work needs it).
Pooling is the long-context play: KV memory and bandwidth divide across machines — two pooled GPUs read a prompt neither could hold alone.

Telemetry & privacy

Signed-in machines report anonymous usage metadata — command used, model alias, tokens/sec, crash location with the panic message hashed. The schema is content-free by construction: prompts, outputs, file paths, and addresses have no fields to live in. Batches upload every 15 minutes; offline they wait on disk, capped at 20 MB.

$ vectorgrid telemetry status   # exactly what's queued
$ vectorgrid telemetry off      # opt out permanently (queue purged)

Troubleshooting

$ vectorgrid doctor

Checks your install, compute backend and GPU initialization, model directory, server health, LAN discovery (firewall rules with exact allow commands, WiFi power-save, cloud-VM broadcast caveats), and can truth-test a specific worker with doctor --worker vgt://host:port. --fix applies the safe fixes automatically.

GPU worker looks idle or slow? Check that nothing else is pinning VRAM, then vectorgrid analyse to confirm the model actually fits — the preflight refuses dishonest configurations rather than swapping to disk.

CLI reference

Command	What it does
`vectorgrid`	Open the agent (same as `vectorgrid agent`)
`vectorgrid run <model> [-p "…"] [--prompt-file f]`	One-shot or interactive generation
`vectorgrid serve --model <m> [--port N] [--api-keys …]`	OpenAI-compatible API server
`vectorgrid worker --listen vgt://0.0.0.0:19310 [--announce]`	Offer this machine's compute to your grid
`vectorgrid run <m> --pool \| --nodes vgt://host:port`	Split the model across pooled machines
`vectorgrid run <m> --grid`	Pool with a peer on the VectorGrid network
`vectorgrid grid models`	What the network serves right now
`vectorgrid pull / list / show / info / rm / gc`	Model management (parallel pulls, LRU cache cleanup)
`vectorgrid daemon start\|stop\|status\|logs · ps · stop`	Background server lifecycle
`vectorgrid login / signup / verify-email / logout / whoami`	Account
`vectorgrid telemetry status\|on\|off`	Telemetry control
`vectorgrid doctor [--fix] · analyse`	Diagnostics · what fits your hardware
`vectorgrid bench · eval perplexity`	Throughput benchmark · accuracy evaluation
`vectorgrid quantize · import · embed`	Quantize GGUFs · import external models · embeddings
`vectorgrid update · verify <attestation>`	Self-update · verify a deterministic-run attestation

Questions or bugs → GitHub issues · your data → console