A private chat with an open model
Talk to the model you choose, in the chat interface you prefer, over your own SSH connection.
Closed betaInference on whole GPUs
Run open models on a whole GPU.
Rent a whole computer with an RTX 4090 (24 GB) or an RTX 5090 (32 GB) for the hours you choose. Bring your own runtime and model over SSH. Your prompts go to the model you run, on a computer no other renter shares.
- By invitation
- Whole computer, not shared
- SSH with your key
What you can run
Your model, for your hours.
Pick the open model and the runtime you know. For your rental window, no other renter shares the computer or its GPU.
An API for your own tools
llama.cpp's server and vLLM each serve an OpenAI-compatible API. Run one inside the rental and point your scripts, apps or coding tools at it from your own computer.
Batch jobs over a dataset
Summarise, label or translate a whole dataset in one sitting, then copy the results out before your window ends.
Embeddings and model tests
Embed your documents for search, or try a model on your own prompts before you build on it.
What fits
What fits in 24 GB and 32 GB.
Graphics memory decides which model you can load, and how much text it holds at once. These are rules of thumb from the model files and each model's own settings, not guarantees.
RTX 4090
24 GB
of graphics memory
The 27B to 35B models below at 4-bit, each with 32k tokens of context or more, and gpt-oss-20b with its full 128k.
RTX 5090
32 GB
of graphics memory
The same models with 128k tokens of context or more, or a 27B model at 8-bit.
Too big for one card: gpt-oss-120b, and any model of 70B parameters or more at 4-bit.
| Model and file | RTX 4090 · 24 GB | RTX 5090 · 32 GB |
|---|---|---|
| gpt-oss-20bMXFP4 · 12.1 GB | Fits, full 128k | Fits, full 128k |
| Gemma 4 26B-A4B4-bit Q4_0 · 14.6 GB | Fits, full 256k | Fits, full 256k |
| Qwen3.6-35B-A3B4-bit Q4_K_M · 20.5 GB | Fits, 32k | Fits, full 256k |
| Gemma 4 31B4-bit Q4_0 · 18.0 GB | Fits, 32k | Fits, 128k |
| Qwen3.8-27B4-bit Q4_K_M · 19.4 GB | Fits, 32k | Fits, 128k |
| Qwen3.8-27B8-bit Q8_0 · 28.6 GB | Does not fit | Fits, 16k |
| gpt-oss-120bMXFP4 · 63.4 GB | Does not fit | Does not fit |
| Llama 3.1 70B4-bit Q4_K_M · 43.1 GB | Does not fit | Does not fit |
With an 8-bit context cache, each fit holds about twice the context, up to the model's own longest. Sending images loads a model's vision part too, 0.6 to 0.8 GB more, and every fit in the table still holds with it.
Work it out for any model
- Weights take about the file's size. In llama.cpp's words, "memory and disk requirements are the same". No file yet? Billions of parameters × bits per weight ÷ 8 gives GB: about 4.9 bits at Q4_K_M, 8.5 at Q8_0 and 16 at 16-bit, as llama.cpp measured them. Real files can differ from this sum either way, so use the file's own size once there is one.
- Context takes memory for every token. Per token: 2 × the layers that keep a cache × KV heads × head size × bytes per value, with the numbers from the model's config.json. For Qwen3.8-27B that is 2 × 16 × 4 × 256 × 2 bytes, 64 KiB a token, so 32k tokens take 2 GiB. An 8-bit cache takes about half.
- Leave room for the runtime. We count a fit only when at least 1.5 GB of the card's 24 GB or 32 GB stays free for the runtime's own buffers; that margin is ours, not a vendor's figure. Your runtime prints what it uses when the model loads, and nvidia-smi shows the card's memory.
Sources: file sizes from the model files that ggml-org, llama.cpp's own team, publishes; layers, heads and longest context from each model's config.json; bits per weight and Llama 3.1 70B from llama.cpp's quantize notes; the context formula from NVIDIA; the 8-bit cache from Ollama's FAQ. OpenAI's model card says gpt-oss-20b runs "within 16GB of memory".
GPU computers on the Compute Market
Open the full Market, with filtersHow it works
Connect, start your server, forward its port.
Bring your own stack. The rental can reach the internet, so your runtime and your model download straight to the computer.
Connect with your key
When your window starts, connect over SSH with your own key. You log in as a normal user, without root.
Bring your runtime and model
Install your runtime in your own folders, then download your model to the rental's disk.
Forward its port
Start your server on localhost inside the rental. If it listens on port 8000, add
-L 8000:localhost:8000to your ssh command, and it answers athttp://localhost:8000on your own computer.Copy your results out
Only SSH reaches the rental from outside, and everything on it is deleted when your window ends. Copy your outputs out before then.
Privacy and price
Your prompts. Your hours.
Where your prompts go, and how your hours are counted.
Your prompts go to your model
They go to the model you run, on a computer no other renter shares, and to no outside model service. The host can still technically access data on the computer: read the data limits, and do not process data you could not accept a host seeing.
Priced by the hour, not the token
The price per hour you see on a computer is for the whole computer, and it does not change under you. For a batch job, your cost per token is that price divided by the tokens an hour of your job produces.
Credit for slow minutes
Minutes when the computer is slow or unavailable are credited back to you. If the computer drops out, only the time that was actually delivered counts.
Good to know
Know this first.
- A rental is a time window you choose ahead, not on demand, and the computers are often in use.
- No disk is kept between rentals: copy your outputs out before your window ends.
- Your runtime and your model download again at the start of each rental.
- Not for always-on endpoints yet.
- The computers are in Qatar, not in a data centre.
- Closed beta: access is by invitation.
Express your interest
Tell us what you would run.
It takes about two minutes, and we read every application. Invitations come by email.