Dedicated Instances
Deploy your own model on a dedicated GPU instance in a few minutes.
Deploy a private GPU endpoint with full control over the model, hardware, and scaling. Dedicated instances are ideal for production workloads, custom models, and models not available on serverless.
1. Get an API key
Click Create API Key.
Copy the key—it's shown only once.
2. Navigate to the Dedicated tab
In the Parasail dashboard, click the Dedicated tab to open the deployment configuration page.
3. Configure your deployment
Deployment Name
A name for your endpoint.
HuggingFace ID / URL
The model to load. For example, neuralmagic/Qwen2-72B-Instruct-FP8 or the full HuggingFace URL.
HuggingFace Token
Required only for private HuggingFace repos. See Private HuggingFace Models.
Hardware Service Tier
Choose from low-cost, low-latency, or ultra-low-latency GPU configurations.
If the model is supported, you'll see a green confirmation message.
4. (Optional) Set advanced options
Speculative Decoding Draft Model—speed up output with a draft model. See Speculative Decoding.
Replicas—spin up multiple load-balanced replicas for high throughput.
Scale Down Policy—automatically shut down after idle time to avoid unexpected costs.
Max Concurrent Connections—set a per-replica concurrency limit to maintain QoS.
5. Deploy
Click Deploy. If the model weights have been cached previously, deployment takes 3–5 minutes. If weights need to be downloaded from HuggingFace, it can take 10 minutes to several hours depending on model size.
6. Call your endpoint
Once the endpoint is live, call it using the OpenAI SDK:
Next steps
Dedicated instances overview—full guide to dedicated endpoints
Auto-scaling—scale replicas with traffic
FP8 quantization—reduce cost with quantization
Management API—manage endpoints programmatically
Last updated