Dedicated Instances

Deploy your own model on a dedicated GPU instance in a few minutes.

Deploy a private GPU endpoint with full control over the model, hardware, and scaling. Dedicated instances are ideal for production workloads, custom models, and models not available on serverless.

1. Get an API key

1. Go to https://www.saas.parasail.io/keys.

2. Click Create API Key.

3. Copy the key—it's shown only once.

2. Navigate to the Dedicated tab

In the Parasail dashboard, click the Dedicated tab to open the deployment configuration page.

3. Configure your deployment

Field	Description
Deployment Name	A name for your endpoint.
HuggingFace ID / URL	The model to load. For example, neuralmagic/Qwen2-72B-Instruct-FP8 or the full HuggingFace URL.
HuggingFace Token	Required only for private HuggingFace repos. See Private HuggingFace Models.
Hardware Service Tier	Choose from low-cost, low-latency, or ultra-low-latency GPU configurations.

If the model is supported, you'll see a green confirmation message.

4. (Optional) Set advanced options

• Speculative Decoding Draft Model—speed up output with a draft model. See Speculative Decoding.

• Replicas—spin up multiple load-balanced replicas for high throughput.

• Scale Down Policy—automatically shut down after idle time to avoid unexpected costs.

• Max Concurrent Connections—set a per-replica concurrency limit to maintain QoS.

5. Deploy

Click Deploy. If the model weights have been cached previously, deployment takes 3–5 minutes. If weights need to be downloaded from HuggingFace, it can take 10 minutes to several hours depending on model size.

6. Call your endpoint

Once the endpoint is live, call it using the OpenAI SDK:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.parasail.io/v1",
    api_key=os.environ["PARASAIL_API_KEY"],
)

response = client.chat.completions.create(
    model="your-deployment-name",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

Next steps

• Dedicated instances overview—full guide to dedicated endpoints

• Auto-scaling—scale replicas with traffic

• FP8 quantization—reduce cost with quantization

• Management API—manage endpoints programmatically