Batch Processing

Submit your first batch job in five lines of Python. Batch processing is 50% off serverless pricing.

Process thousands or millions of LLM inferences at 50% off serverless pricing. Batch is ideal for evaluations, data processing, and any workload that doesn't require real-time responses.

1. Get an API key

1. Go to https://www.saas.parasail.io/keys.

2. Click Create API Key.

3. Copy the key—it's shown only once.

Store it in your environment:

export PARASAIL_API_KEY="your-api-key-here"

2. Install the batch helper library

pip install openai-batch

The library is 100% compatible with both Parasail and OpenAI. Source code: github.com/parasail-ai/openai-batch.

3. Submit your first batch

import random
from openai_batch import Batch

with Batch() as batch:
    objects = ["cat", "robot", "coffee mug", "spaceship", "banana"]
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Tell me a joke about a {random.choice(objects)}"}]
        )
    result, output_path, error_path = batch.submit_wait_download()
    print(f"Batch completed with status {result.status} and stored in {output_path}")

Run it:

PARASAIL_API_KEY=<your-api-key> python3 test_batch.py

4. Monitor via the Batch UI

The Parasail Batch UI lets you upload batches, track status, and download results. Open the Batch tab in the dashboard.

Key features

• 50% off serverless pricing—cached tokens get an additional 50% off.

• OpenAI-compatible—works with the OpenAI batch file format and API.

• Any HuggingFace model—no need for a dedicated deployment; just specify the HuggingFace ID.

• Embeddings—supports embedding models, not just chat.

• Resumable—submit a batch, then monitor and download in a separate process using the batch ID.

• Up to 50,000 requests and 1000 MB per input file.

Next steps

• Batch full guide—comprehensive batch processing guide

• Batch helper GitHub repo—library source and README

• Batch file format—JSONL input format (OpenAI-compatible)

• Batch API reference—OpenAI-compatible batch API spec

• Batch with private models—batch processing with private HuggingFace repos