Batch API

OpenAI-compatible Batch API reference for Parasail batch processing.

Parasail's batch API is a direct drop-in replacement for OpenAI's batch API. The same documentation, guides, and client libraries apply.

Base URL

https://api.parasail.io/v1

Client setup

from openai import OpenAI

client = OpenAI(
    base_url="https://api.parasail.io/v1",
    api_key="<PARASAIL_API_KEY>",
)

Key differences from OpenAI

• Use the Parasail URL instead of OpenAI's.

• Use your Parasail API key (create one here).

• Use an open source model from Hugging Face instead of ChatGPT.

• Up to 50,000 requests and 1000 MB per input file (vs OpenAI's 250 MB).

OpenAI API reference

• OpenAI Batch API reference

• OpenAI Files API reference

Batch helper library

Parasail provides a Python batch helper library that simplifies batch submission and monitoring:

pip install openai-batch

See the openai-batch GitHub repository and batch full guide for details.

Batch file format

The batch input file is a JSONL file where each line is a request. The format is 100% compatible with OpenAI. See Batch file format.

Batch priority

Batches have a default priority of 0. Set a higher priority by passing the PARASAIL_PRIORITY metadata key when creating the batch.

from openai_batch import Batch

with Batch() as batch:
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Prompt #{i}"}],
        )

    batch.submit(metadata={"PARASAIL_PRIORITY": "1"})

Use priority sparingly for urgent jobs. Standard jobs should omit this metadata and use the default priority.