> For the complete documentation index, see [llms.txt](https://docs.parasail.io/parasail-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.parasail.io/parasail-docs/quickstart/batch.md).

# Batch Processing

Process thousands or millions of LLM inferences at 50% off serverless pricing. Batch is ideal for evaluations, data processing, and any workload that doesn't require real-time responses.

## 1. Get an API key

1. Go to <https://www.saas.parasail.io/keys>.
2. Click **Create API Key**.
3. Copy the key—it's shown only once.

Store it in your environment:

```bash
export PARASAIL_API_KEY="your-api-key-here"
```

## 2. Install the batch helper library

```bash
pip install openai-batch
```

The library is 100% compatible with both Parasail and OpenAI. Source code: [github.com/parasail-ai/openai-batch](https://github.com/parasail-ai/openai-batch).

## 3. Submit your first batch

```python
import random
from openai_batch import Batch

with Batch() as batch:
    objects = ["cat", "robot", "coffee mug", "spaceship", "banana"]
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Tell me a joke about a {random.choice(objects)}"}]
        )
    result, output_path, error_path = batch.submit_wait_download()
    print(f"Batch completed with status {result.status} and stored in {output_path}")
```

Run it:

```bash
PARASAIL_API_KEY=<your-api-key> python3 test_batch.py
```

## 4. Monitor via the Batch UI

The Parasail Batch UI lets you upload batches, track status, and download results. Open the **Batch** tab in the dashboard.

## Key features

* **50% off serverless pricing**—cached tokens get an additional 50% off.
* **OpenAI-compatible**—works with the OpenAI batch file format and API.
* **Any HuggingFace model**—no need for a dedicated deployment; just specify the HuggingFace ID.
* **Embeddings**—supports embedding models, not just chat.
* **Resumable**—submit a batch, then monitor and download in a separate process using the batch ID.
* **Up to 50,000 requests** and **1000 MB** per input file.

## Next steps

* [Batch full guide](/parasail-docs/products/quickstart.md)—comprehensive batch processing guide
* [Batch helper GitHub repo](https://github.com/parasail-ai/openai-batch)—library source and README
* [Batch file format](/parasail-docs/products/quickstart/file-format.md)—JSONL input format (OpenAI-compatible)
* [Batch API reference](/parasail-docs/api-reference/batch-api.md)—OpenAI-compatible batch API spec
* [Batch with private models](/parasail-docs/products/quickstart.md#private-models)—batch processing with private HuggingFace repos