> For the complete documentation index, see [llms.txt](https://docs.parasail.io/parasail-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.parasail.io/parasail-docs/use-cases/batch-processing.md).

# Batch Processing at Scale

Process thousands or millions of inferences offline at 50% off serverless pricing. Batch is ideal for evaluations, data processing, content generation, and any workload that doesn't need real-time responses.

## Why batch?

* **50% off** serverless token pricing
* **Additional 50% off** for cached tokens
* **Up to 50,000 requests** and **1000 MB** per input file
* **OpenAI-compatible**—use the same batch file format and API
* **Any HuggingFace model**—no dedicated deployment needed
* **Resumable**—submit in one process, monitor in another

## Quickstart

```python
import random
from openai_batch import Batch

with Batch() as batch:
    objects = ["cat", "robot", "coffee mug", "spaceship", "banana"]
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Tell me a joke about a {random.choice(objects)}"}]
        )
    result, output_path, error_path = batch.submit_wait_download()
    print(f"Batch completed with status {result.status}")
```

See the [batch quickstart](/parasail-docs/quickstart/batch.md) for the full guide.

## What you can batch

| Task                | Example models               | Guide                                                                                        |
| ------------------- | ---------------------------- | -------------------------------------------------------------------------------------------- |
| Chat completions    | Any HuggingFace transformer  | [Batch full guide](/parasail-docs/products/quickstart.md)                                    |
| Embeddings          | GritLM, GTE-Qwen2            | [Batch embedding models](/parasail-docs/products/quickstart.md#batch-embedding-models)       |
| Image understanding | Qwen3-VL-8B-Instruct         | [Batch image understanding](/parasail-docs/products/quickstart.md#batch-image-understanding) |
| Private models      | Any private HuggingFace repo | [Batch private models](/parasail-docs/products/quickstart.md#private-models)                 |

## Pricing

Batch is billed at **50% of serverless pricing**, based on parameter count. FP8 models incur no additional cost; FP16 models are 30% more. See [full pricing details](/parasail-docs/billing/pricing.md#batch-pricing).

## Relevant pages

* [Batch quickstart](/parasail-docs/products/quickstart.md)—comprehensive guide
* [Batch helper GitHub repo](https://github.com/parasail-ai/openai-batch)—library source and README
* [File format](/parasail-docs/products/quickstart/file-format.md)—JSONL input format
* [API reference](/parasail-docs/api-reference/batch-api.md)—OpenAI-compatible batch API
* [Priority](/parasail-docs/api-reference/batch-api.md#batch-priority)—batch priority levels
* [Troubleshooting](/parasail-docs/products/quickstart/troubleshooting.md)