Troubleshooting

Diagnose and fix common batch job failures, from JSONL validation errors to quota and authentication issues.

This page covers the most common problems you'll hit when submitting and processing batch jobs, and how to resolve them. Parasail's batch service is 100% compatible with the OpenAI batch API, so the file formats, status values, and error codes described here match the OpenAI batch model.

JSONL validation failures

A batch input file is a .jsonl file where each line is a single, complete JSON object representing one request. A batch will fail validation if any line is malformed or missing required fields.

Each request line must include:

• custom_id: a unique value you create so you can later match outputs to inputs.

• method: the HTTP method, currently POST.

• url: one of /v1/chat/completions or /v1/embeddings.

• body: the same JSON body you would send to an interactive request.

Common causes of validation failures:

• Malformed lines. A line isn't valid JSON (trailing comma, unescaped quote, or a request split across multiple lines). Each request must be exactly one line.

• Missing required keys. A line is missing custom_id, method, url, or body.

• Duplicate or missing custom_id. Each custom_id should be unique so responses can be matched back unambiguously.

• Wrong url. Only /v1/chat/completions and /v1/embeddings are supported.

• stream set in the body. For chat completions, stream must be omitted or set to false.

To debug, validate the file locally before submitting:

import json

with open("example_batch_input.jsonl") as f:
    for i, line in enumerate(f, start=1):
        try:
            req = json.loads(line)
        except json.JSONDecodeError as e:
            print(f"Line {i} is not valid JSON: {e}")
            continue
        for key in ("custom_id", "method", "url", "body"):
            if key not in req:
                print(f"Line {i} is missing required key: {key}")

See Batch file format for the full request and response schema.

File-size and request-count limits exceeded

Parasail limits the size of each batch input file:

• Up to 50,000 requests (lines) per input file.

• Up to 1000 MB total input file size.

Workloads that exceed either limit must be split into multiple batches. If you use the Parasail Batch Helper Library, add_to_batch raises a ValueError when the file-size or request-count limit for the provider is exceeded. Catch it and roll over into a new batch:

from openai_batch import Batch

def submit_in_chunks(requests):
    batch_ids = []
    batch = Batch()
    for req in requests:
        try:
            batch.add_to_batch(**req)
        except ValueError:
            # Current batch is full; submit it and start a new one.
            batch_ids.append(batch.submit())
            batch = Batch()
            batch.add_to_batch(**req)
    batch_ids.append(batch.submit())
    return batch_ids

If you submit through the API or Batch UI directly, split the .jsonl file yourself so that each file stays under both limits.

Jobs stuck in validating or in_progress

A batch moves through validating, then in_progress, then a terminal state (completed, failed, expired, or cancelled). Jobs run on Parasail's servers, so a long-running batch is processed to completion even if your local script crashes, resets, or exits.

You do not need to keep a process running to wait for results. Record the batch ID at submission time and resume monitoring later from a separate process:

from openai_batch import Batch
import time

with Batch(batch_id="batch-tclfzwczcd",
           output_file="mybatch.jsonl") as batch:
    while True:
        status = batch.status()
        print(f"Batch status: {status.status}")
        if status.status in ["completed", "failed", "expired", "cancelled"]:
            break
        time.sleep(60)  # Check every minute

    output_path, error_path = batch.download()
    print(f"Output saved to: {output_path}")

You can also track progress, view metadata, or cancel a queued or running batch from the Batch UI.

Partial failures

A batch can complete with some requests succeeding and others failing. Failures are reported per request rather than failing the whole batch, so always inspect each output line.

Two things to keep in mind when reading output:

• The order of responses may differ from the order of requests in the input file.

• Match responses to requests using custom_id, not by line position.

Each output line contains a response object with a status_code. A 200 means the request succeeded; any other code is the same HTTP error you would have received for that request interactively.

import json

results = {}
with open("mybatch.jsonl") as f:
    for line in f:
        record = json.loads(line)
        custom_id = record["custom_id"]
        response = record.get("response") or {}
        status = response.get("status_code")
        if status == 200:
            results[custom_id] = response["body"]
        else:
            print(f"{custom_id} failed with status {status}: {record.get('error')}")

Failed requests can be collected and resubmitted in a new batch.

Quota errors

If a batch can't be scheduled because your organization is out of GPU capacity, you'll see an "Insufficient quota" message. By default, each organization has a quota of 4 GPUs, shared across both Batch and Dedicated services regardless of GPU type. A single batch that requires more GPUs than your remaining quota will be rejected.

To resolve this:

• Wait for other jobs that are consuming GPUs to finish, or

• Request more capacity through the Quota Increase form. See GPU quota for details. Increases up to 8 GPUs are available without additional justification.

Authentication errors (401)

The batch tooling reads your key from the PARASAIL_API_KEY environment variable. A 401 response, or an error about a missing API key, usually means the key isn't set or isn't valid.

• Confirm the variable is set in the environment that runs your script:

  PARASAIL_API_KEY=<your-key> python3 test_batch.py

• Store the key in a .env or shell profile rather than pasting it into code, where it can be leaked.

• Create or rotate keys at https://www.saas.parasail.io/keys.

• When using the OpenAI SDK directly, point the client at Parasail and pass the key explicitly:

  import os
  from openai import OpenAI
  
  client = OpenAI(
      base_url="https://api.parasail.io/v1",
      api_key=os.environ["PARASAIL_API_KEY"],
  )

Output parsing

Results are returned as an output .jsonl file, with one response per line. Parse it line by line as JSON and pull response.body out of each record (see the partial-failures snippet above for matching on custom_id).

For embeddings, Parasail strongly recommends submitting requests with "encoding_format": "base64", which returns each embedding as a Base64-encoded binary array instead of a plain-text float array. This typically reduces output file size by nearly 4x with no loss of precision, but it means you must decode the value before use:

import base64
import json
import numpy as np

with open("embeddings-output.jsonl") as f:
    for line in f:
        record = json.loads(line)
        for item in record["response"]["body"]["data"]:
            vector = np.frombuffer(base64.b64decode(item["embedding"]), dtype=np.float32)
            # use vector ...

Next steps

• Quick start

• Batch file format

• Batch API Reference

• GPU quota