# Tool/Function Calling

### Overview

Tool/function calling enables language models (LLMs) to interact seamlessly with external APIs or services, automatically determining when external actions are necessary and executing structured requests accordingly.

### Supported Models

| Model                                  | Tools | Tool Choice |
| -------------------------------------- | ----- | ----------- |
| parasail-llama-33-70b-fp8              | ✅     | ✅           |
| parasail-llama-4-scout-instruct        | ✅     | ✅           |
| parasail-llama-4-maverick-instruct-fp8 | ✅     | ✅           |
| parasail-qwen3-30b-a3b                 | ✅     | ✅           |
| parasail-qwen3-235b-a22b               | ✅     | ✅           |
| parasail-qwen3-32b                     | ✅     | ✅           |
| parasail-mistral-devstral-small        | ✅     | ✅           |

### Quickstart Guide

#### Step 1: Define Your Tool Schema

Define a clear schema describing the tool functionality:

```
tool_schema = {
  "name": "get_weather",
  "description": "Retrieve weather information for a given location and date.",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {"type": "string"},
      "date": {"type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}$"}
    },
    "required": ["location", "date"]
  }
}
```

#### Step 2: Example User Query

```
example_sentence = "What's the weather like in Manhattan Beach on 2025-06-03?"
```

### Using Parasail with OpenAI-Compatible REST Endpoint

#### Example Implementation

```
import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.parasail.io/v1",
    api_key=os.getenv("PARASAIL_API_KEY")
)

response = client.chat.completions.create(
    model="parasail-llama-4-scout-instruct",
    messages=[{"role": "user", "content": example_sentence}],
    tools=[tool_schema],
    tool_choice="auto"
)

args = response.choices[0].message.tool_calls[0].function.arguments
print(json.dumps(args, indent=2))
```

#### Expected Output

```
{
  "location": "Manhattan Beach",
  "date": "2025-06-03"
}
```

*Note:* Schema validation occurs automatically via Parasail's API gateway.

### Using Parasail's vLLM Client

#### Example Implementation

```
from vllm import LLM, SamplingParams
import os
import json

vlm = LLM(
    model="parasail-llama-4-scout-instruct",
    api_key=os.getenv("PARASAIL_API_KEY"),
    base_url="https://api.parasail.io/v1/vllm"
)

prompt = f"""
You have access to this tool:
{json.dumps(tool_schema, indent=2)}

User Query: "{example_sentence}"

Respond **only** in JSON:
{{
  "tool_name": "get_weather",
  "parameters": {{
    "location": "",
    "date": ""
  }}
}}
"""

params = SamplingParams(temperature=0, max_tokens=100, stop=["}"])
raw_output = vlm.generate([prompt], params)[0].outputs[0].text.strip() + "}"
tool_call = json.loads(raw_output)

print(json.dumps(tool_call, indent=2))
```

#### Expected Output

```
{
  "tool_name": "get_weather",
  "parameters": {
    "location": "Manhattan Beach",
    "date": "2025-06-03"
  }
}
```

### Integrating Your Tool

Implement your external service (for example, weather service) to handle the provided parameters:

#### Example Weather Service Implementation

```
def get_weather_service(params):
    location = params['location']
    date = params['date']

    # Example logic to fetch weather data
    # Replace this with actual API calls or logic
    weather_info = {
        "location": location,
        "date": date,
        "weather": "Sunny",
        "temperature": "75°F"
    }

    return weather_info

weather_data = get_weather_service(tool_call["parameters"])
print(json.dumps(weather_data, indent=2))
```

#### Expected Service Output

```
{
  "location": "Manhattan Beach",
  "date": "2025-06-03",
  "weather": "Sunny",
  "temperature": "75°F"
}
```

### Switching Between REST and vLLM

Both methods utilize the same schema, simplifying your workflow and allowing seamless integration and switching between the REST API and vLLM client, depending on your application's needs.