Ollama Tool Calling: The Practical Function Calling Guide

Published on April 23, 2026 • Updated August 3, 2026 • 22 min read

Ollama tool calling — the same feature the OpenAI world calls function calling — lets a local LLM invoke real code. You send a tools array of JSON schemas with your chat request; a model with native tool support returns a structured tool_calls response naming a function and its arguments instead of prose; your code runs the function and feeds the result back for a final answer. No cloud API, no data leaving your machine.

That is the feature that turns a chatbot into an agent. Ollama added native tool support back in version 0.3.0 (July 2024), and it has matured across dozens of releases since — the current builds are reliable enough that I have replaced three OpenAI-based agents in my own stack with local Ollama equivalents.

The catch: function calling is the area where local LLMs are most uneven. Some models nail it. Some technically support it but produce garbage JSON. Some get confused above 3 tools. The official Ollama docs do not warn you. This guide does.

I tested seven popular models against ten real tool-calling tasks. I documented exactly which combinations work, where they break, and how to engineer around the failure modes. By the end, you will have a working multi-tool agent running fully on your machine — plus the LangChain, CrewAI, and MCP wiring to take it further.

---

Quick Start: First Tool Call in 90 Seconds {#quick-start}

# Install Ollama and pull a model that handles tools well
ollama pull llama3.1:8b

# tools_minimal.py
import ollama
import json

def get_weather(city: str) -> str:
    # Stub: in real life, hit a weather API
    return json.dumps({"city": city, "temp_c": 22, "condition": "sunny"})

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"},
            },
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What is the weather in Paris?"}]
res = ollama.chat(model="llama3.1:8b", messages=messages, tools=tools)

if res["message"].get("tool_calls"):
    for call in res["message"]["tool_calls"]:
        result = get_weather(**call["function"]["arguments"])
        messages.append(res["message"])
        messages.append({"role": "tool", "content": result, "name": call["function"]["name"]})
    final = ollama.chat(model="llama3.1:8b", messages=messages, tools=tools)
    print(final["message"]["content"])
else:
    print(res["message"]["content"])

Run it:

pip install ollama
python tools_minimal.py
# > "It is currently 22°C and sunny in Paris."

That is the entire shape of tool calling: model decides to invoke a tool, you execute it, you append the result, the model uses the result to write the final answer.

---

Which Models Actually Work {#models}

This is the question nobody answers honestly. Here is my benchmark across 10 tool-calling tasks (single-tool, multi-tool, error-recovery, and chained workflows). Score is "task completed correctly without intervention" out of 10.

My recommendations from that test round:

• Best small (under 16GB RAM): qwen2.5:7b or llama3.1:8b
• Best medium (32GB RAM): qwen2.5:14b
• Best large (96GB+ RAM): llama3.1:70b or firefunction-v2
• Avoid for tools: gemma2 family, phi3.5 for multi-tool work

One useful habit regardless of era: Ollama only exposes a real tools API for models whose chat template supports it — check the "Tools" capability badge on each model's Ollama library page before you build on it.

For a fuller comparison of these model families, see our best Ollama models guide. For coding-specific tool work, best local AI models for programming goes deeper.

Updated picks: Qwen 3, Hermes 4, and the current agent models {#current-picks}

As of this August 2026 update, the default tool-calling model to pull is qwen3:8b — roughly a 5.2 GB download at Q4_K_M, ~6-8 GB in use, Apache 2.0, native tool support, and the most consistent small model in our agent testing. The table above still describes the Llama 3.1 / Qwen 2.5 generation accurately, but if you are starting a new agent today, start here instead:

• qwen3:8b — the all-round pick for 8 GB cards and 16 GB Macs. Hybrid thinking mode (/think / /no_think) lets you trade latency for deeper planning per step.
• qwen3:30b-a3b — a 30B Mixture-of-Experts with only ~3B active params: reasons like a big model, runs close to a small one. ~18-20 GB at Q4, the sweet spot for a 24 GB card.
• llama3-groq-tool-use:8b — the function-calling specialist (~4.7 GB, 8K context). Fine-tuned purely for tool use; it scored 89.06% on the Berkeley Function Calling Leaderboard at launch. Narrow, but excellent as a dedicated tool-router.
• Hermes 4 14B — NousResearch's reasoning-plus-tools model, built on Qwen3-14B. It emits tool calls inside <tool_call> tags after a visible reasoning step, which makes calls easy to parse and debug. ~9-10 GB at Q4. No first-party entry in the official Ollama library at the time of writing — import the GGUF from the NousResearch repo or use a vetted community upload. Our Hermes agent setup guide covers it end to end.
• gemma4:31b — Gemma 4's 31B dense variant was built with native function-calling and structured JSON output. ~19-21 GB at Q4. Pull the size explicitly: the bare gemma4 tag defaults to the small E4B edge model.
• mistral-small3.2:24b — tuned for low-latency function calls (~14-15 GB at Q4), but its tool parser has had known teething issues in some Ollama builds. Confirm tool calls work on your version before shipping it.

For the full ranked breakdown by VRAM tier — including which model to pair with CrewAI, LangGraph, or Continue — see best Ollama models for AI agents and the reliability-focused best local LLMs for tool calling.

---

How Ollama Function Calling Actually Works {#how-it-works}

Ollama implements an OpenAI-compatible tool calling API. The flow is:

1. You send: messages + tools (JSON schemas)
2. Model returns either:
   a. A normal text message (no tool needed), or
   b. A "tool_calls" list with name + arguments
3. You execute each tool call locally
4. You append the tool result as a "tool" role message
5. You call the model again with the updated messages
6. Model returns the final natural-language answer

Ollama parses the model's structured output into the OpenAI tool-calls format under the hood. This works because Llama 3.1+, Qwen 2.5+, Qwen 3, Mistral, and similar models were post-trained on tool-calling data with consistent special tokens or JSON schemas.

Worth being explicit about, because it trips people up: the model never executes code or accesses the internet directly. It only decides which tool to call and what arguments to pass. Your code handles all execution — which is also why tool calling is safe to experiment with.

The other important consequence: the model decides whether to call a tool. If it thinks the question is conversational ("hello, who are you"), it will not invoke a tool even if one is available. This is correct behavior — but if your application requires structured output 100% of the time, enforce it at the application layer (see the patterns section below).

---

Step 1: Define Tools With Good Schemas {#schemas}

Tool schemas use JSON Schema. The quality of your schema directly determines the model's accuracy. Two principles:

1. Description is everything. The model picks tools and arguments based on the descriptions, not the names.
2. Be strict. Specify required fields, enum values, and exact types. Loose schemas → loose calls.

A well-defined tool:

search_tool = {
    "type": "function",
    "function": {
        "name": "search_internal_docs",
        "description": (
            "Search the company's internal documentation for relevant content. "
            "Use this when the user asks about company policies, procedures, "
            "engineering wikis, or internal codebases. Do not use for public knowledge."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "Search keywords (3-8 words). Use specific technical terms.",
                },
                "department": {
                    "type": "string",
                    "enum": ["engineering", "hr", "security", "finance", "all"],
                    "description": "Filter by department; use 'all' if unknown.",
                },
                "max_results": {
                    "type": "integer",
                    "description": "Number of results to return (1-10).",
                    "default": 5,
                },
            },
            "required": ["query", "department"],
        },
    },
}

A bad version of the same tool:

{
    "type": "function",
    "function": {
        "name": "search",
        "description": "Search docs",
        "parameters": {
            "type": "object",
            "properties": {
                "q": {"type": "string"},
            },
        },
    },
}

The bad version will fire on every question, miss the department filter, and pass weird queries. Description quality is the difference between a 6/10 tool agent and a 9/10 tool agent.

Two schema guardrails worth copying

The description field is also where your safety policy lives. Two patterns I use in every production agent — a database tool that declares itself read-only, and a side-effect tool that requires explicit user intent:

{
    "type": "function",
    "function": {
        "name": "query_database",
        "description": "Run a read-only SQL query against the application database. Only SELECT queries are allowed.",
        "parameters": {
            "type": "object",
            "properties": {
                "sql": {"type": "string", "description": "SQL SELECT query"},
                "limit": {"type": "integer", "description": "Max rows to return (default 10)"}
            },
            "required": ["sql"]
        }
    }
}

{
    "type": "function",
    "function": {
        "name": "send_email",
        "description": "Send an email. Use only when the user explicitly asks to send an email.",
        "parameters": {
            "type": "object",
            "properties": {
                "to": {"type": "string", "description": "Recipient email address"},
                "subject": {"type": "string", "description": "Email subject line"},
                "body": {"type": "string", "description": "Email body text"}
            },
            "required": ["to", "subject", "body"]
        }
    }
}

The description steers the model, but never rely on it alone: enforce the SELECT-only rule and the "explicitly asks" rule in your dispatcher code too. The model proposes; your code disposes.

---

Step 2: Multi-Tool Agent Pattern {#multi-tool}

Real applications expose multiple tools. The agent loop must handle: zero tools called, one tool, multiple tools in one turn, and chained tools across turns.

# agent.py
import ollama
import json

# --- Tool implementations ---
def get_weather(city: str) -> str:
    return json.dumps({"city": city, "temp_c": 22, "condition": "sunny"})

def search_news(query: str, limit: int = 3) -> str:
    return json.dumps([
        {"title": f"Result about {query}", "url": "https://example.com/1"}
    ])

def calculate(expression: str) -> str:
    try:
        return json.dumps({"result": eval(expression, {"__builtins__": {}}, {})})
    except Exception as e:
        return json.dumps({"error": str(e)})

TOOL_REGISTRY = {
    "get_weather": get_weather,
    "search_news": search_news,
    "calculate": calculate,
}

TOOLS_SCHEMA = [
    {"type": "function", "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "City name."}},
            "required": ["city"],
        },
    }},
    {"type": "function", "function": {
        "name": "search_news",
        "description": "Search recent news headlines.",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search keywords."},
                "limit": {"type": "integer", "description": "Number of results.", "default": 3},
            },
            "required": ["query"],
        },
    }},
    {"type": "function", "function": {
        "name": "calculate",
        "description": "Evaluate a math expression. No variables.",
        "parameters": {
            "type": "object",
            "properties": {"expression": {"type": "string"}},
            "required": ["expression"],
        },
    }},
]

# --- Agent loop ---
def run_agent(user_question: str, model="qwen2.5:7b", max_turns=6):
    messages = [
        {"role": "system", "content": (
            "You are a careful assistant. Use the provided tools when needed. "
            "Do not invent tool results. If a tool fails, explain what happened "
            "and try a different approach."
        )},
        {"role": "user", "content": user_question},
    ]

    for turn in range(max_turns):
        res = ollama.chat(model=model, messages=messages, tools=TOOLS_SCHEMA)
        msg = res["message"]
        messages.append(msg)

        tool_calls = msg.get("tool_calls") or []
        if not tool_calls:
            return msg["content"]

        for call in tool_calls:
            name = call["function"]["name"]
            args = call["function"]["arguments"]
            if name not in TOOL_REGISTRY:
                result = json.dumps({"error": f"unknown tool: {name}"})
            else:
                try:
                    result = TOOL_REGISTRY[name](**args)
                except TypeError as e:
                    result = json.dumps({"error": f"bad arguments: {e}"})
                except Exception as e:
                    result = json.dumps({"error": str(e)})
            messages.append({"role": "tool", "name": name, "content": result})

    return "Reached max turns without a final answer."

if __name__ == "__main__":
    print(run_agent("What is the weather in Tokyo, and what is 17 * 23?"))

Key patterns to copy:

1. TOOL_REGISTRY dispatch: maps tool names to Python callables.
2. Bounded loop: max_turns prevents runaway loops if the model keeps calling tools.
3. Error wrapping: every tool call is wrapped in try/except and returns JSON, so the model can recover gracefully.
4. System prompt: enforces grounded behavior without inventing tool results.

This pattern handles 90%+ of practical tool-calling needs.

---

Step 3: Tool Use From JavaScript / TypeScript {#javascript}

For Node and browser apps, the official ollama JS package exposes the same API.

// agent.ts
import ollama from "ollama";

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get current weather for a city.",
      parameters: {
        type: "object",
        properties: { city: { type: "string", description: "City name." } },
        required: ["city"],
      },
    },
  },
];

const TOOLS: Record<string, (args: any) => Promise<string>> = {
  get_weather: async ({ city }) =>
    JSON.stringify({ city, temp_c: 22, condition: "sunny" }),
};

async function runAgent(question: string) {
  const messages: any[] = [{ role: "user", content: question }];

  for (let turn = 0; turn < 6; turn++) {
    const res = await ollama.chat({
      model: "qwen2.5:7b",
      messages,
      tools,
    });
    messages.push(res.message);

    const calls = res.message.tool_calls ?? [];
    if (calls.length === 0) return res.message.content;

    for (const c of calls) {
      const fn = TOOLS[c.function.name];
      const result = fn ? await fn(c.function.arguments) : "{}";
      messages.push({ role: "tool", name: c.function.name, content: result });
    }
  }
  return "Hit max turns.";
}

runAgent("Weather in Paris?").then(console.log);

For full Node/Next.js patterns including streaming and the Vercel AI SDK, see our companion guide on Ollama with JavaScript and TypeScript.

---

Step 4: Common Patterns {#patterns}

Pattern 1: Forcing a tool call. Ollama's native API leaves the call/no-call decision to the model — there is no reliable server-side switch to force one. When a turn absolutely must produce a tool call, enforce it at the application layer: tell the model in the system prompt ("For this request you MUST call one of the provided tools — do not answer in prose"), then validate the response and re-prompt once if it answered in text anyway. If you need a hard guarantee of structured output and do not need tool execution, skip tools and use JSON mode instead:

ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Extract entities from: ..."}],
    format="json",
)

Pattern 2: Structured output without tools. Same format="json" mode — use it when you want JSON you parse yourself rather than functions the model invokes. Extraction, classification, and form-filling all belong here.

Pattern 3: Tool result chaining. When tool A's output feeds tool B, structure tool descriptions to encourage the chain:

"Use search_news first to find article URLs, then summarize_article on each URL."

The model handles the orchestration if your descriptions are explicit.

Pattern 4: Cost-effective routing. Use a small model (qwen2.5:7b or qwen3:8b) for tool selection, hand off to a larger model for the final synthesis. Saves significant time on multi-step agents.

---

Step 5: Error Handling and Retry {#error-handling}

Tool calls fail. Networks drop. APIs return weird JSON. The agent must survive.

def safe_call_tool(tool_fn, args, retries=2):
    last_error = None
    for attempt in range(retries + 1):
        try:
            return tool_fn(**args)
        except Exception as e:
            last_error = str(e)
            if attempt < retries:
                continue
            return json.dumps({"error": f"tool failed after {retries+1} attempts: {last_error}"})

Three failures to plan for:

1. Bad arguments from the model. The model passes a string where you wanted an int. Wrap in try/except and return a structured error so the model can retry.
2. Tool downtime. External APIs return 500s. Always set a timeout and return an error JSON.
3. Hallucinated tool names. The model sometimes invents tool names that do not exist. Catch this in dispatch and return a list of valid tool names to help the model recover.

A robust dispatcher:

def dispatch(name: str, args: dict) -> str:
    if name not in TOOL_REGISTRY:
        return json.dumps({
            "error": f"unknown tool: {name}",
            "available_tools": list(TOOL_REGISTRY.keys()),
        })
    return safe_call_tool(TOOL_REGISTRY[name], args)

The agent recovers gracefully because it sees the available tools and corrects on the next turn.

---

Step 6: Streaming With Tool Calls {#streaming}

Tool calls and streaming have a tricky interaction. In older Ollama builds the tool-call payload arrived only at the end of the stream; newer versions can stream tool calls incrementally as well (see Ollama's own streaming tool calls announcement). The robust pattern handles both — accumulate text as it arrives, and check for tool calls on every chunk as well as at the end:

stream = ollama.chat(
    model="qwen2.5:7b",
    messages=messages,
    tools=tools,
    stream=True,
)

text_parts = []
final_message = None
for chunk in stream:
    msg = chunk.get("message", {})
    if msg.get("content"):
        text_parts.append(msg["content"])
        print(msg["content"], end="", flush=True)
    if chunk.get("done"):
        final_message = msg

if final_message and final_message.get("tool_calls"):
    # process tool calls as usual
    ...

For text-only responses, streaming gives you token-by-token UI updates. For tool-driven responses, the user sees nothing until the tools resolve. To improve UX, render a "Calling search_internal_docs..." indicator the moment you see a tool call.

---

Benchmarks: Latency and Reliability {#benchmarks}

Tested on a MacBook Pro M3 (16GB) with three tools registered, 50 questions per model:

Schema-correct rate = (model produced argument JSON that validated against the schema) / total calls

For most apps qwen2.5:7b is the best balance of latency and reliability in this generation (qwen3:8b is its natural successor — see the updated picks above). llama3.2:3b is fastest but unreliable above 1 tool.

---

Using Tool Calling With LangChain and CrewAI {#frameworks}

You do not have to write the agent loop yourself: LangChain, CrewAI, and LangGraph all speak Ollama's tools API and run the call-execute-respond cycle for you. LangChain binds tools onto a ChatOllama instance; CrewAI takes functions decorated as tools and hands them to role-based agents. Use a framework when you need memory, delegation, or parallel tool execution — use the raw loop from this guide when you want zero dependencies and full control.

LangChain

from langchain_ollama import ChatOllama
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    # Your implementation
    return f"Weather in {city}: 22°C, sunny"

llm = ChatOllama(model="llama3.1")
llm_with_tools = llm.bind_tools([get_weather])

result = llm_with_tools.invoke("What's the weather in Paris?")

bind_tools() converts your decorated functions into the same JSON schemas you saw earlier — everything about description quality still applies. Our Ollama + LangChain integration guide covers chains, memory, and streaming on top of this.

CrewAI

from crewai import Agent
from crewai.tools import tool

@tool("Search Tool")
def search(query: str) -> str:
    """Search the web for information."""
    # Your implementation
    return "search results..."

researcher = Agent(
    role="Researcher",
    goal="Find accurate information",
    tools=[search],
    llm="ollama/llama3.1"
)

CrewAI handles the multi-turn tool loop automatically — including sending results back and getting the final answer. Setup details are in our CrewAI local setup guide, and if you are choosing between frameworks, the AI agent frameworks comparison weighs CrewAI against LangGraph and AutoGen.

---

Ollama and MCP: Where Tool Calling Meets the Ecosystem {#mcp}

Ollama is a model server, not an MCP client — so "Ollama MCP" means putting a bridge in the middle. The bridge (mcphost is the simplest) connects to one or more MCP servers, asks each for its tool manifest, converts those tools into the same tools array you have used throughout this guide, and executes whatever calls the model makes. Every pattern on this page applies unchanged; MCP just standardizes where the tools come from, so one filesystem or GitHub server works with every MCP-capable client instead of being rewritten per framework.

The fastest way to see it working — the official filesystem MCP server driven by a local model:

go install github.com/mark3labs/mcphost@latest
ollama pull qwen2.5:14b

Create ~/.mcp.json:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/you/Documents"]
    }
  }
}

mcphost -m ollama:qwen2.5:14b --config ~/.mcp.json

Ask it to "summarize the three most recent .md files in Documents" and watch it call list_directory and read_file through the MCP server — zero data leaves your machine.

That is the demo; the engineering lives in our dedicated Ollama + MCP integration guide — chaining multiple servers, writing your own MCP server, and which models pick the right tool reliably. Running llama.cpp instead of Ollama? The llama.cpp MCP server guide covers that route.

---

Pitfalls and Gotchas {#pitfalls}

1. The model sometimes ignores tools and answers from training data. Solution: explicit system prompt — "If you do not have current information, you MUST call a tool. Do not answer from memory."

2. Argument types are inconsistent. A model may return "limit": "5" (string) when you specified integer. Coerce types in the dispatcher: int(args.get("limit", 5)).

3. Tool descriptions over 200 chars hurt accuracy. Keep them under 200 chars. Move long context into the system prompt, not the schema.

4. Too many tools = degraded performance. Above 6-8 tools, even good models start mis-routing. Group related tools or split into sub-agents.

5. Models call tools redundantly. They sometimes call get_weather twice in a row for the same city. Add deduplication at the dispatcher: cache results per turn.

6. Local models lag cloud models on chained reasoning. A single tool call is solid; 5+ chained calls is where local models still trail the frontier cloud models. Use larger models or break the workflow into smaller steps.

7. Memory pressure on long agent loops. Each turn appends to the message history. After 10 turns, context can hit 8K+ tokens. Trim older tool results when they are no longer relevant.

8. JSON mode is not tool calling. format="json" returns JSON in the content field but does not invoke tools. Different feature, different use case.

---

Production Hardening {#production}

For a production agent:

• Per-tool timeout (30s default, lower for fast tools)
• Bounded max_turns (4-8 for most agents)
• Structured error responses with retry hints
• Logging of every tool call and result (auditability)
• Rate limiting on expensive tools (web fetches, paid APIs)
• Schema validation of tool arguments before execution
• Dedup of identical consecutive tool calls
• Concurrent tool execution when safe (asyncio gather)
• Graceful fallback to text-only mode if tools repeatedly fail
• Unit tests for each tool and an integration test for the full loop

For broader production patterns including auth, monitoring, and multi-user concurrency, our Ollama production deployment guide covers the hosting layer. For knowledge-augmented agents, pair this with the Ollama + ChromaDB RAG pipeline.

---

Real Use Cases I Have Shipped {#use-cases}

Three agents I run in production today, all on Ollama:

1. Internal support bot. Tools: search_docs, lookup_user, create_jira. Model: qwen2.5:14b. Replaced a Zendesk AI add-on. ~85% deflection rate.

2. Personal finance assistant. Tools: get_transactions, categorize, forecast_balance, flag_anomalies. Model: llama3.1:8b. Runs nightly, sends summary email.

3. Research agent. Tools: search_arxiv, fetch_paper, summarize_paper, save_to_obsidian. Model: llama3.1:70b on Mac Studio. Replaced ChatGPT + manual paper-reading workflow.

In all three cases, the value is not raw model intelligence — it is the LLM acting as a careful router across a small set of well-defined tools. That is exactly what local LLMs are good at.

---

FAQ {#faq}

What is the difference between tool calling and function calling?

Nothing — they are two names for the same capability. "Function calling" was coined by OpenAI when the feature launched in GPT-3.5/4; "tool calling" is the broader term used by Anthropic, Meta, and Ollama. In both cases the model reads your function definitions, decides when to use one, and returns structured JSON with the function name and arguments. Ollama's API uses the tools parameter.

Which Ollama model is best for tool calling?

Today: qwen3:8b for 8-16 GB machines, qwen3:30b-a3b for a 24 GB card, and llama3-groq-tool-use:8b when the agent does nothing but call functions. From the previous generation, qwen2.5:7b and llama3.1:8b remain solid (both scored 8/10 in our tests), with qwen2.5:14b the step-up at 9/10. Avoid gemma2 and phi3.5 for multi-tool work. Full ranking: best Ollama models for AI agents.

Does Ollama function calling work the same as the OpenAI API?

Largely yes. Ollama implements an OpenAI-compatible tools schema using the same JSON Schema format and the same message types (assistant tool_calls, tool role responses). You can usually port OpenAI tool-calling code by changing the base URL and model. The differences: chained reasoning over 5+ tools is weaker on local models, and JSON adherence varies more by model.

How many tools can I expose without degrading performance?

Six to eight is the practical ceiling for 7B-14B models; 70B-class models handle more. Above your model's ceiling, even good models start mis-routing. If you have more capabilities, group related ones into a single tool with an enum action parameter, or split your agent into sub-agents (a router agent that picks a specialist per turn). Only define the tools relevant to the current task, not everything you have.

How do I debug tool calling issues?

Work through four checks: (1) Model ignores tools → confirm the model actually has the Tools badge and the tools array is correctly formatted. (2) Invalid JSON in arguments → drop temperature to 0.1-0.3. (3) Wrong tool selected → make the descriptions more specific and more distinct from each other. (4) Still stuck → run the server with OLLAMA_DEBUG=1 to see the raw model output before tool parsing.

How do I prevent the model from making up tool results instead of calling tools?

Three things: (1) a system prompt with an explicit "If you do not have current information, you MUST call a tool. Do not answer from memory." (2) validate every turn — if a required tool call did not happen, re-prompt once. (3) a lower temperature (0.1-0.3) to reduce hallucination. Letting the model decide is right for chatbots but wrong for structured workflows.

Can I stream the response while using tools?

Yes. Set stream=True. Text streams token-by-token; tool calls arrive as a final chunk on older Ollama builds and can stream incrementally on newer ones — handle both. Render text as it streams, show a "Calling tool_name..." indicator the moment a tool call appears, then run the tool and continue. The user only waits during tool execution.

What is the difference between JSON mode and function calling?

JSON mode (format="json") forces well-formed JSON in the content field — no tool execution, you parse it yourself. Function calling exposes a tool registry the model decides when to invoke, runs actual code, and feeds results back for synthesis. Use JSON mode for structured extraction; use function calling for agents.

Can the model call multiple tools in a single turn?

Yes. Ollama returns a tool_calls array, not a single call — a model can invoke 2-4 tools in one turn for parallelizable queries like "get weather in Paris, London, and Tokyo." Iterate the whole array, append all results, then call the model once for synthesis. This is the single biggest perf win for multi-tool agents.

Does Ollama tool calling work with LangChain or CrewAI?

Yes — see the frameworks section above. LangChain via ChatOllama + bind_tools(), CrewAI via its @tool decorator and llm="ollama/...", and LangGraph with explicit tool nodes for the most control. All three run the multi-turn call-execute-respond loop for you.

---

Closing Take {#closing}

Function calling is what makes local LLMs genuinely useful for real workflows. Anyone can build a local chatbot. Building a local agent that books meetings, searches your docs, runs SQL queries, and summarizes the results — that is the unlock. Today's Ollama is good enough for production tool calling on the right models with the right schemas.

If you are starting today, my exact recipe: qwen3:8b for development, the agent loop pattern above, three to five well-described tools, a tight system prompt, and an evaluation harness with 20 prompts that exercise every tool. Ship that and iterate.

Sources and further reference: Ollama API documentation · Ollama tool support announcement · Ollama streaming tool calls · Hugging Face Llama 3.1 tool-calling deep dive · LangChain Ollama integration