Tool calling on open-weight models: why it depends on who runs them

How tool calling works

You describe the functions your code offers — a name, a description and a JSON schema for the arguments — in the tools field of the request. The model then either answers in text or returns tool_calls: the function it wants and the arguments it chose. Your code runs the function, sends the result back in a message with the role “tool”, and the model continues. That loop is what agent frameworks are built on.

The model never runs anything itself. It writes a request, and your code decides whether to carry it out. That is also where you keep control: which functions exist, what they are allowed to touch, and what gets logged.

Two details trip up first implementations. A model may ask for several calls in one answer, so treat tool_calls as a list. And each result you send back must carry the id of the call it answers, in tool_call_id, so the model can match results to requests.

It depends on the serving stack, not only the weights

For a tool call to come back clean, the operator's serving software has to apply the model's chat template, recognise the special tokens that mark a call, and turn them into the tool_calls structure. An operator can leave the feature off, or support it for one model and not for another.

Qwen2.5-VL 72B showed the gap. Its card says it “can reason and dynamically direct tools” (model card, read on 6 October 2026); yet OVHcloud, which ran it for Evanesca, answered a tool call with an HTTP 400 error — “feature 'tool calls' is not currently supported” — when it was measured on 6 August 2026, so it was served without tool calling until its pause on 20 October 2026. Pixtral 12B, which now reads images at Scaleway, returned a clean tool call when it was measured that day. Templates matter too: Mistral AI lists a “more robust” function-calling template among the changes in Mistral Small 3.2 (model card, read the same day).

That is why Evanesca's catalogue sets the tool-calling flag only after a real call at the operator that serves the model. A model without the flag has not been measured; it is never assumed from the weights. If you serve models yourself, that wiring is yours to configure in the inference server you choose.

Check it in one call

You do not need a benchmark to know whether an operator serves tool calling for a model. Declare one small function and ask a question that requires it:

curl https://evanesca.app/api/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-26b",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather in a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

A good answer stops with finish_reason “tool_calls” and names the function you declared. Its arguments arrive as a string that must parse as JSON, with the required fields and sensible values — here, the city you asked about:

"finish_reason": "tool_calls",
"message": {
  "role": "assistant",
  "tool_calls": [{
    "id": "call_…",
    "type": "function",
    "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
  }]
}

How it fails in practice

• Arguments that do not parse: prose, a truncated string, or JSON encoded twice. The call “succeeds”, and your parser throws.
• A plain-text answer instead of a call — “I can't check the weather” — which quietly ends an automated loop instead of raising an error.
• A function that was never declared, or the right one with missing or invented parameters.
• A reasoning model that spends its max_tokens thinking and stops before it writes the call.

A test that only checks whether tool_calls is present misses most of these. Check the name, parse the arguments, and compare their values with what the prompt asked for.

Which models call tools on Evanesca

Building a loop that holds

1. Validate every call against your schema before running it, and refuse functions you did not declare.
2. When a call is malformed, send the error back as the tool result: most models correct themselves on the next turn.
3. Cap the number of rounds and set a timeout, so a confused model cannot loop for ever.
4. Keep permissions in your code: read-only functions by default, and a human confirmation before anything that writes, pays or deletes.
5. Log the tokens of each round. Every round is a new request that carries the whole conversation, and a reasoning model also bills its thinking each time.

Reasoning and tool calling often go together in agents, and the cost adds up quickly; what thinking costs explains how to measure it.

Agents, data and jurisdiction

Tool results become part of the prompt. Whatever your function returns — a customer record, an email, a row from a database — is sent to the model's operator on the next turn. The question of who runs the model therefore covers your tool results as well as your questions.

On Evanesca, every model runs in Europe — most of them in France, at Scaleway in Paris, with OVHcloud in Gravelines as the fallback for Qwen3.8 27B, GPT-OSS 120B, Mistral Small 3.2 and Llama 3.3 70B (API documentation). Evanesca does not store prompts or tool results: they stay in memory while the answer is written. What the operators keep is theirs to say: Scaleway may keep and read requests that cause errors or look abusive, for up to 14 days (Scaleway, read on 6 October 2026), and Mistral AI keeps its models' inputs and outputs 30 days to monitor abuse (Mistral AI, read on 12 October 2026).

Frequently asked questions

Read next

• Open-weight reasoning models: what thinking costs, and how to choose oneReasoning models think before they answer, and that thinking is billed as output. Which open-weight models reason, what it costs, and how to choose one.
• Open-weight AI models in 2026: which to use for what, and who runs themThe open-weight models of 2026 — GLM, Qwen, DeepSeek, GPT-OSS, Llama, Mistral — sorted by use, with size, licence, price and the company that runs them.
• What is open-weight AI, and what can you do with the weights?An open-weight model publishes its weights, the numbers it learned in training. What you get, what you don't, how much memory they need, and who can run them.