bandarra.me

Does WebMCP `outputSchema` Help Agents Pick Better Tools?

Animated diagram illustrating how an LLM and JSON schema work together to generate structured data: an input prompt ("User/System Prompt") passes into a Language Model, while a predefined JSON schema defines the target output structure. The LLM streams raw tokens that are simultaneously parsed and validated against the schema constraints, ultimately compiling into a valid, formatted JSON object.

WebMCP recently added support for outputSchema (webmachinelearning/webmcp#254), allowing developers to declare the JSON Schema of the value a tool returns alongside its inputSchema.

When the spec change landed, I wanted to test a simple hypothesis: if a toolset already follows WebMCP best practices, when does declaring outputSchema actually improve how an agent selects and chains tools?

When Tool Consolidation Meets the 500-Character Limit

Two of WebMCP's core design rules pull in opposite directions on larger applications:

  1. Consolidate tools to avoid getter bloat: Instead of exposing 15 narrow single-field tools like get_preferred_airport, get_saved_shirt_size, or get_tax_exempt_cert_id, WebMCP recommends grouping related state into broader domain tools such as get_user_preferences, get_loyalty_benefits, or get_corporate_policy.
  2. Keep tool descriptions under 500 characters: Tool descriptions should stay short and follow a "What + When" formula ("Retrieves the signed-in user's saved personal preferences and default profile settings. Use when personalizing searches or pre-filling actions with the user's usual defaults.").

When a consolidated read tool returns a dozen fields, listing all of them in prose easily exceeds the 500-character limit. If you leave those field names out of description to stay within the budget, and you don't have an outputSchema, the fields returned by each tool are invisible to the model before it calls the tool.

Take a simple example: a user asks "Book a flight from NYC to my home airport". To search for flights, the agent needs a destination airport code. If the page exposes a few broad account tools (get_user_preferences, get_loyalty_benefits, get_corporate_policy) and their short descriptions don't list every field they return, the agent doesn't even know whether the user's home airport is available from a tool on the page, let alone which tool returns it.

Without outputSchema, the agent either gives up and asks the user for information the page already has, or guesses and calls tools one by one hoping one of them contains an airport code. The same problem comes up when chaining tools together: if an action tool requires an input like line_item_id or voucher_code, the agent can't tell ahead of time whether another tool on the page produces that value.

Separating Intent from Structure with outputSchema

outputSchema separates intent from structure:

{
  "name": "get_loyalty_benefits",
  "description": "Retrieves the customer's rewards program tier, point balance...",
  "inputSchema": { "type": "object", "properties": {} },
  "outputSchema": {
    "type": "object",
    "properties": {
      "membership_tier": {
        "type": "string",
        "description": "Current loyalty program tier name"
      },
      "home_airport_iata": {
        "type": "string",
        "description": "Preferred home departure or arrival airport 3-letter IATA code"
      },
      "companion_voucher_code": {
        "type": "string",
        "description": "Active companion airfare voucher code for flight bookings"
      }
    }
  }
}

When the agent sends the page's tools to the model, it forwards outputSchema alongside inputSchema.

Note: Not every model API supports a dedicated output schema field on tool declarations today. When an agent uses an API that only accepts name, description, and inputSchema, the agent is responsible for serializing or summarizing outputSchema into the tool description before sending the request.

To measure the difference, I built an evaluation suite (andreban/outputschema-eval) comparing two identical 17-tool schemas across single-turn lookups, multi-step chains, and multi-turn conversations (3 runs per case):

MetricWithout outputSchemaWith outputSchemaDelta
Case Pass Rate1 / 18 (5.6%)16 / 18 (88.9%)+83.3%
Step Pass Rate2 / 57 (3.5%)26 / 33 (78.8%)+75.3%
Total Tool Calls57 calls33 calls-42.1% tool calls

Instead of guessing and calling multiple read tools to find a field or an ID, the agent can inspect outputSchema up front and pick the right tool on its first try.

Context Window Trade-Off

Whether the agent passes outputSchema through a native API field or inlines it into description, those extra tokens are sent on every turn where the tool is registered. Across the 17 tools in the benchmark, adding full outputSchema definitions roughly tripled the tool definition payload (from ~1,100 tokens to ~3,400 tokens).

Because of that token cost, you don't need an outputSchema on every mutating action tool or narrow getter whose return value is already obvious from its name. It pays off specifically on consolidated read and context tools, where spending ~150 extra schema tokens up front saves 2 to 4 wasted tool call round-trips.

Takeaways

Consolidating WebMCP tools and keeping descriptions under 500 characters are both good practices, but together they hide the full list of fields a read tool returns. Adding outputSchema gives the agent a map of those returned fields before it makes its first call, cutting total tool calls by 42% in our tests and raising pass rates from 5.6% to 88.9%.

The full benchmark, schemas, and evaluation code are on GitHub at andreban/outputschema-eval.

Comments

No comments yet. Be the first to comment!

Leave a comment

Comments are moderated and may take some time to appear.

You may also want to read