Anthropic Claude API: tool use, vision, and prompt caching in practice

What the Claude API can do beyond basic chat

Most teams that integrate Claude start with a simple messages endpoint call: system prompt, user message, text response. That workflow covers a surprising amount of ground, but it leaves three capabilities almost entirely untouched — capabilities that change what kinds of applications are possible.

Tool use (also called function calling) lets Claude invoke functions you define and coordinate multi-step actions without you writing a state machine. Vision lets Claude reason over images, screenshots, diagrams, and documents as first-class inputs alongside text. Prompt caching lets you reuse expensive prefix computation so repeated calls against the same large context cost a fraction of their normal price in both latency and tokens.

This article is a working guide to all three. Every code example uses the official @anthropic-ai/sdk TypeScript package, but the concepts translate directly to the Python SDK and raw HTTP.

Tool use: giving Claude structured function calls

Tool use turns Claude from a text generator into a decision engine. You declare a set of tools — each with a name, description, and JSON Schema input spec — and Claude decides which tool to call, with which arguments, in response to a user message. Your code executes the tool and feeds the result back. Claude can chain multiple tool calls across multiple turns until it has enough information to produce a final answer.

Defining tools

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const tools: Anthropic.Tool[] = [
  {
    name: "get_order_status",
    description:
      "Retrieve the current status and estimated delivery date for an order. " +
      "Use this when the customer asks about their order.",
    input_schema: {
      type: "object" as const,
      properties: {
        order_id: {
          type: "string",
          description: "The order ID, e.g. ORD-12345",
        },
        customer_email: {
          type: "string",
          description: "Customer email address to verify ownership",
        },
      },
      required: ["order_id", "customer_email"],
    },
  },
  {
    name: "search_products",
    description:
      "Search the product catalogue by keyword. Returns a list of matching products with prices.",
    input_schema: {
      type: "object" as const,
      properties: {
        query: { type: "string", description: "Search keywords" },
        max_results: {
          type: "number",
          description: "Maximum number of results to return (default 5)",
        },
      },
      required: ["query"],
    },
  },
];

Each tool's description field is the most important part — Claude uses it to decide when and whether to call the tool. Be specific about what the tool does and what data it expects. Vague descriptions lead to missed calls or incorrect argument construction.

Running an agentic loop

The core pattern for tool use is a loop: send a message, check if Claude wants to call a tool, execute the tool, append the result, repeat until Claude returns a final text response.

type Message = Anthropic.MessageParam;

async function runAgentLoop(userMessage: string): Promise<string> {
  const messages: Message[] = [{ role: "user", content: userMessage }];

  while (true) {
    const response = await client.messages.create({
      model: "claude-opus-4-5",
      max_tokens: 4096,
      tools,
      messages,
    });

    // If Claude finished with a text response, return it
    if (response.stop_reason === "end_turn") {
      const textBlock = response.content.find((b) => b.type === "text");
      return textBlock?.type === "text" ? textBlock.text : "";
    }

    // Claude wants to call one or more tools
    if (response.stop_reason === "tool_use") {
      // Append Claude's response (which contains tool_use blocks)
      messages.push({ role: "assistant", content: response.content });

      // Build tool results
      const toolResults: Anthropic.ToolResultBlockParam[] = [];
      for (const block of response.content) {
        if (block.type !== "tool_use") continue;

        const result = await executeToolCall(block.name, block.input as Record<string, unknown>);
        toolResults.push({
          type: "tool_result",
          tool_use_id: block.id,
          content: JSON.stringify(result),
        });
      }

      // Append all tool results in a single user turn
      messages.push({ role: "user", content: toolResults });
    }
  }
}

async function executeToolCall(
  name: string,
  input: Record<string, unknown>
): Promise<unknown> {
  switch (name) {
    case "get_order_status":
      return fetchOrderStatus(input.order_id as string, input.customer_email as string);
    case "search_products":
      return searchCatalogue(input.query as string, (input.max_results as number) ?? 5);
    default:
      throw new Error(`Unknown tool: ${name}`);
  }
}

Several things matter here that aren't obvious from the API docs alone.

First, Claude can request multiple tool calls in a single turn — the content array may contain multiple tool_use blocks. Always iterate over all of them, execute each, and return all results together in one user message. Sending them separately breaks the conversation format.

Second, the tool result content is a string, but that string can be JSON. Serialising structured data as JSON inside the content field is the correct approach.

Third, always handle the end_turn stop reason explicitly. If you forget and Claude returns text with end_turn, a naive loop will spin until it hits a max-iteration guard.

Forcing tool use and controlling tool choice

By default, Claude decides whether to call a tool or respond directly. You can override this:

// Force Claude to call a specific tool
const response = await client.messages.create({
  model: "claude-opus-4-5",
  max_tokens: 1024,
  tools,
  tool_choice: { type: "tool", name: "search_products" },
  messages: [{ role: "user", content: "Show me some running shoes" }],
});

// Allow tool use but don't require it
const response2 = await client.messages.create({
  model: "claude-opus-4-5",
  max_tokens: 1024,
  tools,
  tool_choice: { type: "auto" }, // default
  messages,
});

// Disable all tool calls for this request
const response3 = await client.messages.create({
  model: "claude-opus-4-5",
  max_tokens: 1024,
  tools,
  tool_choice: { type: "none" },
  messages,
});

tool_choice: { type: "tool", name: "..." } is useful when you know exactly what extraction you need and don't want Claude to decide. A common pattern is forcing a structured extraction tool on the first turn to parse an unstructured user request into typed fields, then proceeding with a normal loop from there.

Vision: images and documents as first-class inputs

Claude's vision capability accepts images either as base64-encoded data or as URLs. It can read screenshots, diagrams, charts, scanned documents, and photographs, and reason over them in combination with text.

Sending an image as base64

import fs from "fs";

async function analyseImage(imagePath: string, question: string): Promise<string> {
  const imageData = fs.readFileSync(imagePath);
  const base64 = imageData.toString("base64");
  const mediaType = "image/png"; // or image/jpeg, image/gif, image/webp

  const response = await client.messages.create({
    model: "claude-opus-4-5",
    max_tokens: 1024,
    messages: [
      {
        role: "user",
        content: [
          {
            type: "image",
            source: {
              type: "base64",
              media_type: mediaType,
              data: base64,
            },
          },
          {
            type: "text",
            text: question,
          },
        ],
      },
    ],
  });

  const textBlock = response.content.find((b) => b.type === "text");
  return textBlock?.type === "text" ? textBlock.text : "";
}

// Usage
const summary = await analyseImage(
  "./screenshot.png",
  "List all the error messages visible in this screenshot and suggest fixes."
);

Sending an image by URL

When your images are publicly accessible, URLs are simpler and avoid the base64 payload size:

const response = await client.messages.create({
  model: "claude-opus-4-5",
  max_tokens: 1024,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "image",
          source: {
            type: "url",
            url: "https://example.com/chart.png",
          },
        },
        {
          type: "text",
          text: "Describe the trend shown in this chart and identify any anomalies.",
        },
      ],
    },
  ],
});

URL-sourced images are fetched by Anthropic's servers at request time. The URL must be publicly accessible without authentication.

Combining vision with tool use

Vision and tool use compose naturally. A practical pattern is an image analyser that can also look things up:

const imageAnalysisTools: Anthropic.Tool[] = [
  {
    name: "lookup_product",
    description: "Look up product details by SKU or barcode number",
    input_schema: {
      type: "object" as const,
      properties: {
        identifier: { type: "string", description: "SKU or barcode" },
      },
      required: ["identifier"],
    },
  },
];

async function analyseProductImage(imageBase64: string): Promise<string> {
  const messages: Message[] = [
    {
      role: "user",
      content: [
        {
          type: "image",
          source: { type: "base64", media_type: "image/jpeg", data: imageBase64 },
        },
        {
          type: "text",
          text: "Identify any product barcodes or SKUs in this image and look up their details.",
        },
      ],
    },
  ];

  // reuse the same agent loop from before
  return runAgentLoopWithTools(messages, imageAnalysisTools);
}

Document inputs with the Files API

For PDFs and longer documents, the Files API lets you upload once and reference by ID across multiple requests — avoiding repeated base64 payloads:

async function uploadDocument(pdfPath: string): Promise<string> {
  const file = fs.createReadStream(pdfPath);
  const uploadedFile = await client.beta.files.upload({
    file: new File([fs.readFileSync(pdfPath)], "document.pdf", { type: "application/pdf" }),
  });
  return uploadedFile.id;
}

async function queryDocument(fileId: string, question: string): Promise<string> {
  const response = await client.beta.messages.create({
    model: "claude-opus-4-5",
    max_tokens: 2048,
    messages: [
      {
        role: "user",
        content: [
          {
            type: "document",
            source: { type: "file", file_id: fileId },
          } as Anthropic.Beta.BetaRequestDocumentBlock,
          { type: "text", text: question },
        ],
      },
    ],
    betas: ["files-api-2025-04-14"],
  });

  const textBlock = response.content.find((b) => b.type === "text");
  return textBlock?.type === "text" ? textBlock.text : "";
}

Upload the file once during application startup or when the document changes, then reuse the file_id for all queries against that document.

Prompt caching: cut latency and cost on repeated context

Prompt caching is the most impactful optimisation for applications that send the same large prefix on every request — long system prompts, reference documents, few-shot examples, tool definitions.

When you mark a prefix with cache_control: { type: "ephemeral" }, Anthropic caches the KV state after processing that prefix. Subsequent requests that share the same cached prefix skip the prefill computation for those tokens. Cache hits are billed at 10% of the normal input token price and typically reduce time-to-first-token by 40–80% depending on prefix length.

Caching a large system prompt

const LARGE_SYSTEM_PROMPT = `
You are a senior software engineer assistant specialised in TypeScript, React, and Node.js.
You follow these coding standards:
[... imagine 2000 words of detailed standards, patterns, and examples ...]
`;

async function askCodeQuestion(question: string): Promise<string> {
  const response = await client.messages.create({
    model: "claude-opus-4-5",
    max_tokens: 2048,
    system: [
      {
        type: "text",
        text: LARGE_SYSTEM_PROMPT,
        cache_control: { type: "ephemeral" },
      },
    ],
    messages: [{ role: "user", content: question }],
  });

  const textBlock = response.content.find((b) => b.type === "text");
  return textBlock?.type === "text" ? textBlock.text : "";
}

The first call creates the cache. Subsequent calls within the cache TTL (5 minutes, extendable to 1 hour with extended cache type) hit the cache. The response includes cache_creation_input_tokens and cache_read_input_tokens in usage so you can verify cache behaviour.

Checking cache metrics in the response

const response = await client.messages.create({ /* ... */ });

const usage = response.usage;
console.log({
  inputTokens: usage.input_tokens,
  cacheCreationTokens: usage.cache_creation_input_tokens,
  cacheReadTokens: usage.cache_read_input_tokens,
  outputTokens: usage.output_tokens,
});
// First call: cacheCreationTokens > 0, cacheReadTokens = 0
// Subsequent calls within TTL: cacheCreationTokens = 0, cacheReadTokens > 0

Caching reference documents alongside a system prompt

For RAG-style applications, you can cache a set of retrieved documents so the model processes them once and subsequent follow-up questions reuse the cached state:

async function createCachedDocumentSession(
  systemPrompt: string,
  referenceDocuments: string[]
): Promise<(question: string) => Promise<string>> {
  const docsText = referenceDocuments
    .map((doc, i) => `<document index="${i + 1}">\n${doc}\n</document>`)
    .join("\n\n");

  const baseSystem: Anthropic.TextBlockParam[] = [
    {
      type: "text",
      text: systemPrompt,
      cache_control: { type: "ephemeral" },
    },
    {
      type: "text",
      text: `Reference documents:\n\n${docsText}`,
      cache_control: { type: "ephemeral" },
    },
  ];

  // Return a closure that reuses the cached prefix
  return async (question: string) => {
    const response = await client.messages.create({
      model: "claude-opus-4-5",
      max_tokens: 2048,
      system: baseSystem,
      messages: [{ role: "user", content: question }],
    });

    const textBlock = response.content.find((b) => b.type === "text");
    return textBlock?.type === "text" ? textBlock.text : "";
  };
}

// Usage
const ask = await createCachedDocumentSession(
  "You are a helpful assistant. Answer questions using only the provided documents.",
  [doc1Text, doc2Text, doc3Text]
);

const answer1 = await ask("What is the return policy?");
const answer2 = await ask("What payment methods are accepted?");
// Both questions reuse the cached documents

Cache placement rules

Only a cache boundary at the end of the system block or at the end of a user turn will be effective. You cannot cache mid-message. The recommended pattern is to put your largest stable content earliest, mark it with cache_control, and put dynamic or question-specific content after the cache boundary.

You can have up to four cache breakpoints per request. Using more than one breakpoint makes sense when you have tiered content — a permanent system prompt, a per-session document set, and per-conversation history, each cached at different TTLs.

Combining all three features in a real application

Here is a pattern that uses tool use, vision, and caching together — a document analysis agent that can visually inspect uploaded files, answer questions about them, and look up external data:

const analysisTools: Anthropic.Tool[] = [
  {
    name: "fetch_regulation",
    description: "Fetch the text of a specific regulation by code",
    input_schema: {
      type: "object" as const,
      properties: {
        code: { type: "string" },
      },
      required: ["code"],
    },
  },
];

async function analyseComplianceDocument(
  documentBase64: string,
  companyContext: string
): Promise<string> {
  const cachedSystem: Anthropic.TextBlockParam[] = [
    {
      type: "text",
      text:
        "You are a compliance analyst. " +
        "Identify potential regulatory issues in documents, " +
        "and use the fetch_regulation tool to retrieve full regulation text when needed.",
      cache_control: { type: "ephemeral" },
    },
    {
      type: "text",
      text: `Company context:\n${companyContext}`,
      cache_control: { type: "ephemeral" },
    },
  ];

  const messages: Message[] = [
    {
      role: "user",
      content: [
        {
          type: "image",
          source: {
            type: "base64",
            media_type: "image/png",
            data: documentBase64,
          },
        },
        {
          type: "text",
          text: "Review this document for compliance issues. Fetch any relevant regulations you reference.",
        },
      ],
    },
  ];

  while (true) {
    const response = await client.messages.create({
      model: "claude-opus-4-5",
      max_tokens: 4096,
      system: cachedSystem,
      tools: analysisTools,
      messages,
    });

    if (response.stop_reason === "end_turn") {
      const textBlock = response.content.find((b) => b.type === "text");
      return textBlock?.type === "text" ? textBlock.text : "";
    }

    if (response.stop_reason === "tool_use") {
      messages.push({ role: "assistant", content: response.content });
      const results: Anthropic.ToolResultBlockParam[] = [];
      for (const block of response.content) {
        if (block.type !== "tool_use") continue;
        const result = await fetchRegulation((block.input as { code: string }).code);
        results.push({
          type: "tool_result",
          tool_use_id: block.id,
          content: result,
        });
      }
      messages.push({ role: "user", content: results });
    }
  }
}

Error handling and rate limits

The Claude API returns structured errors. The SDK throws typed exceptions you should handle explicitly:

import { APIError, RateLimitError, APIConnectionError } from "@anthropic-ai/sdk";

async function safeCall(messages: Message[]): Promise<string> {
  const MAX_RETRIES = 3;
  let attempt = 0;

  while (attempt < MAX_RETRIES) {
    try {
      const response = await client.messages.create({
        model: "claude-opus-4-5",
        max_tokens: 1024,
        messages,
      });
      const textBlock = response.content.find((b) => b.type === "text");
      return textBlock?.type === "text" ? textBlock.text : "";
    } catch (err) {
      if (err instanceof RateLimitError) {
        const retryAfter = parseInt(err.headers?.["retry-after"] ?? "5", 10);
        await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000));
        attempt++;
      } else if (err instanceof APIConnectionError) {
        await new Promise((resolve) => setTimeout(resolve, 2 ** attempt * 1000));
        attempt++;
      } else if (err instanceof APIError && err.status === 529) {
        // Overloaded — back off
        await new Promise((resolve) => setTimeout(resolve, 10_000));
        attempt++;
      } else {
        throw err;
      }
    }
  }
  throw new Error("Max retries exceeded");
}

Rate limit headers include anthropic-ratelimit-requests-remaining and anthropic-ratelimit-tokens-remaining, letting you implement proactive throttling before hitting the limit.

Practical guidance on model selection

Claude Opus 4.5 has the highest reasoning quality and the best tool-use decision-making but costs more per token. Claude Sonnet 4.5 hits a strong quality/cost balance and handles most production tool-use and vision workloads well. Claude Haiku 3.5 is the fastest and cheapest option, appropriate for high-volume classification or extraction tasks where you control the output format tightly.

For prompt-caching ROI, the cache prefix must be at least 1024 tokens for the cache write to happen. If your system prompt is shorter than that, caching adds overhead without benefit. For most production applications with detailed system prompts, few-shot examples, and tool definitions, the 1024-token threshold is crossed easily.

The combination of tool use, vision, and prompt caching covers the majority of production AI application patterns — structured data extraction, document analysis, agentic automation, and interactive assistants — all within a single, well-documented API surface.