← Articles

How token counting works and why it matters for cost control

By Daniela Rios · 16 July 20260 views
#tokens#cost-optimization#claude#llm#performance

Tokens are the unit of everything in LLM economics

Every interaction with an LLM is measured and billed in tokens. Latency is measured in tokens per second. Context limits are measured in tokens. Costs are calculated per million tokens. Yet most developers building on top of LLMs have only a vague intuition for what a token is, how many their requests consume, or how to reduce that number.

This matters more than it might seem. The difference between a well-optimized prompt and a naive one can easily be a factor of three in token consumption. At scale — tens of thousands of requests per day — that factor of three is the difference between a feature that fits within budget and one that doesn't. Even at smaller scale, understanding tokens helps you predict costs before they surprise you, stay within context window limits, and design prompts that get better results with fewer resources.

This article covers how tokenization actually works, how to measure token counts before sending a request, what drives token consumption in practice, and a set of concrete techniques to reduce token usage without degrading output quality.

What is a token?

A token is a unit of text as the model processes it. It's not a word, not a character, and not a byte — it's somewhere between a character and a word, determined by a vocabulary of common subword units trained on large corpora.

The tokenizer most widely used in current LLMs is Byte Pair Encoding (BPE), originally described in a 2015 paper by Sennrich et al. BPE starts with individual bytes as the vocabulary, then iteratively merges the most frequent adjacent pairs into single tokens until reaching a target vocabulary size (typically 50,000–100,000 tokens). The result is a vocabulary where:

  • Common English words are single tokens: hello, world, function, return
  • Less common words are split: tokenization → token + ization
  • Very rare words split further: cryptocurrency → crypto + currency
  • Non-English text often splits into many small pieces
  • Code has its own patterns: === is often one token, but !== might be two

The practical implication: token count is not predictable from character count alone. It depends on the language, the vocabulary of the specific tokenizer, and the content type (prose vs. code vs. JSON vs. binary data encoded as text).

Rough heuristics for English text

For English prose, a useful approximation is:

  • 1 token ≈ 4 characters
  • 1 token ≈ 0.75 words
  • 1,000 tokens ≈ 750 words

These are averages that break down quickly:

  • Technical writing with long compound words tokenizes worse (more tokens per character)
  • Very common short words tokenize efficiently
  • Code is roughly similar to technical prose but punctuation-heavy code can be denser

For code specifically:

  • Python code: roughly 1 token per 3.5 characters
  • JavaScript/TypeScript: roughly 1 token per 3.5–4 characters
  • JSON: roughly 1 token per 3 characters (keys and syntax overhead)
  • Base64-encoded data: roughly 1 token per 3 characters (performs poorly — avoid sending binary as base64 when possible)

Measuring tokens with the Anthropic API

Don't guess at token counts in production — measure them. The Anthropic API provides a countTokens endpoint that tells you exactly how many tokens your request will consume before you send it.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

async function countRequestTokens(
  systemPrompt: string,
  userMessage: string,
  model = "claude-sonnet-4-5"
): Promise<{
  systemTokens: number;
  messageTokens: number;
  totalInput: number;
}> {
  const response = await client.messages.countTokens({
    model,
    system: systemPrompt,
    messages: [{ role: "user", content: userMessage }],
  });

  // The API returns total input tokens
  // We can estimate the split by counting individually
  const systemOnly = await client.messages.countTokens({
    model,
    system: systemPrompt,
    messages: [{ role: "user", content: "." }],
  });

  return {
    systemTokens: systemOnly.input_tokens,
    messageTokens: response.input_tokens - systemOnly.input_tokens,
    totalInput: response.input_tokens,
  };
}

// Example: check if a document fits before sending
async function fitsInContext(
  document: string,
  model = "claude-sonnet-4-5",
  contextLimit = 200_000
): Promise<{ fits: boolean; inputTokens: number; remaining: number }> {
  const { input_tokens } = await client.messages.countTokens({
    model,
    messages: [{ role: "user", content: document }],
  });

  const safeLimit = contextLimit * 0.9; // 10% safety margin
  return {
    fits: input_tokens < safeLimit,
    inputTokens: input_tokens,
    remaining: safeLimit - input_tokens,
  };
}

Using token counts to enforce budgets pre-flight

interface TokenBudget {
  maxInput: number;
  maxOutput: number;
  warningThreshold: number; // fraction of max, e.g. 0.8
}

const BUDGETS: Record<string, TokenBudget> = {
  "claude-haiku-4-5": { maxInput: 200_000, maxOutput: 8_192, warningThreshold: 0.8 },
  "claude-sonnet-4-5": { maxInput: 200_000, maxOutput: 8_192, warningThreshold: 0.8 },
  "claude-opus-4-5": { maxInput: 200_000, maxOutput: 8_192, warningThreshold: 0.8 },
};

async function callWithBudgetCheck(
  messages: Anthropic.MessageParam[],
  options: {
    model: string;
    maxTokens: number;
    system?: string;
  }
): Promise<Anthropic.Message> {
  const budget = BUDGETS[options.model];
  if (!budget) throw new Error(`Unknown model: ${options.model}`);

  const { input_tokens } = await client.messages.countTokens({
    model: options.model,
    system: options.system,
    messages,
  });

  const totalEstimated = input_tokens + options.maxTokens;

  if (input_tokens > budget.maxInput) {
    throw new Error(
      `Request exceeds context limit: ${input_tokens} input tokens > ${budget.maxInput}`
    );
  }

  if (input_tokens > budget.maxInput * budget.warningThreshold) {
    console.warn(
      `Warning: using ${Math.round((input_tokens / budget.maxInput) * 100)}% of context window`
    );
  }

  return client.messages.create({
    ...options,
    messages,
  });
}

What drives token consumption in practice

Understanding the major token sinks in a typical LLM application helps you find where to optimize.

System prompts

System prompts are paid on every request. A 500-token system prompt that runs 10,000 times costs 5 million tokens — more than many user conversations. Keep system prompts tight.

// Verbose: 284 tokens
const verboseSystem = `You are an AI assistant that helps users with customer support questions. 
You should always be polite and professional in your responses. When answering questions, 
make sure to be thorough and provide complete information. If you don't know the answer to 
something, it's okay to say that you don't know. Always try to be helpful and provide the 
most accurate information possible. You should never make up information or provide false 
details. Be concise but complete in your answers.`;

// Concise: 47 tokens — same semantics
const conciseSystem = `Customer support assistant. Be accurate and helpful. 
Say "I don't know" when uncertain. Never fabricate information.`;

The verbose version isn't more effective. Hedges like "always try to" and "make sure to" don't change model behavior — the model already behaves this way. Instructions that restate the model's default behavior waste tokens.

Conversation history

In multi-turn applications, the entire conversation history is resent with every request. As conversations grow, costs compound.

interface ConversationManager {
  messages: Anthropic.MessageParam[];
  totalInputTokens: number;
}

async function addMessageWithTrimming(
  manager: ConversationManager,
  newMessage: Anthropic.MessageParam,
  model: string,
  maxHistoryTokens = 50_000
): Promise<ConversationManager> {
  const allMessages = [...manager.messages, newMessage];

  // Count tokens for the full history
  const { input_tokens } = await client.messages.countTokens({
    model,
    messages: allMessages,
  });

  if (input_tokens <= maxHistoryTokens) {
    return { messages: allMessages, totalInputTokens: input_tokens };
  }

  // Trim oldest messages (preserve at least the first and last 2)
  let trimmed = allMessages;
  while (trimmed.length > 4) {
    trimmed = [trimmed[0], ...trimmed.slice(2)]; // remove second message
    const { input_tokens: trimmedTokens } = await client.messages.countTokens({
      model,
      messages: trimmed,
    });
    if (trimmedTokens <= maxHistoryTokens) {
      return { messages: trimmed, totalInputTokens: trimmedTokens };
    }
  }

  return { messages: trimmed, totalInputTokens: input_tokens };
}

A smarter approach is summarization: when the conversation exceeds a threshold, compress the older portion into a summary that captures the key information in fewer tokens.

async function summarizeHistory(
  messages: Anthropic.MessageParam[],
  keepRecent = 4
): Promise<Anthropic.MessageParam[]> {
  const toSummarize = messages.slice(0, -keepRecent);
  const toKeep = messages.slice(-keepRecent);

  if (toSummarize.length === 0) return messages;

  const summaryResponse = await client.messages.create({
    model: "claude-haiku-4-5", // use cheapest model for summarization
    max_tokens: 512,
    system:
      "Summarize the following conversation history concisely. " +
      "Preserve key facts, decisions, and context needed to continue the conversation.",
    messages: [
      {
        role: "user",
        content: toSummarize
          .map((m) => `${m.role}: ${typeof m.content === "string" ? m.content : "[content]"}`)
          .join("\n"),
      },
    ],
  });

  const summary =
    summaryResponse.content[0].type === "text"
      ? summaryResponse.content[0].text
      : "";

  return [
    { role: "user", content: `[Previous conversation summary: ${summary}]` },
    { role: "assistant", content: "Understood, I have the context from our previous discussion." },
    ...toKeep,
  ];
}

Document context in RAG applications

Retrieval-augmented generation that includes full document chunks is a major token sink. Typical pitfalls:

  • Including chunks that are irrelevant because retrieval isn't precise enough
  • Using fixed chunk sizes that include a lot of padding text
  • Including the same information in multiple chunks (document overlap)
  • Not truncating chunks that exceed what's actually useful
interface RetrievedChunk {
  content: string;
  relevanceScore: number;
  source: string;
}

async function buildRagContext(
  chunks: RetrievedChunk[],
  maxContextTokens: number,
  model: string
): Promise<string> {
  // Sort by relevance, most relevant first
  const sorted = [...chunks].sort((a, b) => b.relevanceScore - a.relevanceScore);

  const included: string[] = [];
  let totalTokens = 0;

  for (const chunk of sorted) {
    const chunkText = `[Source: ${chunk.source}]\n${chunk.content}`;

    const { input_tokens } = await client.messages.countTokens({
      model,
      messages: [{ role: "user", content: chunkText }],
    });

    if (totalTokens + input_tokens > maxContextTokens) {
      // Try truncating this chunk to fit remaining budget
      const remainingBudget = maxContextTokens - totalTokens;
      if (remainingBudget > 200) {
        // Truncate to approximately fit (rough estimation: 4 chars/token)
        const truncated = chunk.content.slice(0, remainingBudget * 3.5);
        included.push(`[Source: ${chunk.source}]\n${truncated}...[truncated]`);
      }
      break;
    }

    included.push(chunkText);
    totalTokens += input_tokens;
  }

  return included.join("\n\n---\n\n");
}

Prompt caching: paying for tokens once

The Anthropic API supports prompt caching, which lets you pay the input token cost once and then pay a much lower cache read cost on subsequent requests with the same prefix. This is transformative for applications where a large system prompt or document is included in every request.

async function callWithPromptCache(
  cachedSystemContent: string,  // large, stable content
  userMessage: string
): Promise<string> {
  const response = await client.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 1024,
    system: [
      {
        type: "text",
        text: cachedSystemContent,
        // Mark as cacheable — must be >= 1024 tokens to be cached
        cache_control: { type: "ephemeral" },
      },
    ],
    messages: [{ role: "user", content: userMessage }],
  });

  // Check cache performance in usage
  const usage = response.usage as Anthropic.Usage & {
    cache_creation_input_tokens?: number;
    cache_read_input_tokens?: number;
  };

  if (usage.cache_read_input_tokens) {
    console.log(`Cache hit: ${usage.cache_read_input_tokens} tokens read from cache`);
  } else if (usage.cache_creation_input_tokens) {
    console.log(`Cache miss: ${usage.cache_creation_input_tokens} tokens cached`);
  }

  return response.content[0].type === "text" ? response.content[0].text : "";
}

Prompt cache pricing (as of mid-2026):

  • Cache write: 25% more expensive than standard input
  • Cache read: 90% cheaper than standard input
  • Cache lifetime: 5 minutes (refreshed each time it's accessed)

The break-even for caching is 1.3 requests with the same prefix (the write cost is recovered after 1.3 reads). For any system prompt used more than twice in a 5-minute window, caching pays for itself.

// Calculate expected savings from caching
function cachingSavingsEstimate(
  promptTokens: number,
  requestsPerMinute: number,
  inputPricePerMillion: number // standard input price
): { cacheCostPerHour: number; noCacheCostPerHour: number; savingsPercent: number } {
  const requestsPerHour = requestsPerMinute * 60;
  const tokensPerHour = promptTokens * requestsPerHour;

  // Without caching: pay full price every time
  const noCacheCost = (tokensPerHour / 1_000_000) * inputPricePerMillion;

  // With caching: first request per 5-min window pays write price (1.25x),
  // subsequent requests pay read price (0.1x)
  const windowsPerHour = 12; // 60/5
  const writeCostPerHour =
    (promptTokens * windowsPerHour * inputPricePerMillion * 1.25) / 1_000_000;
  const readRequestsPerHour = requestsPerHour - windowsPerHour;
  const readCostPerHour =
    (promptTokens * readRequestsPerHour * inputPricePerMillion * 0.1) / 1_000_000;
  const cacheCost = writeCostPerHour + readCostPerHour;

  return {
    cacheCostPerHour: cacheCost,
    noCacheCostPerHour: noCacheCost,
    savingsPercent: Math.round((1 - cacheCost / noCacheCost) * 100),
  };
}

// Example: 5000-token system prompt, 20 requests/minute, $3/M tokens
const estimate = cachingSavingsEstimate(5000, 20, 3);
console.log(`Cache saves ${estimate.savingsPercent}% per hour`); // ~83%

Output token control

Input tokens get most of the attention, but output tokens are also significant — and you can control them.

Set realistic max_tokens

Always set max_tokens to the minimum you actually need. The API doesn't charge for unused token budget, but setting it unrealistically high signals to the model that verbose responses are expected.

// For classification tasks — brief outputs needed
const classifyResponse = await client.messages.create({
  model: "claude-haiku-4-5",
  max_tokens: 50,  // classification shouldn't need more
  messages: [{ role: "user", content: `Classify this review as positive/negative/neutral: ${reviewText}` }],
});

// For summarization — medium outputs
const summaryResponse = await client.messages.create({
  model: "claude-sonnet-4-5",
  max_tokens: 300,
  messages: [{ role: "user", content: `Summarize in 2-3 sentences: ${documentText}` }],
});

Instruct for brevity in the prompt

The model responds to explicit length constraints in the prompt:

// Without length instruction — model might output 400 tokens
const prompt1 = "Explain what a JWT token is.";

// With length instruction — output constrained appropriately
const prompt2 = "Explain what a JWT token is. Answer in 2-3 sentences, no examples needed.";

Use stop sequences to end output early

Stop sequences tell the model to stop generating when it reaches a specific string. This is useful when you only need a specific part of the output.

const response = await client.messages.create({
  model: "claude-haiku-4-5",
  max_tokens: 200,
  stop_sequences: ["\n\n", "---"],  // stop at first blank line or separator
  messages: [{
    role: "user",
    content: "In one sentence, what is the main purpose of dependency injection?",
  }],
});

Token cost tracking in production

Log actual token usage on every API call to track costs in real time:

interface TokenUsageRecord {
  timestamp: string;
  model: string;
  feature: string;
  inputTokens: number;
  outputTokens: number;
  cacheReadTokens: number;
  cacheWriteTokens: number;
  estimatedCostUsd: number;
}

const MODEL_PRICING: Record<string, { input: number; output: number; cacheRead: number; cacheWrite: number }> = {
  "claude-haiku-4-5":   { input: 0.80,  output: 4.00,  cacheRead: 0.08,  cacheWrite: 1.00 },
  "claude-sonnet-4-5":  { input: 3.00,  output: 15.00, cacheRead: 0.30,  cacheWrite: 3.75 },
  "claude-opus-4-5":    { input: 15.00, output: 75.00, cacheRead: 1.50,  cacheWrite: 18.75 },
};

function recordUsage(
  response: Anthropic.Message,
  feature: string
): TokenUsageRecord {
  const pricing = MODEL_PRICING[response.model] ?? MODEL_PRICING["claude-sonnet-4-5"];
  const usage = response.usage as Anthropic.Usage & {
    cache_read_input_tokens?: number;
    cache_creation_input_tokens?: number;
  };

  const inputTokens = usage.input_tokens;
  const outputTokens = usage.output_tokens;
  const cacheRead = usage.cache_read_input_tokens ?? 0;
  const cacheWrite = usage.cache_creation_input_tokens ?? 0;

  const costUsd =
    ((inputTokens - cacheRead) * pricing.input +
      cacheRead * pricing.cacheRead +
      cacheWrite * pricing.cacheWrite +
      outputTokens * pricing.output) /
    1_000_000;

  const record: TokenUsageRecord = {
    timestamp: new Date().toISOString(),
    model: response.model,
    feature,
    inputTokens,
    outputTokens,
    cacheReadTokens: cacheRead,
    cacheWriteTokens: cacheWrite,
    estimatedCostUsd: costUsd,
  };

  // Log to your metrics system
  console.log("token_usage", record);

  return record;
}

With per-request cost records, you can build dashboards showing cost by feature, model, and time period. This makes it straightforward to answer "which feature is responsible for 40% of our AI spend?" and target optimizations where they matter most.

Token counting and cost control aren't glamorous, but they're what separates AI features that remain economical at scale from ones that surprise you with a $30,000 monthly bill. The investment in instrumentation, pre-flight token checks, prompt caching, and output constraints pays off as soon as your feature starts receiving real traffic.

Comments

No comments yet. Be the first!

Sign in to leave a comment.