Skip to content

Part 1: Why Tokens Matter

← Back to Guide


1.1 What Are Tokens?

Large language models read and write tokens. A token can represent a word, part of a word, or punctuation. Byte-Pair Encoding (BPE) builds these units from recurring byte sequences.

How BPE works

  1. Start with raw bytes (256 possible values)
  2. Find the most frequent byte pairs in the training data
  3. Merge those pairs into new tokens
  4. Repeat until the vocabulary reaches its target size, which varies by tokenizer

Common words can become single tokens; rare or long words often split into pieces.

Examples

Text Tokens Count
hello hello 1
unhappiness un + happiness 2-3
authentication authentication 2-3
I met a huge dog I met a huge dog 5
Sure, I'd be happy to help you with that! (each word/punctuation) ~10

These are illustrative splits, not exact counts for every tokenizer. The last example spends tokens on an opener without answering the question.

In the Claude 3 tokenizer, 8,311 of the 10,000 most common English words tokenize as a single token. Common words are cheap individually, but unnecessary words still add up.

1.2 Why You Should Care

Token usage affects cost, limits, capacity, and speed:

Dimension Impact
Cost API pricing is per-token, and GitHub Copilot usage limits are token-based. In the Anthropic examples, output costs 5× as much per token as input: $1/$5 for Haiku, $3/$15 for Sonnet, and $5/$25 for Opus per MTok. Copilot UBB also separates token buckets; see Model Selection & Pricing before applying vendor rates to Copilot spend
Rate limits Fewer tokens can help when token-based limits are the constraint; request-based limits still apply
Context window Input and output share finite space. Unnecessary input leaves less room for useful context or output
Speed Less input can reduce time-to-first-token; caching and model behavior also matter
Billing As of June 1, 2026, GitHub Copilot bills usage as usage-based billing (UBB). Each interaction is metered in three buckets — input tokens (prompt + context + history), output tokens (the response), and cached tokens (reused context, billed at ~10% of input rate). Cost = (input × input rate) + (cached × cache rate) + (output × output rate). Business and Enterprise seats include pooled AI credits ($30/seat for CB, $70/seat for CE) shared across the org. Code completion stays unlimited

File context, history, instructions, and loaded tool schemas can dwarf your typed prompt. Agent-mode input can reach 5,000–50,000+ tokens. Focus input reductions on what actually loads; ask for only the output you need.

1.3 How GitHub Copilot Uses Tokens

Copilot's input includes:

Token cost anatomy: input tokens include hidden context, output tokens are the visible answer, and stable cached tokens get cheaper.

  • System prompt: Copilot's own instructions (you can't control this)
  • copilot-instructions.md: Your project-level instructions — loaded on every interaction
  • File context: Files you reference with #file, plus files Copilot auto-includes (open tabs, imports, nearby files)
  • Conversation history: Previous messages retained in the request, subject to compaction
  • Your prompt: What you actually type

Copilot also adds repo structure, file contents, and import graphs. A 20-word prompt can therefore consume 2000+ input tokens.

The GitHub Coding Agent reads files, makes decisions, and writes code across multiple steps. A single issue-to-PR session can consume tens of thousands of tokens, so reducing repeated context matters throughout the session.

1.4 Who This Guide Is For

  • GitHub Copilot Chat users — the primary audience. Most techniques apply here
  • GitHub Copilot Inline/Edits users — some techniques apply (limited prompt control)
  • GitHub Coding Agent users — high-value audience. Token savings compound across long sessions

Next: Prompt Compression →