Part 1: Why Tokens Matter
1.1 What Are Tokens?
Large language models read and write tokens. A token can represent a word, part of a word, or punctuation. Byte-Pair Encoding (BPE) builds these units from recurring byte sequences.
How BPE works
- Start with raw bytes (256 possible values)
- Find the most frequent byte pairs in the training data
- Merge those pairs into new tokens
- Repeat until the vocabulary reaches its target size, which varies by tokenizer
Common words can become single tokens; rare or long words often split into pieces.
Examples
| Text | Tokens | Count |
|---|---|---|
hello |
hello |
1 |
unhappiness |
un + happiness |
2-3 |
authentication |
authentication |
2-3 |
I met a huge dog |
I met a huge dog |
5 |
Sure, I'd be happy to help you with that! |
(each word/punctuation) | ~10 |
These are illustrative splits, not exact counts for every tokenizer. The last example spends tokens on an opener without answering the question.
In the Claude 3 tokenizer, 8,311 of the 10,000 most common English words tokenize as a single token. Common words are cheap individually, but unnecessary words still add up.
1.2 Why You Should Care
Token usage affects cost, limits, capacity, and speed:
| Dimension | Impact |
|---|---|
| Cost | API pricing is per-token, and GitHub Copilot usage limits are token-based. In the Anthropic examples, output costs 5× as much per token as input: $1/$5 for Haiku, $3/$15 for Sonnet, and $5/$25 for Opus per MTok. Copilot UBB also separates token buckets; see Model Selection & Pricing before applying vendor rates to Copilot spend |
| Rate limits | Fewer tokens can help when token-based limits are the constraint; request-based limits still apply |
| Context window | Input and output share finite space. Unnecessary input leaves less room for useful context or output |
| Speed | Less input can reduce time-to-first-token; caching and model behavior also matter |
| Billing | As of June 1, 2026, GitHub Copilot bills usage as usage-based billing (UBB). Each interaction is metered in three buckets — input tokens (prompt + context + history), output tokens (the response), and cached tokens (reused context, billed at ~10% of input rate). Cost = (input × input rate) + (cached × cache rate) + (output × output rate). Business and Enterprise seats include pooled AI credits ($30/seat for CB, $70/seat for CE) shared across the org. Code completion stays unlimited |
File context, history, instructions, and loaded tool schemas can dwarf your typed prompt. Agent-mode input can reach 5,000–50,000+ tokens. Focus input reductions on what actually loads; ask for only the output you need.
1.3 How GitHub Copilot Uses Tokens
Copilot's input includes:
- System prompt: Copilot's own instructions (you can't control this)
copilot-instructions.md: Your project-level instructions — loaded on every interaction- File context: Files you reference with
#file, plus files Copilot auto-includes (open tabs, imports, nearby files) - Conversation history: Previous messages retained in the request, subject to compaction
- Your prompt: What you actually type
Copilot also adds repo structure, file contents, and import graphs. A 20-word prompt can therefore consume 2000+ input tokens.
The GitHub Coding Agent reads files, makes decisions, and writes code across multiple steps. A single issue-to-PR session can consume tens of thousands of tokens, so reducing repeated context matters throughout the session.
1.4 Who This Guide Is For
- GitHub Copilot Chat users — the primary audience. Most techniques apply here
- GitHub Copilot Inline/Edits users — some techniques apply (limited prompt control)
- GitHub Coding Agent users — high-value audience. Token savings compound across long sessions
Next: Prompt Compression →