# LLM Fundamentals Exercises

## Exercise 1: Token Budget Planning

**Task:** You have a 4K context window. Your system prompt is 500 tokens. You want to include 3 few-shot examples (each ~150 tokens) and reserve 1K tokens for the response. How many tokens are left for the user's actual query? Express as a formula: `query_max = context - system - examples - response`.

**Validation:**
- [ ] Correct calculation: 4000 - 500 - 450 - 1000 = 2050 tokens
- [ ] Can explain why each component consumes context

**Hints:**
1. Sum all fixed parts: system + examples + response reserve
2. Remaining = context - fixed
3. User query must fit in remaining

---

## Exercise 2: Compare Tokenization Across Inputs

**Task:** Tokenize these with the same tool: (a) "The quick brown fox", (b) "def hello(): return 42", (c) "東京" (Tokyo in Japanese). Report tokens per character for each. Why do the ratios differ?

**Validation:**
- [ ] Reports token counts for all three
- [ ] Notes that code and non-Latin may use more tokens per character
- [ ] Can explain impact on cost/length for different input types

**Hints:**
1. Use OpenAI tokenizer or tiktoken
2. Compare tokens / characters
3. Code has many symbols; CJK often 1–2 chars per token

---

## Exercise 3: Design a Temperature Strategy

**Task:** For each use case, choose temperature (0, 0.3, 0.7, 1.0) and write one sentence justifying it:
1. Extracting JSON from a paragraph
2. Brainstorming blog title ideas
3. Answering "What is 2+2?"
4. Writing a product description in a specific brand voice

**Validation:**
- [ ] Extraction: 0 or 0.3 (deterministic)
- [ ] Brainstorm: 0.7–1.0 (creative)
- [ ] Math: 0 (deterministic)
- [ ] Brand voice: 0.3–0.5 (consistent but some variety)

**Hints:**
1. Factual/deterministic → low
2. Creative/varied → high
3. Brand voice: need consistency but not rigidity

---

## Exercise 4: Explain Hallucination in Your Own Words

**Task:** Write a 2–3 sentence explanation of why LLMs hallucinate, suitable for a non-technical colleague. Use the concepts: pattern completion, no verification, confidence. Avoid jargon like "logits" or "softmax".

**Validation:**
- [ ] Mentions that the model completes patterns
- [ ] Notes that it doesn't verify against knowledge
- [ ] Explains that confidence ≠ correctness

**Hints:**
1. "The model predicts what comes next, not what is true"
2. "It was trained on text patterns, not a fact database"
3. "It can sound confident even when wrong"

---

## Exercise 5: Trace the Pipeline for a Short Sentence

**Task:** For "The sky is", trace through: (1) How many tokens? (2) What might the model attend to when predicting the next token? (3) What are 3 plausible next tokens and why?

**Validation:**
- [ ] Correct token count (~4 tokens)
- [ ] Identifies "sky" and "is" as highly relevant for "blue" / "cloudy" / etc.
- [ ] Gives plausible completions with brief reasoning

**Hints:**
1. Tokenize first
2. "Sky" + "is" → color/weather adjectives
3. Plausible: blue, cloudy, gray, clear