{
  "version": 1,
  "id": "monorepos-and-submodules",
  "title": "Monorepos and Submodules",
  "summary": "Trade-offs between monorepos, polyrepos and submodules, and the tooling each needs.",
  "estimatedMinutes": 25,
  "prerequisites": [],
  "source": "courses-notebook ENV202_git_mastery 06_monorepos_and_submodules",
  "lesson": "## 6.1  Git Submodules\n\n```bash\n# Add submodule\ngit submodule add git@github.com:user/lib.git libs/lib\n\n# Clone repo with submodules\ngit clone --recurse-submodules git@github.com:user/repo.git\n\n# Update all submodules\ngit submodule update --remote --merge\n\n# Remove submodule\ngit submodule deinit libs/lib\ngit rm libs/lib\n```\n\n## 6.2  Sparse Checkout (Large Repos)\n\n```bash\ngit clone --filter=blob:none --sparse git@github.com:big/repo.git\ngit sparse-checkout set path/to/subdir another/path\n```\n\n## 6.3  Useful Git Aliases\n\n```bash\n# ~/.gitconfig\n[alias]\n    st = status --short\n    lg = log --oneline --graph --decorate\n    undo = reset --soft HEAD~1\n    wip = !git add -A && git commit -m 'WIP'\n    unwip = reset HEAD~1 --mixed\n    aliases = config --get-regexp alias\n    recent = branch --sort=-committerdate --format='%(refname:short)' -10\n```\n\n```python\nimport subprocess\n\ndef git(cmd):\n    r = subprocess.run(f'git {cmd}', shell=True, capture_output=True, text=True)\n    return r.stdout.strip()\n\nprint('=== Git Aliases Configured ===')\naliases = git('config --get-regexp alias')\nif aliases:\n    for line in aliases.splitlines():\n        parts = line.split(None, 1)\n        if len(parts) == 2:\n            print(f'  {parts[0].replace(\"alias.\",\"\"):15} = {parts[1]}')\nelse:\n    print('  (no aliases configured)')\n    print()\n    print('Add to ~/.gitconfig:')\n    print('  [alias]')\n    print('  lg = log --oneline --graph --decorate')\n```\n\n## Monorepos and Submodules\n\n```bash\n# Git submodules: embed a repo within a repo\ngit submodule add https://github.com/org/shared-lib libs/shared\ngit submodule update --init --recursive   # clone all submodules\ngit submodule foreach git pull origin main  # update all\n\n# Alternatives to submodules:\n# - git subtree (merge history, no .gitmodules)\n# - npm/pip packages (separate releases)\n# - monorepo with workspace tools\n\n# Monorepo with pnpm workspaces\n# pnpm-workspace.yaml:\n# packages:\n#   - 'packages/*'\n#   - 'apps/*'\npnpm -r build           # build all packages\npnpm -r test            # test all packages\npnpm --filter=my-app dev  # run only my-app\n```\n\n```python\nmonorepo_structure = '''\nMonorepo structure (Python + TypeScript):\n\nmy-monorepo/\n\u251c\u2500\u2500 packages/\n\u2502   \u251c\u2500\u2500 shared-utils/\n\u2502   \u2502   \u2514\u2500\u2500 pyproject.toml      # shared Python utilities\n\u2502   \u2514\u2500\u2500 ml-models/\n\u2502       \u2514\u2500\u2500 pyproject.toml      # ML model package\n\u251c\u2500\u2500 apps/\n\u2502   \u251c\u2500\u2500 web-frontend/\n\u2502   \u2502   \u2514\u2500\u2500 package.json        # React app\n\u2502   \u2514\u2500\u2500 api-server/\n\u2502       \u2514\u2500\u2500 pyproject.toml      # FastAPI app\n\u251c\u2500\u2500 uv.toml                     # workspace config\n\u251c\u2500\u2500 pnpm-workspace.yaml         # JS workspace\n\u2514\u2500\u2500 Makefile                    # cross-language commands\n\n# uv workspaces (uv.toml)\n[tool.uv.workspace]\nmembers = [\"packages/*\", \"apps/api-server\"]\n\n# Makefile\ntest:\n    uv run --package shared-utils pytest packages/shared-utils/tests\n    uv run --package api-server pytest apps/api-server/tests\n    pnpm --filter web-frontend test\n'''\nprint(monorepo_structure)\n```\n\n```python\n# Verification helpers \u2014 exercise checker\n\ndef assert_equal(got, want, label=\"\"):\n    status = \"\u2713\" if got == want else f\"\u2717 got={got!r} want={want!r}\"\n    print(f\"  {label or 'test'}: {status}\")\n\ndef assert_approx(got, want, tol=1e-6, label=\"\"):\n    diff = abs(got - want)\n    status = \"\u2713\" if diff < tol else f\"\u2717 diff={diff:.2e}\"\n    print(f\"  {label or 'test'}: {status}\")\n\ndef run_tests(fn, cases):\n    \"\"\"cases: list of (args_tuple, expected)\"\"\"\n    passed = sum(1 for args, want in cases\n                 if (result := fn(*args) if isinstance(args, tuple) else fn(args)) == want\n                 or (print(f\"  FAIL {fn.__name__}{args}: got {result!r}, want {want!r}\"), False))\n    print(f\"{fn.__name__}: {passed}/{len(cases)} passed {'\u2713' if passed==len(cases) else '\u2717'}\")\n\n# \u2500\u2500\u2500 Example: test your implementations below \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n\ndef flatten(nested):\n    \"\"\"Flatten arbitrarily nested lists.\"\"\"\n    result = []\n    for item in nested:\n        if isinstance(item, list):\n            result.extend(flatten(item))\n        else:\n            result.append(item)\n    return result\n\nrun_tests(flatten, [\n    (([1,[2,[3,[4]]],5],),  [1,2,3,4,5]),\n    (([],),                  []),\n    (([[1],[2],[3]],),        [1,2,3]),\n])\n\ndef chunk(lst, size):\n    \"\"\"Split list into chunks of given size.\"\"\"\n    return [lst[i:i+size] for i in range(0, len(lst), size)]\n\nrun_tests(chunk, [\n    (([1,2,3,4,5], 2),  [[1,2],[3,4],[5]]),\n    (([1,2,3], 3),       [[1,2,3]]),\n    (([], 2),            []),\n])\n```\n\n```python\n# Advanced: Git object model \u2014 blob, tree, commit simulation\nimport hashlib, json\nfrom dataclasses import dataclass, field\nfrom typing import Optional\n\ndef sha1(content: str) -> str:\n    return hashlib.sha1(content.encode()).hexdigest()[:8]\n\n@dataclass\nclass GitObject:\n    kind:    str   # blob | tree | commit\n    content: str\n    hash:    str = field(init=False)\n    def __post_init__(self):\n        self.hash = sha1(f\"{self.kind}:{self.content}\")\n\nclass GitRepo:\n    def __init__(self):\n        self.objects: dict[str, GitObject] = {}\n        self.refs:    dict[str, str] = {\"HEAD\": None}\n        self.index:   dict[str, str] = {}   # filename \u2192 blob hash\n\n    def hash_object(self, kind: str, content: str) -> str:\n        obj = GitObject(kind=kind, content=content)\n        self.objects[obj.hash] = obj\n        return obj.hash\n\n    def add(self, filename: str, content: str) -> str:\n        blob_hash = self.hash_object(\"blob\", content)\n        self.index[filename] = blob_hash\n        print(f\"  staged {filename} \u2192 blob {blob_hash}\")\n        return blob_hash\n\n    def commit(self, message: str, author: str = \"dev\") -> str:\n        tree_content = json.dumps(dict(sorted(self.index.items())))\n        tree_hash    = self.hash_object(\"tree\", tree_content)\n        parent       = self.refs.get(\"HEAD\")\n        commit_data  = json.dumps({\"tree\": tree_hash, \"parent\": parent,\n                                   \"author\": author, \"message\": message})\n        commit_hash  = self.hash_object(\"commit\", commit_data)\n        self.refs[\"HEAD\"] = commit_hash\n        print(f\"  commit {commit_hash}  '{message}'  tree={tree_hash}\")\n        return commit_hash\n\n    def log(self):\n        cur = self.refs.get(\"HEAD\")\n        while cur:\n            obj = self.objects.get(cur)\n            if not obj: break\n            data = json.loads(obj.content)\n            print(f'  commit {cur}  \"{data['message']}\"  tree={data['tree']}')\n            cur  = data.get(\"parent\")\n\nrepo = GitRepo()\nprint(\"=== git add ===\")\nrepo.add(\"README.md\",  \"# My Project\")\nrepo.add(\"main.py\",    \"print('hello')\")\nprint(\"=== git commit ===\")\nrepo.commit(\"Initial commit\")\n\nrepo.add(\"main.py\", \"print('hello world')\")   # modify file\nrepo.commit(\"Update main.py\")\n\nprint(\"=== git log ===\")\nrepo.log()\nprint(f\"=== Objects stored: {len(repo.objects)} ===\")\nfor h, obj in list(repo.objects.items())[:6]:\n    print(f\"  {h}  {obj.kind}\")\n```\n\n## Git Quick Reference\n\n```bash\n# Daily workflow\ngit status -s                     # compact status\ngit diff --staged                 # review before commit\ngit commit -m \"type: description\" # conventional commit\ngit push -u origin feature/name   # first push\n\n# Branching\ngit switch -c feature/name        # create + switch\ngit switch main && git pull        # update main\ngit rebase main                    # rebase feature onto main\ngit merge --no-ff feature/name     # merge with history\n\n# Undoing things (safe)\ngit restore <file>                 # discard working tree changes\ngit restore --staged <file>        # unstage\ngit revert HEAD                    # new commit that undoes last\n\n# History inspection\ngit log --oneline --graph --all    # visual branch graph\ngit log -p --follow -- <file>      # file history with diffs\ngit blame -L 10,20 <file>          # who changed lines 10-20\ngit bisect start                   # binary search for bug commit\n\n# Stash\ngit stash push -m \"WIP: feature\"\ngit stash list\ngit stash pop                      # restore latest\n\n# Remote\ngit remote -v\ngit fetch --prune                  # remove stale remote branches\ngit pull --rebase origin main      # rebase instead of merge pull\n```\n\n**Commit types (Conventional Commits):**\n`feat` \u00b7 `fix` \u00b7 `refactor` \u00b7 `docs` \u00b7 `test` \u00b7 `chore` \u00b7 `perf` \u00b7 `ci`\n\n## Failure Modes\n\n- **Submodule detached HEAD**: after `submodule update`, submodules are in detached HEAD \u2014 `cd libs/shared && git checkout main` to track branch\n- **Forgetting `--init --recursive`**: cloned repo without submodules \u2014 run `git submodule update --init --recursive`\n\n## Exercises\n1. Add a git submodule for a shared library, update it to a specific tag\n2. Set up a uv workspace with 2 packages that depend on each other\n3. Configure Turborepo or Nx for monorepo build caching and dependency graph\n\n## Key Takeaways\n\n- **`git submodule add <url> path`**: embeds another repository at a specific commit inside your repo \u2014 the submodule pointer is tracked as a special entry in the parent repo.\n- **`git submodule update --init --recursive`**: the command to run after cloning a repo with submodules \u2014 without it, submodule directories are empty.\n- **Submodule detached HEAD**: after `git submodule update`, submodules are pinned to a commit not a branch \u2014 `cd submodule && git checkout main` to track the branch actively.\n- **`git subtree`**: alternative to submodules that merges the external repo's history \u2014 simpler for contributors but creates noisier history.\n- **Monorepo benefits**: a single repo for multiple packages enables atomic cross-package commits, shared CI, and unified code search \u2014 tools like Turborepo and Nx add caching.\n\n## \ud83e\udde0 Quick Quiz\n\n**Q1.** Git submodules allow:\n- A) Splitting a repo into multiple branches\n- B) **Including another git repository at a specific commit inside your repo** \u2713\n- C) Sharing branches between repos\n- D) Syncing histories between repos\n\n**Q2.** A monorepo contains:\n- A) A single application only\n- B) **Multiple projects in one repository, sharing tooling and version history** \u2713\n- C) Only shared libraries\n- D) Multiple branches per project\n\n**Q3.** `git submodule update --init --recursive` is needed because:\n- A) Submodules auto-update on clone\n- B) **Submodule directories are empty after a fresh clone; this command populates them** \u2713\n- C) It upgrades submodule branches\n- D) It re-initialises the parent repo\n\n<details><summary>Answers</summary>1-B, 2-B, 3-B</details>\n",
  "quiz": []
}