View source for Monorepo Management with Git – Best Practices and Workflows

<span id="monorepo-management-with-git-best-practices-and-workflows"></span>
= Monorepo Management with Git – Best Practices and '''Workflows''' =

<span id="introduction"></span>
== 1) Introduction ==

'''What is a monorepo?''' 

A ''monorepo'' is a single Git repository that stores many projects (apps, services, libraries, tools) together. 

Example: <code>apps/web</code>, <code>apps/mobile</code>, <code>libs/ui</code>, <code>libs/auth</code>, <code>infra/terraform</code>.

'''Why teams use it'''

* One place for all code → easier discovery and code search.
* One set of tools → shared linters, formatters, CI jobs.
* Atomic changes → update many modules in one commit.
* Strong reuse → shared libraries stay in sync.


-----

<span id="benefits-drawbacks"></span>

== 2) Benefits &amp; Drawbacks ==

'''When a monorepo is a good choice'''

* You share code across many apps.
* You do frequent cross-module refactors.
* You want one CI system and one set of rules.
* You prefer one version of dependencies per language.

'''Potential challenges'''

* Repository size can become very large.
* Full clone can be slow (we fix this below with partial clone + sparse-checkout).
* Conflicts can affect many teams if boundaries are unclear.
* Ownership and permissions are trickier (use <code>CODEOWNERS</code> and docs).

'''Rule of thumb''' 

If modules are tightly related and often change together → monorepo is helpful. 

If modules are unrelated, have different lifecycles, or strict isolation → consider multiple repos.


-----

<span id="repository-setup"></span>

== 3) Repository Setup ==

<span id="folder-structure-examples"></span>
=== 3.1 Folder structure examples ===

'''Option A: Apps + Libs (common)'''

<pre>.
├─ apps/
│  ├─ web/
│  ├─ mobile/
│  └─ api/
├─ libs/
│  ├─ ui/
│  ├─ auth/
│  └─ data/
├─ tools/            # scripts, generators, code mods
├─ infra/            # IaC / Terraform / k8s manifests
├─ docs/
├─ .github/          # CI workflows (GitHub Actions) or .gitlab-ci.yml
├─ package.json / pyproject.toml / etc.
└─ README.md</pre>
'''Option B: Services by domain'''

<pre>.
├─ services/
│  ├─ billing/
│  ├─ identity/
│  └─ notifications/
├─ shared/
│  ├─ logging/
│  ├─ metrics/
│  └─ test-utils/
└─ ...</pre>
'''Tips'''

* Put a '''README.md''' in each module (<code>apps/*</code>, <code>libs/*</code>) with: purpose, owners, how to build/test.
* Keep '''module boundaries clear''': what is public API vs. internal.
* Store '''module metadata''' (owners, build/test commands) in a small file (e.g., <code>module.yaml</code>).

<span id="gitignore-setup"></span>
=== 3.2 <code>.gitignore</code> setup ===

Root <code>.gitignore</code> (example):

<pre class="gitignore"># OS junk
.DS_Store
Thumbs.db

# Editors
.vscode/
.idea/

# Build outputs
/build/
/dist/
/out/
/coverage/
/logs/
/tmp/

# Languages/tools (add more as needed)
node_modules/
*.log
*.tsbuildinfo
.venv/
__pycache__/
target/          # Rust
bin/
obj/

# Keep empty dirs tracked
!.gitkeep</pre>
Per-language ignores can also live inside each module (e.g., <code>apps/web/.gitignore</code>) if needed.

'''Why?''' 

Keeping build outputs and local files out of Git keeps the repo '''clean and fast'''.

<span id="tips-for-subfolders-and-shared-libraries"></span>

=== 3.3 Tips for subfolders and shared libraries ===

* Keep shared code in <code>libs/</code>. Publish a '''clear API'''; avoid deep imports between modules.
* Use language workspace features:
** JS/TS: npm/yarn/pnpm '''workspaces'''.
** Python: editable installs (<code>pip install -e .</code>), Poetry workspaces.
** Go: multi-module or workspaces as needed.
* Add '''CODEOWNERS''' at the root to route reviews by path.
* Consider '''versioning''' shared libs internally (e.g., tags or changelogs) so consumers know what changed.


-----

<span id="managing-large-monorepos-with-git"></span>
== 4) Managing Large Monorepos with Git ==

<span id="git-sparse-checkout-concept-quick-start"></span>
=== 4.1 <code>git sparse-checkout</code> (concept + quick start) ===

'''Concept''' 

Only materialize selected folders in your working tree. You ''see and build'' only what you need.

'''Steps (cone mode is easiest):'''

<syntaxhighlight lang="bash"># 1) Partial clone + no checkout yet (faster)
git clone --filter=blob:none --no-checkout <REPO_URL> monorepo
cd monorepo

# 2) Enable sparse-checkout in "cone" mode (fast path-based)
git sparse-checkout init --cone

# 3) Choose the folders you want
git sparse-checkout set apps/web libs/ui

# 4) Checkout the branch
git checkout main</syntaxhighlight>
'''Common commands'''

<syntaxhighlight lang="bash"># Add more folders later
git sparse-checkout add apps/api

# Replace the set (switch modules)
git sparse-checkout set apps/mobile libs/auth

# Reapply patterns after branch changes
git sparse-checkout reapply

# Turn it off and get full tree (careful with size)
git sparse-checkout disable</syntaxhighlight>
'''Why?''' 

You avoid checking out huge directories you don’t need → '''faster status/build/test'''.

<span id="partial-clone---filterblobnone"></span>

=== 4.2 Partial clone (<code>--filter=blob:none</code>) ===

'''Concept''' 

Download commits and trees, but '''skip file blobs''' until you actually need a file. Git will lazily fetch the blob on demand.

'''Examples'''

<syntaxhighlight lang="bash"># Fresh clone
git clone --filter=blob:none <REPO_URL>

# Pull with filter (keep future fetches light)
git pull --filter=blob:none

# Fetch with filter
git fetch origin main --filter=blob:none
git switch main</syntaxhighlight>
'''Why?''' 

Saves bandwidth and disk space, especially for large file histories.

<blockquote>💡 '''Best combo''': Partial clone '''plus''' sparse-checkout gives the biggest speedup.
</blockquote>
<span id="git-worktree-for-multi-branch-work"></span>

=== 4.3 <code>git worktree</code> for multi-branch work ===

Work on multiple branches or reviews at the same time '''without re-cloning'''.

<syntaxhighlight lang="bash"># From your main checkout (e.g., ~/src/monorepo)
# Create a new working tree in a sibling folder on a new branch
git worktree add ../monorepo-doc-fix -b docs/fix origin/main

# List worktrees
git worktree list

# Remove when done (from the main repo)
git worktree remove ../monorepo-doc-fix</syntaxhighlight>
'''Combine with sparse-checkout inside each worktree:'''

<syntaxhighlight lang="bash">cd ../monorepo-doc-fix
git sparse-checkout init --cone
git sparse-checkout set docs tools</syntaxhighlight>
<span id="switch-between-modules-efficiently"></span>
=== 4.4 Switch between modules efficiently ===

'''Approach A: one-liner with sparse-checkout'''

<syntaxhighlight lang="bash"># Switch focus to apps/mobile + libs/auth
git sparse-checkout set apps/mobile libs/auth</syntaxhighlight>
'''Approach B: helper script'''

Create <code>scripts/use-module.sh</code>:

<syntaxhighlight lang="bash">#!/usr/bin/env bash
set -euo pipefail
mod="$1"
git sparse-checkout init --cone >/dev/null 2>&1 || true
git sparse-checkout set "$mod" libs/shared
echo "Now focused on: $mod (and libs/shared)"</syntaxhighlight>
Usage:

<syntaxhighlight lang="bash">./scripts/use-module.sh apps/web</syntaxhighlight>

-----

<span id="branching-strategy"></span>
== 5) Branching Strategy ==

<span id="recommended-trunk-based-development-tbd"></span>
=== 5.1 Recommended: Trunk-Based Development (TBD) ===

* '''Short-lived branches''' (hours or 1–2 days).
* '''Small PRs''' with fast reviews.
* '''Feature flags''' for incomplete work instead of long-lived branches.
* '''Protected <code>main</code>''' branch with required checks.

Example branch names:

<pre>feat/apps-web-header
fix/libs-auth-token-refresh
docs/infra-runbook</pre>
Example commit message (Conventional Commits style helps in monorepos):

<pre>feat(apps/web): add sticky header to product page</pre>
<span id="isolate-module-specific-changes"></span>
=== 5.2 Isolate module-specific changes ===

Only stage and commit files in the target module:

<syntaxhighlight lang="bash"># Stage only module A
git add apps/web

# Show diffs for that module
git diff -- apps/web

# Commit
git commit -m "fix(apps/web): correct cart subtotal"</syntaxhighlight>
This keeps history clean and makes reviews easier.


-----

<span id="cicd-integration-tips"></span>
== 6) CI/CD Integration Tips ==

<span id="fetch-only-needed-folders-in-ci"></span>
=== 6.1 Fetch only needed folders in CI ===

'''Idea''': Check out only the paths used by a job (faster checkout, faster build).

'''GitHub Actions example'''

<syntaxhighlight lang="yaml">name: build-web

on:
  push:
    paths:
      - 'apps/web/**'
      - 'libs/ui/**'
      - '.github/workflows/build-web.yml'

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
          # Limit working tree to the paths we care about
          sparse-checkout: |
            apps/web
            libs/ui
          sparse-checkout-cone: true
      - name: Speed up network
        run: git fetch --filter=blob:none
      - name: Install deps (example for JS workspaces)
        run: npm ci --workspaces=false && npm ci --workspace apps/web --workspace libs/ui
      - name: Build
        run: npm run -w apps/web build
      - name: Test
        run: npm test -w apps/web</syntaxhighlight>
'''GitLab CI example'''

<syntaxhighlight lang="yaml">build_web:
  image: node:20
  rules:
    - changes:
        - apps/web/**/*
        - libs/ui/**/*
  script:
    - git config --global advice.detachedHead false
    - git sparse-checkout init --cone
    - git sparse-checkout set apps/web libs/ui
    - git fetch --filter=blob:none
    - npm ci --workspaces=false
    - npm ci --workspace apps/web --workspace libs/ui
    - npm run -w apps/web build</syntaxhighlight>
'''Other tips'''

* Cache per-path (<code>apps/web/.cache</code>, <code>libs/ui/.cache</code>) to avoid cross-module cache pollution.
* Split jobs by path to keep pipelines parallel and fast.


-----

<span id="common-issues-troubleshooting"></span>
== 7) Common Issues &amp; Troubleshooting ==

<span id="merge-conflicts-across-modules"></span>
=== 7.1 Merge conflicts across modules ===

'''Why it happens'''

Two branches touched the same files or shared APIs changed without coordination.

'''Fix (step by step)'''

<ol style="list-style-type: decimal;">
<li><p>Pull latest <code>main</code>.</p>
<syntaxhighlight lang="bash">git switch your-branch
git fetch origin
git rebase origin/main</syntaxhighlight></li>
<li><p>Resolve conflicts locally (use your IDE’s merge tool).</p></li>
<li><p>'''Run affected tests only''' (see below).</p></li>
<li><p>Commit the resolution.</p>
<syntaxhighlight lang="bash">git add .
git rebase --continue</syntaxhighlight></li>
<li><p>Push.</p>
<syntaxhighlight lang="bash">git push --force-with-lease</syntaxhighlight></li></ol>

'''Tip''': Enable Git’s conflict reuse:

<syntaxhighlight lang="bash">git config --global rerere.enabled true</syntaxhighlight>
This lets Git remember how you resolved a conflict before.

<span id="sparse-checkout-sync-issues"></span>

=== 7.2 Sparse-checkout sync issues ===

'''Symptoms'''

* Files appear missing after a branch switch.
* Build tools complain a folder is not present.

'''Fix'''

<syntaxhighlight lang="bash"># Ensure cone mode + reapply patterns
git sparse-checkout init --cone
git sparse-checkout reapply

# If patterns changed, set them again
git sparse-checkout set apps/web libs/ui</syntaxhighlight>
If you ''really'' need a full tree (rare), disable:

<syntaxhighlight lang="bash">git sparse-checkout disable</syntaxhighlight>
'''Gotcha''': Sparse patterns are per-working-tree. If you use <code>git worktree</code>, set patterns in each worktree.

<span id="partial-clone-missing-blob-during-build"></span>
=== 7.3 Partial clone “missing blob” during build ===

This is normal. Git fetches file blobs on demand. Ensure network access and try:

<syntaxhighlight lang="bash">git fetch --filter=blob:none --recurse-submodules=no</syntaxhighlight>
<span id="accidentally-staged-the-whole-repo"></span>
=== 7.4 Accidentally staged the whole repo ===

Unstage everything, then add only your module:

<syntaxhighlight lang="bash">git reset
git add apps/mobile</syntaxhighlight>

-----

<span id="best-practices-summary"></span>
== 8) Best Practices Summary ==

<span id="naming-conventions"></span>
=== 8.1 Naming conventions ===

* '''Branches''': <code>feat/&lt;path&gt;</code>, <code>fix/&lt;path&gt;</code>, <code>docs/&lt;path&gt;</code>. Example: <code>feat/apps-web-header</code>.
* '''Commits''': Conventional Commits with module scope: <code>feat(apps/web): …</code>, <code>fix(libs/auth): …</code>.
* '''Tags/Releases''' (optional): tag per app if you release apps separately (e.g., <code>web-v2.3.0</code>).

<span id="scripts-for-common-developer-tasks"></span>
=== 8.2 Scripts for common developer tasks ===

Create top-level scripts (Bash, Makefile, npm scripts, or Taskfile):

<pre>scripts/
  use-module.sh         # set sparse-checkout to a module
  affected.sh           # list modules changed since main
  fmt.sh                # run formatters across repo
  test.sh               # run tests for selected modules</pre>
Example “affected” (very simple):

<syntaxhighlight lang="bash">#!/usr/bin/env bash
base=${1:-origin/main}
git diff --name-only "$base"...HEAD | cut -d/ -f1-2 | sort -u</syntaxhighlight>
<span id="keep-the-repo-clean-and-fast"></span>
=== 8.3 Keep the repo clean and fast ===

<ul>
<li><p>Prefer '''partial clone + sparse-checkout''' for devs and CI.</p></li>
<li><p>Use '''Git LFS''' for large binary assets (images, media, big models).</p></li>
<li><p>Do '''periodic cleanup''':</p>
<syntaxhighlight lang="bash">git gc --prune=now</syntaxhighlight></li>
<li><p>Avoid committing build outputs and lockfiles from other tools accidentally.</p></li>
<li><p>Add '''CODEOWNERS''' and module READMEs to document ''ownership and boundaries''.</p></li>
<li><p>Keep PRs small and focused on '''one module''' when possible.</p></li></ul>


-----

<span id="step-by-step-quickstart-copypaste"></span>
== 9) Step-by-step Quickstart (copy/paste) ==

# '''Clone fast'''

<syntaxhighlight lang="bash">git clone --filter=blob:none --no-checkout <REPO_URL> monorepo
cd monorepo</syntaxhighlight>
<ol start="2" style="list-style-type: decimal;">
<li>'''Focus on a module'''</li></ol>

<syntaxhighlight lang="bash">git sparse-checkout init --cone
git sparse-checkout set apps/web libs/ui
git checkout main</syntaxhighlight>
<ol start="3" style="list-style-type: decimal;">
<li>'''Create a short-lived feature branch'''</li></ol>

<syntaxhighlight lang="bash">git switch -c feat/apps-web-header</syntaxhighlight>
<ol start="4" style="list-style-type: decimal;">
<li>'''Work only in your module'''</li></ol>

<syntaxhighlight lang="bash"># edit files under apps/web and libs/ui
git add apps/web libs/ui
git commit -m "feat(apps/web): add sticky header"
git push -u origin HEAD</syntaxhighlight>
<ol start="5" style="list-style-type: decimal;">
<li>'''Open a PR''' (small, fast review), then merge.</li></ol>


-----

<span id="collaboration-habits"></span>
== 10) Collaboration Habits ==

* Use '''presets''': shared prettier/eslint/black/gofmt configs at the root.
* Keep '''docs up to date''': each module’s README shows build, test, and owners.
* Respect '''ownership''': CODEOWNERS routes reviews to the right team.
* Communicate '''breaking changes''' in shared libs; consider changelogs.
* Prefer '''feature flags''' over long-lived branches.


-----

<span id="appendix-useful-commands-cheat-sheet"></span>
=== Appendix: Useful Commands (cheat sheet) ===

<syntaxhighlight lang="bash"># Show only changes under a path
git status -- apps/web
git diff -- apps/web

# Stage only a path
git add -- apps/web

# Reapply sparse-checkout patterns (after switching branches)
git sparse-checkout reapply

# Add another folder to your sparse set
git sparse-checkout add libs/auth

# Create another working tree on a different branch
git worktree add ../mono-exp -b exp/new-idea origin/main</syntaxhighlight>