Skip to content

paul-hammant/googles-monorepo-demo

Repository files navigation

Demo of a expanding/contracting monorepo for mortals

This repo is a fork of https://github.com/google/guava - with no intention of merging back. This is about demonstrating how Google uses a "sparse checkout" (Git and Subversion language) with their gigantic Monorepo (86TB of history, 9 million unique files).

Guava uses Maven - which is why I chose it. (you can do expanding/contracting monorepos with Maven too). Google uses Blaze internally (which was open sourced as Bazel - with the expand/contract feature missing).

Doing a 'quick' experiment with this repo/branch

Say that you goal is to only build/test 'guava-testlib' and not any of the other sub-modules that Maven were to recurse to by habit if run from the root level:

Initial setup:

git clone [email protected]:paul-hammant/googles-monorepo-demo.git
cd googles-monorepo-demo

To run the experiment:

git config core.sparsecheckout true
echo '/mr' > .git/info/sparse-checkout
echo '/README.md' >> .git/info/sparse-checkout
echo '/pom*' >> .git/info/sparse-checkout
echo '/guava/' >> .git/info/sparse-checkout
echo '/guava-testlib/' >> .git/info/sparse-checkout
mr/checkout.sh
mvn install

The huge 'samples' directory isn't in the checkout (yes yes, it is in the clone, relax). It isn't in the root POM's modules either - which is what we wanted.

Making your own Maven setup like this

In your repo, all pom.xml files need to be renamed to pom-template.xml:

find . -name pom.xml -type f | while read a; do n=$(echo $a | sed -e 's/pom.xml/pom-template.xml/'); git mv $a $n; done

You'll need to checkin the mr directory we have above. You'll also want to add pom.xml to .gitignore - they're not under source control any more, pom-template.xml is instead.

About

Small demo of a expanding contracting monorepo, like Google has internally

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages