Migrate a Large (2GB+) Repository to GitHub

Intro

If you are migrating to GitHub having a repository larger than 2GB or have files in history larger than 100MB, you might encounter some resistance from GitHub.

GH001: Large files detected

If you have files larger, in your git history, larger than 100MB, you will most likely receive and error message including "GH001: Large files detected".

The solution for this problem is to rewrite your git history.

The question is "How". After trying a few approaches and failing miserably, I finally stumbled upon an awesome repository on GitHub.

Git-Filter-Repo can clean up large files from the git history blazing fast.

Remove Large Files From Git History

After git-filter-repo installation:

1. Make a fresh clone of your repository with the flag --mirror

2. Inside the new clone folder run:‌‌‌‌‌

git filter-repo --strip-blobs-bigger-than 99MB

After filter-repo is finished you will have cleaned up the bare repository.

If your repository is not too large (less than 2GB) then you can run the following command to push the repository and all branches to the new GitHub repository:

git push --mirror {github-repo-link}

Push a Large Repository to GitHub Incrementally

1. If your repository is too large (2GB+) then you will need to push it to GitHub incrementally with the following command (Mac OS):

git rev-list --reverse --all | ruby -ne 'x ||=0; x += 1; print $_ if x % 30000 == 0;' | xargs -I{} git push {github-repo-link} +{}:refs/heads/master

2. The above command will leave a few commits out so to push the rest of the master branch run:

git push {github-repo-link} +master

3. And finally push all branches:

git push --mirror {github-repo-link}

That's it!

That's how I managed to migrate a large repository with unnecessary large files to GitHub.