Caution: please use the below knowledge with caution — all the tools described below cannot be used with blind copy-pasting. Be aware of that — you have been warned (commands are not working). (But if you do have any funny/scary stories, please share in the comments).
One day, when working with your git-based project, you may accidentally save your personal logins, passwords or SSH keys to the repository. Of course, you can use git rm to remove the file, but the file will still be in the history.
Once a file with private data is in the repository, all data in it can be considered compromised, and immediate action must be taken (changing passwords, etc.). There is no way to trace if anyone has seen or downloaded these files.
Fortunately, there are tools that allow you to remove a file from a git repository completely. This article describes how to use the BFG Repo-Cleaner and git-filter-branch to completely remove a file from a git repository.
git-filter-branch
git-filter-branch the utility is part of git and does not require additional installation.
Suppose that we accidentally saved the .secret file in the repository, which stores the password for the database. The project is in active development and a test database is being used, but out of habit, the password is the same as the one you use to log in to the system or anywhere else. Sound familiar? I hope not.
After making sure we have the latest version and that there are no local changes, we can delete the file using git-filter-branch:
$ git` filter-branch --force --index-filter \
'git rm --cached --ignore-unmatch .secret' \
--prune-empty --tag-name-filter cat -- --all
Rewrite d402133542d0f0f1578e916b8a350842cc955870 (1327/1333) (216 seconds passed, remaining 0 predicted) rm '.environment'
Ref 'refs/heads/master' was rewritten
Ref 'refs/remotes/origin/master' was rewritten
...
This command will delete the .secret file in each repository commit. If you want to remove the directory, you must add the -f switch to git rm:
$ git` filter-branch --force --index-filter \
'git rm -f --cached --ignore-unmatch folder_to_remove/' \
--prune-empty --tag-name-filter cat -- --all
If there are other files that need to be removed, run this command for each of them.
Now we need to commit all the changes, unfortunately using force push:
$ git` push origin --force --all
After all the changes have been made, everyone else who has worked with this repository needs to rebase. Or delete your local repository and clone it again. The latter is better, since there is less chance of shooting yourself in the foot.
Note: to avoid a repeat of this nasty situation, the file should be added to the.gitignore file.
BFG Repo-Cleaner
BFG Repo-Cleaner — is a simpler and easier alternative to git-filter-branch for removing unwanted files from the git repository.
To install for macOS, but you can build it yourself:
$ brew install bfg
For example, to remove the same .secret file as in the git-filter-branch example above using BFG:
$ bfg` --delete-files .secret
Found 775 objects to protect
Found 161 commit-pointing refs : ...
Protected commits
-----------------
These are your protected commits, and so their contents will NOT be altered:
* commit 89943765 (protected by 'HEAD')
Cleaning
--------
Found 1726 commits
Cleaning commits: 100% (1726/1726)
Cleaning commits completed in 471 ms.
BFG aborting: No refs to update - no dirty commits found??
BFG can even look through all files in the repository for passwords and replace them with **REMOVED**:
$ bfg` --replace-text protected.txt
As a result of this command, each file in each commit will be looked through and if a password from the protected.txt file is found, it will be replaced with **REMOVED**.
Just in case you’re worried about performance: The BFG vs git-filter-branch - speed comparison
Buy
me a coffee