I am trying to deploy my app to Heroku however I rely on using some private git repos as modules. I do this for code reuse between projects, e.g. I have a custom logger I use in multiple apps.
"logger":"git+ssh://git@bitbucket.org..............#master"
The problem is Heroku obviously does not have ssh access to this code. I can't find anything on this problem. Ideally Heroku have a public key I can can just add to the modules.
GitHub has support for basic auth:
"dependencies" : {
"my-module" : "git+https://my_username:my_password@github.com/my_github_account/my_repo.git"
}
As does BitBucket:
"dependencies" : {
"my-module": "git+https://my_username:my_password@bitbucket.org/my_bitbucket_account/my_repo.git"
}
But having plain passwords in your package.json
is probably not desired.
To make this answer more up-to-date, I would now suggest using an access token instead of username/password combo.
So instead of:
"dependencies" : {
"my-module" : "git+https://my_username:my_password@github.com/my_github_account/my_repo.git"
}
You should now use:
"dependencies" : {
"my-module" : "git+https://<token>:x-oauth-basic@github.com/my_github_account/my_repo.git"
}
For Github you can generate a new token here:
https://github.com/settings/applications#personal-access-tokens
For BitBucket you can generate an API Key on the Manage Team page and then use this URL:
"dependencies" : {
"my-module" : "git+https://x-oauth-basic:<api-key>@bitbucket.org/team_name/repo_name.git"
}
Take note of the difference with the GitHub url, that for BitBucket the username instead of the password in substituted with x-oauth-basic
.
It's a REALLY bad idea to have plain text passwords in your git repo, using an access token is better, but you will still want to be super careful.
"my_module": "git+https://ACCESS_TOKEN:x-oauth-basic@github.com/me/my_module.git"
I created a custom nodeJS buildpack that will allow you to specify an SSH key that is registered with ssh-agent and used by npm when dynos are first setup. It seamlessly allows you to specify your module as an ssh url in your package.json
like shown:
"private_module": "git+ssh://git@github.com:me/my_module.git"
To setup your app to use your private key:
ssh-keygen -t rsa -C "your_email@example.com"
(Enter no passphrase. The buildpack does not support keys with passphrases)pbcopy < ~/.ssh/id_rsa.pub
(in OS X) and paste the results into the github admincat id_rsa | base64 | pbcopy
, then heroku config:set GIT_SSH_KEY=<paste_here> --app your-app-name
My custom buildpack can be found here: https://github.com/thirdiron/heroku-buildpack-nodejs and it works for my system. Comments and pull requests are more than welcome.
I've come up with an alternative to Michael's answer, retaining the (IMO) favourable requirement of keeping your credentials out of source control, whilst not requiring a custom buildpack. This was borne out of frustration that the buildpack linked by Michael is rather out of date.
The solution is to setup and tear down the SSH environment in npm's preinstall
and postinstall
scripts, instead of in the buildpack.
Follow these instructions:
setup-ssh.script
and cleanup-ssh.script
.chmod +x *.script
).setup-ssh.script
: #!/bin/bash
# Generates an SSH config file for connections if a config var exists.
if [ "$GIT_SSH_KEY" != "" ]; then
echo "Detected SSH key for git. Adding SSH config" >&1
echo "" >&1
# Ensure we have an ssh folder
if [ ! -d ~/.ssh ]; then
mkdir -p ~/.ssh
chmod 700 ~/.ssh
fi
# Load the private key into a file.
echo $GIT_SSH_KEY | base64 --decode > ~/.ssh/deploy_key
# Change the permissions on the file to
# be read-only for this user.
chmod 400 ~/.ssh/deploy_key
# Setup the ssh config file.
echo -e "Host github.com\n"\
" IdentityFile ~/.ssh/deploy_key\n"\
" IdentitiesOnly yes\n"\
" UserKnownHostsFile=/dev/null\n"\
" StrictHostKeyChecking no"\
> ~/.ssh/config
fi
cleanup-ssh.script
: #!/bin/bash
if [ "$GIT_SSH_KEY" != "" ]; then
echo "Cleaning up SSH config" >&1
echo "" >&1
# Now that npm has finished running, we shouldn't need the ssh key/config anymore.
# Remove the files that we created.
rm -f ~/.ssh/config
rm -f ~/.ssh/deploy_key
# Clear that sensitive key data from the environment
export GIT_SSH_KEY=0
fi
Add the following to your package.json
:
"scripts": {
"preinstall": "bash setup-ssh.script",
"postinstall": "bash cleanup-ssh.script"
}
Generate a private/public key pair using ssh-agent
.
GIT_SSH_KEY
.When Heroku builds your app, before npm installs your dependencies, the setup-ssh.script
script is run. This creates a private key file from the decoded contents of the GIT_SSH_KEY
environment variable, and creates an SSH config file to tell SSH to use this file when connecting to github.com
. (If you are connecting to Bitbucket instead, then update the Host
entry in the setup-ssh.script
script to bitbucket.org
). npm then installs the modules using this SSH config. After installation, the private key is removed and the config is wiped.
This allows Heroku to pull down your private modules via SSH, while keeping the private key out of the codebase. If your private key becomes compromised, since it is just one half of a deploy key, you can revoke the public key in GitHub and regenerate the keypair.
As an aside, since GitHub deploy keys have read/write permissions, if you are hosting the module in a GitHub organization, you can instead create a read-only team and assign a 'deploy' user to it. The deploy user can then be configured with the public half of the keypair. This adds an extra layer of security to your module.
You can use in package.json private repository with authentication example below:
https://usernamegit:passwordgit@github.com/reponame/web/tarball/branchname
In short it is not possible. The best solution to this problem I came up with is to use the new git subtree's. At the time of writing they are not in the official git source and so needs to be installed manual but they will be included in v1.7.11. At the moment it is available on homebrew and apt-get. it is then a case of doing
git subtree add -P /node_modules/someprivatemodue git@github.......someprivatemodule {master|tag|commit}
this bulks out the repo size but an update is easy by doing the command above with gitsubtree pull.
I have done this before with modules from github. Npm currently accepts the name of the package or a link to a tar.gz
file which contains the package.
For example if you want to use express.js directly from Github (grab the link via the download section) you could do:
"dependencies" : {
"express" : "https://github.com/visionmedia/express/tarball/2.5.9"
}
So you need to find a way to access you repository as a tar.gz
file via http(s).