Notes

Load Go modules from private repos

fetching Go modules from private GitHub repos in CI, using a PAT in .netrc or git config on Netlify instead of SSH

Edit on GitHub


Go
5 minutes

i ran into this in June 2024, when loading the notes for fazalashfaq.com from a private repo as a Hugo module, and the Netlify build couldn’t fetch it.

Private repos and non-public modules

Setting up the Hugo module itself (init, mounts, hugo mod commands) is in Load content from a GitHub repo for a Hugo website as a module. This note is about getting it to load when the repo is private.

By default go get uses HTTPS. In a continuous deployment scenario, specially if you have 2FA enabled with your Git provider, you will need to generate a PAT (Personal Access Token) in order to use HTTPS without a prompt, or use SSH instead of HTTPS.

If you’re deploying with Netlify and have continuous deployment, you will likely use PAT instead of SSH.

PAT (Personal Access Token)

  • Generate an access token from Github settings. It should have read access to your code
 1add_github_token() {
 2  # Add a PAT so that git (and so Go) can fetch modules from private repos
 3  # GITHUB_USERNAME and GITHUB_PAT come from environment variables (set them in Netlify),
 4  # don't set them to "" here or you overwrite the real values
 5  FILE="${HOME}/.netrc"
 6
 7  echo "Adding GitHub credentials to ${FILE}"
 8  printf 'machine github.com login %s password %s\n' "${GITHUB_USERNAME}" "${GITHUB_PAT}" >> "${FILE}"
 9  chmod 600 "${FILE}" # only this user can read it, git and curl expect that
10}
11
12add_github_token
PAT on Netlify
  • Save the PAT as an environment variable in Netlify

The value for $HOME in Netlify build context is /opt/buildhome/. If you create a file at $HOME/.netrc it will end up at the location /opt/buildhome/.netrc, and not ~/.netrc or /home/FOO/.netrc as you would normally expect

You can set git config --global inside a Netlify build script. To access a private repo, you can pass your credentials as part of the repo URL. Git repo URLs with credentials look like this:

https://<TOKEN>@github.com/<REPO_OWNER>/<REPO_NAME>
https://<USERNAME>:<TOKEN>@github.com/<REPO_PATH>
https://<USERNAME>:<TOKEN>@<SELF_HOSTED_GITLAB_SERVER>/<PATH_TO_REPO>.git

You’ll replace the values for TOKEN, USERNAME, REPO_NAME, REPO_PATH etc. and then use the final URL to set git config with a insteadOf directive. The config command will look like this:

1git config --global --add url.https://${USERNAME}:${TOKEN}@github.com/.insteadof https://github.com/
2
3# or a more specific
4git config --global --add url."https://${USERNAME}:${TOKEN}@github.com/${REPO_PATH}".insteadOf "https://github.com/${REPO_PATH}"

Note: Both insteadOf and insteadof (with lowercase ‘o’) work. Quoting the strings is optional but recommended.

Careful: with insteadOf the token is part of the URL, and a URL can end up in an error message in the build log. .netrc keeps the token out of URLs, so i prefer that now.

1# netlify.toml
2[build]
3  command = "./netlify_build.sh"
 1# netlify_build.sh
 2settings_for_private_go_modules() {
 3  REPO_PATH="blah_username/blah_repo"
 4
 5  # REPO_PATH is a Go module inside a private repo
 6  # GITHUB_PAT is a Personal Access Token, set as Netlify env var, that has read access for that repo
 7  # token will expire after the validity period
 8  git config --global --add url."https://${GITHUB_USERNAME}:${GITHUB_PAT}@github.com/${REPO_PATH}".insteadOf "https://github.com/${REPO_PATH}"
 9}
10
11settings_for_private_go_modules

Enforce SSH

Whether you clone a repo over HTTPS or SSH is determined by the repo URL.

1# HTTPS
2https://github.com/USER/BLAH.git
1# SSH
2git@github.com:USER/BLAH.git

In order to force Git to always get the repos over SSH, we can use the insteadOf directive in our (global) git config.

1# Force all URL to be SSH 
2git config --global --add url."git@github.com:".insteadOf "https://github.com/"

It will add code that looks like this to your ~/.gitconfig

1[url "git@github.com:"]
2 insteadOf = https://github.com/

GOPRIVATE

You should also set the GOPRIVATE variable (go 1.13 onwards)

The new GOPRIVATE environment variable indicates module paths that are not publicly available. It serves as the default value for the lower-level GONOPROXY and GONOSUMDB variables, which provide finer-grained control over which modules are fetched via proxy and verified using the checksum database. ref

go env -w GOPRIVATE=github.com/{ORG_NAME/USER_NAME}/*

OR Add this to your ~/.bashrc

1export GOPRIVATE="github.com/{ORG_NAME/USER_NAME}/*"

Multiple links can be provided as a comma separated list

1export GOPRIVATE="gitlab.com/FOO,bitbucket.org/FOO,github.com/FOO"

You can verify your changes with go env

Hugo modules from a private repo on Netlify (2026)

My site (aamnah.com) pulls the notes repo in as a Hugo module. Hugo modules are Go modules, so everything above applies, with two differences:

  • No GOPRIVATE needed. Hugo’s module defaults are proxy = 'direct' and private = '*.*': it always fetches straight from the git host and never asks the public Go proxy or checksum database. Check with hugo config | grep -A5 '^\[module\]'.
  • The only missing piece is credentials. Netlify’s GitHub access covers the site’s own repo, not other private repos, so hugo mod get fails with could not read Username as soon as the module repo goes private.

here’s the updated version of what should be done:

  1. A fine-grained token, not a classic PAT (GitHub > Settings > Developer settings > Fine-grained tokens): repository access only the module repo, permission Contents: read-only, and an expiry date with a reminder to renew it. If it leaks, it can only read that one repo.

  2. Save it in Netlify as an environment variable (e.g. NOTES_REPO_TOKEN), marked as a secret, scoped to builds.

  3. Write .netrc in the build script, before hugo mod get:

    1# netlify_build.sh: let git (and so Hugo modules) read the private notes repo
    2if [ -n "${NOTES_REPO_TOKEN:-}" ]; then
    3  printf 'machine github.com login %s password %s\n' "${GITHUB_USERNAME:-aamnah}" "${NOTES_REPO_TOKEN}" > "${HOME}/.netrc"
    4  chmod 600 "${HOME}/.netrc"
    5fi
    
  4. Test it: deploy once while the repo is still public (the script shouldn’t change anything), then make the repo private and trigger a deploy.

An alternative is a read-only deploy key (an SSH key added to just that repo). It never expires, but the private key has to go into a Netlify variable and the build needs SSH set up, so the token is less work.

Deploying the latest notes

Netlify only watches the site repo, so the notes repo has a GitHub webhook that calls a Netlify build hook: every push to the notes rebuilds the site with the latest notes. See trigger Netlify builds with GitHub webhooks.

Related

Browse all topics