Skip to content

Keypair Rotation

Jump to bottom
Aaron D Borden edited this page Oct 14, 2020 · 25 revisions

This document describes what secret credentials we maintain and how to rotate them.

Ansible Vault

We currently have two Ansible Vaults: production and sandbox, each with their own vault password.

You'll need both the existing key and the new key. Adjust the parameters for the new secret and run ansible rekey in datagov-deploy. Don't forget to add the new key to Google Drive.

For production, you'll want to rekey all the vault files for production, staging, and mgmt inventories (i.e. filter out the sandbox vault files).

$ vault_id=production
$ vault_password_file=.secrets/production-$(date +%Y%m%d).txt
$ find . -name vault.yml | grep -v 'sandbox\|local' | xargs pipenv run ansible-vault rekey --new-vault-id ${vault_id}@${vault_password_file}

To rekey the sandbox vault:

$ vault_id=sandbox
$ vault_password_file=.secrets/sandbox-$(date +%Y%m%d).txt
$ find . -name vault.yml | grep 'sandbox\|local' | xargs pipenv run ansible-vault rekey --new-vault-id ${vault_id}@${vault_password_file}

Vault re-keys should be done as a hotfix to minimize the chance of merge conflicts within vaults and so all branches (master and develop) aren't using multiple key versions.

On the jumpbox, add a new file for the key (/etc/datagov/ansible-secret-v20200709.txt) with permissions root:operators 0640. Update .env to point to the new key file. Validate with pipenv run ansible -m ping all.

SSH root keys

The root ssh keys allow access to the ubuntu user. These keys are only used for the initial provision of the jumpbox. After that, operators use their personal keys to access all hosts.

  1. Generate a new SSH key. When prompted, set a 16+ character password. For the comment, use the key name (datagov-environment-vX).

    $ ssh-keygen -f ~/.ssh/datagov-environment-vX

  2. Upload the private and public keys to the shared Drive. Share the password with teammates over a secure channel.

  3. For BSP environments (production and staging), create a Service Now ticket to update the SSH key in the CloudFormation templates. Attach the public key to the ticket. BSP does not need the private key.

  4. For AWS Sandbox environments, import the key in the AWS Console. Update the keypair name in datagov-infrastructure-live.

Clone this wiki locally