Customer hosting CipherStash Token Service and ZeroKMS in AWS

This guide covers how to deploy CipherStash Token Service (CTS) and ZeroKMS to your own AWS infrastructure. These services are the core services that back CipherStash DynamoDB and CipherStash Proxy.

Note

Hosting CTS and ZeroKMS on your own infrastructure should be considered a more advanced use case, as the CipherStash Cloud hosted instances will meet the needs of most users.

At the end of this guide, you will have deployed ZeroKMS and CTS to your own infrastructure.

Prerequisites

Before starting, you'll need to have the following dependencies installed on your machine:

Git
AWS CLI
Node.JS or Bun: In this guide we'll be using Bun.
direnv: This is used to load environment variables from .envrc files.

You'll also need the following resources:

An AWS account to host CTS and ZeroKMS.
A Route53 hosted zone configured in the same AWS account.
- Make sure the NS records for the hosted zone are set up in your domain registrar, pointing to the Route53 hosted zone.
A Cloudsmith token provided by our team for downloading CTS and ZeroKMS artifacts.
An Auth0 or Okta account.

Security Note

It's recommended to use two AWS accounts and two Route53 zones to deploy CTS and ZeroKMS in separate accounts for security best practices.

Getting started

Step 1: Prepare the CipherStash CDK repo

Start by cloning the GitHub repo:

1git clone [email protected]:cipherstash/cipherstash-aws-cdk-typescript.git

Navigate to the directory and install the dependencies:

1cd cipherstash-aws-cdk-typescript
2bun install

Step 1.1: Set up the `.envrc` files

You will be using two different .envrc files for this guide. Create your .envrc files by copying the example files:

1cp .envrc.example .envrc
2cp cli-workspace/.envrc.example cli-workspace/.envrc

Allow the .envrc files to be loaded by direnv:

1direnv allow .
2direnv allow cli-workspace

You will be updating these files with the required environment variables in the following steps.

Step 2: Configure your identity provider

Depending on whether you want to use Okta or Auth0, use the following steps to configure your identity provider:

Step 3: Download Lambda Zips

Lambda zips are distributed via Cloudsmith. Before doing the deployment you'll need to download them and place them in the zips folder in the CDK example repo.

To download the zips, you'll need to export the following environment variables which will be configured if you've followed the previous steps for setting up the .envrc file and have direnv installed. If you haven't done this, you can export them manually:

1export CLOUDSMITH_TOKEN=
2export LAMBDA_VERSION=latest

Step 3.1: Download the Lambda zips

1wget -O zips/cts-migrations.zip \
2  "https://dl.cloudsmith.io/${CLOUDSMITH_TOKEN}/cipherstash/lambdas/raw/names/cts-migrations/versions/${LAMBDA_VERSION}/cts-migrations.zip"
3
4wget -O zips/cts.zip \
5  "https://dl.cloudsmith.io/${CLOUDSMITH_TOKEN}/cipherstash/lambdas/raw/names/cts/versions/${LAMBDA_VERSION}/cts.zip"
6
7wget -O zips/zerokms-migrations.zip \
8  "https://dl.cloudsmith.io/${CLOUDSMITH_TOKEN}/cipherstash/lambdas/raw/names/zerokms-migrations/versions/${LAMBDA_VERSION}/zerokms-migrations.zip"
9
10wget -O zips/zerokms.zip \
11  "https://dl.cloudsmith.io/${CLOUDSMITH_TOKEN}/cipherstash/lambdas/raw/names/zerokms/versions/${LAMBDA_VERSION}/zerokms.zip"

Verify that the lambdas have downloaded correctly by checking the zips folder:

1ls zips

Step 4: Bootstrapping and deploying

Before deploying the CDK stacks, you'll need to set some environment variables to configure the deployment. If you've followed the previous steps for setting up the .envrc file and have direnv installed, these will be configured for you. If you haven't done this, you can export them manually:

1# AWS account ID for CTS
2export CTS_ACCOUNT_ID=
3
4# Your Identity provider URL (with a trailing slash)
5export CTS_TOKEN_ISSUER=
6
7# The IDP provider that will generate user tokens.
8# Either auth0 or okta.
9export CTS_TOKEN_PROVIDER=
10
11# Name of an existing Route53 hosted zone to use for ACM and API GW
12export CTS_ROUTE53_ZONE_NAME=
13
14# AWS account ID for ZeroKMS
15export ZEROKMS_ACCOUNT_ID=
16
17# Name of an existing Route53 hosted zone to use for ACM and API GW
18export ZEROKMS_ROUTE53_ZONE_NAME=
19
20# Name of the AWS region for deployment of the resources
21export AWS_REGION=
22
23# Name of the AWS profile which is configured the the AWS CLI
24export AWS_PROFILE=

Step 4.1: Authenticate the AWS CLI with a profile

Use the following command to configure an SSO profile for the AWS CLI, if you haven't already:

1aws configure sso

Follow the prompts to configure the profile with the SSO URL, the account ID, and the role name. Then login with the SSO profile:

1aws sso login --profile <your SSO profile>

Step 4.2: Bootstrap and deploy CTS

Use the CDK CLI to bootstrap the account that CTS will be deployed to (the AWS_PROFILE is defined in bin/cipherstash-aws-cdk.ts file):

1bun cdk bootstrap "${CTS_ACCOUNT_ID}/${AWS_REGION}"

After the bootstrap process completes, deploy the CTS stack with the following:

1bun cdk deploy CipherStashCtsStack

This process will take some time, but once it completes it will return the name of the migration lambda. Keep note of this as it will be used in future steps.

1Outputs:
2CipherStashCtsStack.ApiUrl = https://cts.<your CTS Route53 zone name>/
3CipherStashCtsStack.MigrationFunctionName = CipherStashCtsStack-MigrationsFunction<random string>

Take note of the ApiUrl and MigrationFunctionName as they will be used in the next steps.

Step 4.2a: Add your ApiUrl to CLI workspace

Update the cli-workspace/.envrc file with the ApiUrl output from the CTS stack:

1# Output of CipherStashCtsStack.ApiUrl
2export CS_IDP_AUDIENCE="<CipherStashCtsStack.ApiUrl>"
3
4# Output of CipherStashCtsStack.ApiUrl
5export CS_MANAGEMENT_HOST="<CipherStashCtsStack.ApiUrl>"
6
7# Output of CipherStashCtsStack.ApiUrl
8export CS_CTS_HOST="<CipherStashCtsStack.ApiUrl>"

Step 4.3: Bootstrap and deploy ZeroKMS

If you're using the same AWS account as CTS, you can skip the bootstrap step, if not then you'll also need to bootstrap the ZeroKMS account and update the AWS_PROFILE to the ZeroKMS account before executing the following commands.

1bun cdk bootstrap "${ZEROKMS_ACCOUNT_ID}/${AWS_REGION}"

After the bootstrap process completes, deploy the ZeroKMS stack with the following:

1bun cdk deploy CipherStashZeroKmsStack

This process will take some time, but once it completes it will return the following that you will use in the next step:

1Outputs:
2CipherStashZeroKmsStack.ApiUrl = https://zerokms.<your ZeroKMS Route53 zone name>/
3CipherStashZeroKmsStack.MigrationFunctionName = CipherStashZeroKmsStack-MigrationsFunction<random string>

Take note of the ApiUrl and MigrationFunctionName as they will be used in the next steps.

Step 4.3a: Add your ZeroKMS ApiUrl to CLI workspace

Update the cli-workspace/.envrc file with the ApiUrl output from the ZeroKMS stack:

1# Output of CipherStashZeroKmsStack.ApiUrl
2export CS_ZEROKMS_HOST="<CipherStashZeroKmsStack.ApiUrl>"

Step 5: Migrate the database dependencies

Migrations are currently handled using a specific Lambda deployed to the same account as the service. It has the same access to the underlying database but unlike the service can't be accessed publically.

Step 5.1: Migrate the CTS databases

With the name of your CTS MigrationFunctionName from the previous step, invoke the lambda with the setup command to init the database and run the migrations:

1aws lambda invoke --profile $AWS_PROFILE \
2  --cli-binary-format raw-in-base64-out \
3  --function-name '<your CTS MigrationFunctionName here>' \
4  --payload '{ "command": "setup" }' \
5  response.json

Confirm that it returned a status code of 200:

1{
2  "StatusCode": 200,
3  "ExecutedVersion": "$LATEST"
4}

Step 5.2: Migrate the ZeroKMS databases

With the name of your ZeroKMS MigrationFunctionName from the previous step, invoke the lambda with the setup command to init the database and run the migrations. Make sure to set your AWS_PROFILE accordingly:

1aws lambda invoke --profile $AWS_PROFILE \
2  --cli-binary-format raw-in-base64-out \
3  --function-name '<your ZeroKMS MigrationFunctionName here>' \
4  --payload '{ "command": "setup" }' \
5  response.json

Confirm that it returned a status code of 200:

1{
2  "StatusCode": 200,
3  "ExecutedVersion": "$LATEST"
4}

Step 6: Using the CipherStash CLI with your self-hosted CTS and ZeroKMS

Now that CTS and ZeroKMS have been successfully deployed, you can test them using the CipherStash CLI. Navigate to the cli-workspace directory which will load the environment variables for the CLI with direnv:

1cd cli-workspace

Configuration Note

All the environment variables defined in cli-workspace/.envrc are required in use with CipherStash Proxy or CipherStash DynamoDB when self-hosting CTS and ZeroKMS.

To login to the CLI, run the following command:

1stash login

This will prompt you in your browser to verify whether a code matches that that was returned by CipherStash CLI. Verify that it is correct, and click confirm. Then follow the steps to create a new account.

Step 6.2: Verify that everything is working

You can verify that everything is working by running the following:

1stash workspaces

Since you don't have any workspaces at this point, look for an empty table to verify that everything is working.

Step 7: Creating a workspace and host

Now that you've verified that the CLI is working, you can create a new workspace and host in your CTS instance, which will be used to create datasets in ZeroKMS.

Step 7.1: Set permissions for your user

The user you created during the CLI login process will need to have the correct permissions to create a host and workspace in your CTS instance.

Navigate to your identity provider dashboard and assign the cipherstash:admin permission to your user.

You created this permission in step 2.

Step 7.2: Refresh your auth token

Identity provider tokens are short-lived and need to be refreshed to continue using the CLI with the correct permissions:

1stash login

Step 7.3: Create a host and workspace

Hosts are used to define the location of your ZeroKMS instance, and workspaces are used to group datasets together which are used for access control and encryption.

Step 7.3a: Create a host

Create a new host in your CTS instance using the following command:

1stash hosts create --identifier $CS_ZEROKMS_HOST

You should see a response with the HostId of the host created.

1Host created:
2
3ID   : HostId
4Identifier : https://zerokms.<ZeroKMS Route53 zone>/

Step 7.3b: Create a workspace

With the HostId of the host created from the previous step, create a new workspace for that host:

1stash workspaces create --host-id <HostId of the host created>

You should see a response with the ID of the workspace created.

1Workspace created:
2
3ID   : WorkspaceId

Step 7.4: Add your user to the workspace

With the workspace created, you can add your user to the workspace using the following command and the user ID you used to login to the CLI.

You can find your user_id in your identity provider dashboard.

1stash memberships create --user-id '<identity provider user_id>' --workspace-id <WorkspaceId>

You should see a response with the ID of the membership created.

1Membership created:
2
3ID   : MembershipId

Step 7.5: Set the workspace as default

Set your CipherStash CLI default workspace, which will also validate your membership to the workspace:

1stash workspaces switch '<your workspace id>'

Step 7.6: Verify that everything is working