CipherStash Encryption Migrator

CipherStash Encryption Migrator is used to encrypt existing data or to apply index changes after encryption configuration changes of a protected database.

Installation

CipherStash Encryption Migrator is bundled into the cipherstash-proxy Docker image.

Prerequisites

CipherStash Encryption Migrator uses CipherStash Proxy to encrypt.

This guide assumes:

CipherStash Proxy is installed, configured and running
the target table has a corresponding dataset configured to encrypt one or more columns.

See the Getting started with CipherStash Proxy guide. A CipherStash account can be created at https://dashboard.cipherstash.com/.

Usage

Encrypt the source column data in a table into the specified encrypted target column.

1cipherstash-migrator [OPTIONS] --table <TABLE> --columns <SOURCE_COLUMN=TARGET_COLUMN> --database-name <DATABASE_NAME> --username <USERNAME> --password <PASSWORD>

Options

Setting	Description	Default	Environment Variables
`-t`, `--table`	Specifies the table to migrate	None (Required)	`CS_TABLE`
`-k`, `--primary-key`	List of primary key columns (space-delimited)	`id`	`CS_PRIMARY_KEY`
`-c`, `--columns`	List of columns to migrate (key=value pairs space-delimited)	None (Required)	`CS_COLUMNS`
`-H`, `--host`	Host address of CipherStash Proxy instance	`127.0.0.1`	`CS_HOST`
`-P`, `--port`	Port of CipherStash Proxy instance	`6432`	`CS_PORT`
`-N`, `--database-name`	Database name (or CipherStash Proxy pool name)	None (Required)	`CS_DATABASE__NAME`
`-U`, `--username`	Username for the CipherStash Proxy pool	None (Required)	`CS_USERNAME`, `CS_DATABASE__USERNAME`
`-p`, `--password`	Password for the CipherStash Proxy pool	None (Required)	`CS_PASSWORD`, `CS_DATABASE__PASSWORD`
`-b`, `--batch-size`	Number of records to process at once	`100`	`CS_BATCH_SIZE`
`-d`, `--dry-run`	Runs without updating. Loads data but does not perform updates	None (Optional)	`CS_DRY_RUN`
`-v`, `--verbose`	Enables verbose logging	None (Optional)	`CS_VERBOSE`
`-D`, `--debug`	Enables debug output	None (Optional)	`CS_DEBUG`
`-f`, `--log-format`	Log format (`text` or `structured`)	`text`	`CS_LOG_FORMAT`
`-h`, `--help`	Displays this help message	-	-
`-V`, `--version`	Prints the version of the tool	-	-
`--decrypt`	Decrypts. Assumes source is encrypted and target is plaintext	None (Optional)

Database Details

As Migrator relies on Proxy for encryption, the database connection options should reference the CipherStash Proxy pool, not the actual Postgres database connection details.

Note

The database connection ENV variable definitions are shared with CipherStash Proxy, allowing for reuse of the shared configuration options.

How it works

Encrypting existing data requires running an UPDATE of the column to be encrypted through CipherStash Proxy.

At a high level, the process is:

1For each record in {table}
2 - SELECT {pk} {column} FROM {table};
3 - UPDATE {table} SET column = {column_value} WHERE {pk} = {pk_value};

CipherStash Proxy encrypts the {column_value} and creates any encrypted indexes required.

Examples

Encrypt `name` and `email` columns in `users` table

1cipherstash-migrator --table users --columns name=encrypted_name email=encrypted_email --database-name postgres --username postgres --password password

Specify the primary key column:

1cipherstash-migrator --table users --primary-key user_id --columns name=encrypted_name email=encrypted_email --database-name postgres --username postgres --password password

Specify multiple primary key columns (compound key):

1cipherstash-migrator --table users --primary-key user_id order_id  --columns name=encrypted_name email=encrypted_email --database-name postgres --username postgres --password password

Run as dry-run

1cipherstash-migrator --table users --primary-key user_id --columns name=encrypted_name email=encrypted_email --database-name postgres --username postgres --password password --dry-run

The --dry-run option will SELECT and load data from the table, but does execute any UPDATES

Via Docker `exec`

Assuming a container called cipherstash-proxy:

1docker exec cipherstash-proxy cipherstash-migrator --table users --columns name=encrypted_name email=encrypted_email

If the container is running with the CS_DATABASE__NAME, CS_DATABASE__USERNAME, CS_DATABASE__PASSWORD ENV variables, they will be reused by CipherStash Migrator and don't need to passed.

Decryption

Data can be decrypted from an encrypted source to a plaintext target using the --decrypt flag.

All other command libne options remain the same.

Encrypt `name` and `email` columns in `users` table

1cipherstash-migrator --decrypt --table users --columns encrypted_name=name encrypted_email=email --database-name postgres --username postgres --password password