Getting started

Encrypting your data

At this point, you have set up your example application and running Postgres traffic through the CipherStash Tandem proxy in passthrough mode. Now you will be encrypting your data and configuring Tandem to perform the encryptions and decryptions as traffic is routed through the proxy.

Migrations

CipherStash requires additional colums to be added to the database tables that you want to encrypt. These columns are used to store the encrypted data and the indexes that are used to search the encrypted data. For this example application, we have already configured the database to have these columns.

In a production system, you will need to run a migration to add these columns to your database tables. For your understanding, below are the columns we have added to the users table:

ColumnTypeDescription
__name_encryptedtextEncrypted source value for name
__name_orepublic.ore_64_8_v1Encrypted ORE index for name
__name_matchinteger[]Encrypted match index for name
__name_uniquetextEncrypted unique index for name
__email_encryptedtextEncrypted source value for email
__email_orepublic.ore_64_8_v1Encrypted ORE index for email
__email_matchinteger[]Encrypted match index for email
__email_uniquetextEncrypted unique index for name

Postgres custom type

A custom type is used for the ORE index. This type needs to be added to the database before the migration is run and is already added to the example application database to save time.

Disabling Tandem passthrough mode

Tandem is initially configured in passthrough mode. This means that it will not encrypt or decrypt any data. You need to change this setting before proceeding any further by updating the configuration file and reloading the example application. Passthrough mode can be useful to validate that the networking configuration is correct for your environment before proceeding to the next step.

Open the docker-compose.yaml file and update the CS_PASSTHROUGH environment variable to false:

1CS_PASSTHROUGH: false
2

Now you need to restart the example application:

1docker compose restart
2

Encrypting the data

Before you can use Tandem, you need to encrypt the existing data in the name and email fields. This can be done by running the following curl statement:

1curl -X POST http://localhost:3000/encrypt-data
2

The process of encrypting the data for this example application simply uses UPDATE statments to allow Tandem to encrypt the existing data via this POST request.

Validating the encryption

You can use the following command to validate the encrypted fields in the database have been populated:

1docker compose exec postgres psql -U postgres -d stash -x -c "SELECT * FROM users;"
2

The output is a bit verbose, but you should see the encrypted data in the __name_encrypted, __email_encrypted, and corresponding index columns.

Verbose output

The x flag in the psql command is used to enable expanded output mode. This will make it easier to see the encrypted data.

Updating the dataset config

The dataset configuration supports a few different modes. The default mode is plaintext-duplicate which we have been using so far. The other modes are encrypted-duplicate and encrypted.

  • plaintext-duplicate - This mode will write to all plaintext, encrypted, and index columns, while only reading from the plaintext column. This mode is useful for testing and debugging.
  • encrypted-duplicate - This mode will write to all plaintext, encrypted, and index columns, while only reading from the encrypted column. This mode is useful quality assurance practices.
  • encrypted - This mode expects all the records to be encrypted for the fields you want protected. This mode is the most secure and is recommended for production environments.

You will be using the encrypted mode for your example application. Open the config/dataset.yml file and update the mode setting for both the name and email fields to encrypted:

1tables:
2  - path: users
3    fields:
4      - name: name
5        in_place: false
6        cast_type: utf8-str
7        mode: encrypted
8        indexes:
9          - version: 1
10            kind: match
11            tokenizer:
12              kind: ngram
13              token_length: 3
14            token_filters:
15              - kind: downcase
16            k: 6
17            m: 2048
18            include_original: true
19          - version: 1
20            kind: ore
21          - version: 1
22            kind: unique
23      - name: email
24        in_place: false
25        cast_type: utf8-str
26        mode: encrypted
27        indexes:
28          - version: 1
29            kind: match
30            tokenizer:
31              kind: ngram
32              token_length: 3
33            token_filters:
34              - kind: downcase
35            k: 6
36            m: 2048
37            include_original: true
38          - version: 1
39            kind: ore
40          - version: 1
41            kind: unique
42

Now you need to push the updated config to CipherStash and restart our example application:

1stash datasets config upload --file config/dataset.yml --client-id $CS_CLIENT_ID --client-key $CS_CLIENT_KEY
2docker compose restart
3

In a production system, you will want to be more careful about how you migrate through the dataset configurations.

  • Make sure that you have a plan for how you will handle the data that is already in the database.

  • Make sure that you have a tested backup/recovery plan in place prior to encrypting the data.

    • Verify that our queries are working

      At this point, you have deployed Tandem, routed database traffic through Tandem, and encrypted your data. Now you need to verify that your application is still working as expected.

      Send a request to the http://localhost:3000/users route and you should expect the same results as before, but this time Tandem is querying the encrypted fields and decrypting the data as it is routed through the proxy.

      Removing the plaintext columns

      Now that you have verified that your application is working as expected, you can remove the plaintext columns from the database. This will ensure that the data is only stored in the encrypted columns.

      1docker compose exec postgres psql -U postgres -d stash -c "ALTER TABLE users DROP COLUMN name;"
      2docker compose exec postgres psql -U postgres -d stash -c "ALTER TABLE users DROP COLUMN email;"
      3

      Achieving encryption-in-use

      Tandem is mapping the name and email columns to the __name_encrypted and __email_encrypted columns respectively and then ordering by the encrypted columns, which is what we call encryption-in-use.

      1SELECT id, __name_encrypted, __email_encrypted FROM users ORDER BY __name_encrypted ASC;
      2

      Whereas the application is still querying the name and email columns:

      1SELECT id, name, email FROM users ORDER BY name ASC;
      2

      The results when hitting the application's /users route, which is querying the name and email columns should continue to expect the same results as before:

      1[
      2  {
      3    "id": 1,
      4    "name": "Anakin",
      5    "email": "[email protected]"
      6  },
      7  {
      8    "id": 2,
      9    "name": "Luke",
      10    "email": "[email protected]"
      11  }
      12]
      13

      If you ran the mapped Tandem query directly against the database you'd only see the encrypted data:

      1docker compose exec postgres psql -U postgres -d stash -x -c "SELECT id, __name_encrypted, __email_encrypted FROM users ORDER BY __name_encrypted ASC;"
      2

      While the unencrypted name and email columns no longer exist, your application can now decipher the plaintext data from the encrypted columns because you have successfully deployed Tandem.

      Conclusion

      This concludes the getting started guide. You have now successfully deployed Tandem and encrypted your data. You now have a solid foundation to build on and can continue to the next steps to learn more about how to use Tandem in your application.

    Previous
    Step 3 - Deploy Tandem