James Sadler
James Sadler
July 16, 2021

TypeScript and excellent developer experience (part 1)

How smart use of the type-manipulation features of TypeScript can deliver a first class developer experience to users of your library

CipherStash - TypeScript and excellent developer experience (part 1)

Intro

This article is part 1 of a short series on how CipherStash is using TypeScript to provide a high quality developer experience for our new client library, StashJS.

The success of TypeScript has set a high baseline for a good developer experience in the JavaScript ecosystem these days. Even marking up some fairly straightforward JavaScript code with some simple types can provide a great experience for developers.

But the advanced type manipulation that TypeScript provides helps a determined library author go the extra mile and produce library code with compile time superpowers, code hints, and auto-complete that seem like magic, but add so much value.

At CipherStash we want to provide our customers and their developers the best experience that we can — and it turns out that TypeScript lets us push that quite far!

StashJS is the new CipherStash client

CipherStash is new kind of data store where data is stored and queryable in an always-encrypted form — not only at rest like with a traditional database. At no point does CipherStash ever see a plaintext of customer data, even when responding to queries. The CipherStash platform does not even have access to the necessary cryptographic keys to decrypt the data.

CipherStash exposes an API not unlike a traditional document store. A developer can create "collections" which will contain records/documents, and the collections can be queried. Queries currently support equality, comparison operations (less than, greater than, between, etc) and full text search.

In order for CipherStash to support the desired query operations, the developer must define mappings that tell the CipherStash client how to index their data, which in turn defines which query operators are available.

It turns out that TypeScript's type system is sufficiently powerful to express the chain of type derivations, all the way from a developer-defined record type to the mappings that can be defined on fields of that type, to the permitted query operations supported by a collection of that type.

Importantly, this ability to enforce strong type guarantees does not depend on code generation or third party tools — it is all achievable with vanilla TypeScript.

The StashJS module lets you define and query a CipherStash collection

Hovering your cursor over the identifiers in the code samples will reveal type information. Try it!

Here is a here is an example of how we define and query a CipherStash collection.

Firstly, we need to import the the basics from @cipherstash/stashjs.

ts
import {
Stash, CollectionSchema,
downcase, standard
} from "@cipherstash/stashjs"

Then we need to define our record type. This is developer-defined. Any type that can be serialized to JSON will do.

ts
type Employee = {
id?: string,
name: string,
email: string,
position: string,
salary: bigint
}

Next, we define a schema for a collection that will ultimately store Employee records. A collection schema consists of a name for the schema, plus all of the mappings that define how a collection can be searched.

ts
const employeeSchema = CollectionSchema.define<Employee>("employees").indexedWith(
mapping => ({
idxEmail: mapping.Exact("email"),
idxSalary: mapping.Range("salary"),
idxNameAndPosition: mapping.Match(["name", "position"], {
tokenFilters: [ downcase ],
tokenizer: standard
})
})
)

Hover your cursor over the idxEmail field. You will see a pop up that shows the type.

The type is ExactMapping<Employee, "email"> and "email" looks like a string but it is actually a type — the type representing a field that must exist.

Take a look at what happens when we try to define a mapping for a field that does not exist:

ts
const employeeSchema = CollectionSchema.define<Employee>("employees").indexedWith(
mapping => ({
idxEmail: mapping.Exact("fieldThatDoesNotExist"),
Argument of type '"fieldThatDoesNotExist"' is not assignable to parameter of type 'FieldOfType<Employee, ExactMappingFieldType>'.2345Argument of type '"fieldThatDoesNotExist"' is not assignable to parameter of type 'FieldOfType<Employee, ExactMappingFieldType>'.
})
)

The compilation error is informative:

FieldOfType is providing the static type checking of the field names. We will take a closer look at how that works in the next part of this blog series. For now, the key takeaway is that the compiler can prevent a class of error: namely that it is impossible to define a mapping for a field that does not exist.

We've looked at how to define a collection schema, so the next step is connect to CipherStash and create the collection.

ts
const stash = await Stash.connect()
const employees = await stash.createCollection(employeeSchema)

Earlier in this article we talked about the idea of a "chain of type derivations". You can see that employees is a Collection defined on the Employee record type and its mappings — hover your cursor over the identifier to see for yourself. The user-defined Employee type and mappings have been propagated to the employees collection.

Let's insert some records into our collection:

ts
await employees.put({
name: "Ada Lovelace",
position: "Chief Executive Officer (CEO)",
email: "ada@security4u.example",
salary: 350000n
})
 
await employees.put({
name: "Grace Hopper",
position: "Chief Science Officer (CSO)",
email: "grace@security4u.example",
salary: 225000n
})
 
await employees.put({
name: "Joan Clark",
position: "Chief Information Security Officer (CISO)",
email: "joan@security4u.example",
salary: 250000n
})

There is nothing particularly interesting here from a types perspective. Suffice it to say that the put method on the collection will only accept an Employee type as an argument.

The real benefits happen at query time. Queries are type checked against a query DSL type that is derived from the record type, in this case Employee and the mappings defined on it.

ts
let queryResult = await employees.query(
employee => employee.idxSalary.between(240000n, 400000n)
)

Below we can see the auto-complete suggestions. The completion options are the indices derived from the user-defined mappings.

ts
let queryResult = await employees.query(
employee => employee.idx
                          
)
 
 

After choosing an index, we see auto-complete options for the query operations that are valid for that kind of mapping.

ts
let queryResult = await employees.query(
employee => employee.idxSalary.l
                                  
)
 

And because the idxSalary is an index defined on a bigint field, it is a type error to pass any other type as an argument to an operation.

ts
let queryResult = await employees.query(
employee => employee.idxSalary.lt("ten thousand")
Argument of type 'string' is not assignable to parameter of type 'bigint & RangeMappingFieldType'.2345Argument of type 'string' is not assignable to parameter of type 'bigint & RangeMappingFieldType'.
)

What's next

We've looked at what is possible to achieve in API built with TypeScript. In StashJS we:

  1. Began with a user-defined record type
  2. In terms of the record type, we defined a collection schema with mappings
  3. A statically-typed query DSL type was implicitly generated from those mappings for free
  4. Queries on the collection are statically checked against the query DSL

In the next article, we'll take a deep dive on our type manipulation utilities that make all of the above possible.

Links

About the Author

James Sadler
James Sadler
CTO

James is an experienced software engineer and consultant with 20 years of experience across a variety of industries including banking, finance, logistics and telecoms in the UK and Australia. As CTO, James takes pride in delivering outsized outcomes with small highly skilled teams and a focus on precision, robustness and fast iteration. He studied Computer Science and is always on the lookout for tooling that advances the state of the art in both developer productivity and correctness.

Latest Posts

View all articles
Linting your GitHub Actions
Engineering

Linting your GitHub Actions

Your infrastructure-as-code is still code, and all that YAML needs to be checked for correctness. So does ours, and we did something about it.

Matt Palmer
Matt Palmer
November 25, 2021
3 security improvements databases can learn from APIs
Product

3 security improvements databases can learn from APIs

It turns out there’s heaps we can learn from API security improvements and apply to databases. Here are the top 3!

Lindsay Holmwood
Lindsay Holmwood
November 18, 2021
Cryptographic Failures is now #2 on the OWASP Top 10
Radar

Cryptographic Failures is now #2 on the OWASP Top 10

The OWASP Top 10 has recently been updated, and it has recognised Cryptographic Failures as the #2 vulnerability category. Here's how CipherStash can help.

Lindsay Holmwood
Lindsay Holmwood
October 13, 2021