Behind the scenes of the Nightscout API

Like peanut butter and chocolate, it’s awesome when two great things come together. This post originated as part of the Hack Together: Microsoft Graph and .NET. You can get more details and participate by at the registration link. Enjoy!

Developers today are building ever more complex apps that, increasingly, apply capabilities from a wide range of services. One common practice is weaving together the capabilities of multiple different apps to create something new and unique. In this scenario, let’s call it an example of IoT predictive maintenance, we wanted to build an application that could remind someone to check their glucose monitor by putting a reminder on their outlook calendar. While this scenario is rather simple, it requires integration with Microsoft Graph, the Glucose monitor, and user authentication for both systems, which use different security schemes. You can see how even in this simple example, things get complex rather quickly!

For developers, APIs are the engine that drives applications that pull data from multiple services. OpenAPI is the industry standard specification that is a developer’s best friend when it comes to integrating platforms. To build this app, we need to use APIs for both Microsoft Graph and the Glucose monitor. But despite being a standard, there are still many ways to express the same thing. Consider that for each operation the developer needs to specify the HTTP method, headers, the path, query parameters, return codes, valid values, optional and required parameters, etc. This detail must be provided both the request and response. Then, most importantly, the developer must describe the information–the types–that their service provides.

Given the flexibility of OpenAPI, it’s easy to see how different developers–even within the same organization–can create specifications that are similar, but different. Because the OpenAPI specification is the “contract” for the service, how the API is declared can significantly affect downstream toolchains. For example, some of the initial design decisions of the Nightscout API made it difficult to apply the code generation capabilities of Kiota](https://microsoft.github.io/kiota/get-started/). We chose to “refactor” the original specification using TypeSpec (née Cadl) to feed a more precise API definition into our client code generator, Kiota. The complete set of code for this example is located in the Nightscout Description repository in the APIPatterns organization in GitHub.

To become more familiar with TypeSpec, please check out the TypeSpec Docs and the TypeSpec playground. The following blogs, The Value of TypeSpec in designing APIs, Describing a real API using TypeSpec: The Moostodon Story shows another example of using TypeSpec to describe APIs and Kiota to generate client libraries.

This scenario requires access data from Microsoft Graph and a glucose monitor. For our team, how to get data from Microsoft Graph is well known and something we do everyday. Glucose monitoring is new to us, so we went looking for an OpenAPI description… and found one! We immediately ran the OpenAPI description through Kiota to create our dotNet client, and were “bitten” by some design decisions made by the original developer of the Nightscout API.

The Nightscout API has a discriminator as a required parameter on the path. The issue is that OpenAPI doesn’t support using a path parameter as a discriminator. As a consequence, Kiota isn’t able to properly generate client libraries. The OpenAPI was likely described using a discriminator because the API supports many different document types, each with the same API capabilities. Writing OpenAPI to fully specify endpoints, each with the same capabilities, requires duplicating all of the operations for each document type. Authoring an API without discriminators is error prone, time consuming, and results in a large document. Enter Cadl, err… TypeSpec!

TypeSpec is an open-source language inspired by TypeScript that’s designed to make the authoring of APIs easier and less cumbersome. Originally called Cadl (pronounced “cattle”), the team is in the process of renaming the project to TypeSpec to give it a more accurate and descriptive name. Because TypeSpec is a language, TypeSpec has better capabilities to reuse API designs and separating concerns, making the generation of complex OpenAPI documents incredibly easy. However, we still like cow puns, so, let’s round up the herd and do some refactoring!

Create the models

We used the original Nightscout API as the basis for our refactoring, and expressed its APIs using TypeSpec. We start by defining the models (also known as types) that are used by the service. In our example, the models are located in the ./spec/models folder. We capture common properties in a “base” model, and then extend it for specific document types. The common properties for Nightscout documents are factored out into the DocumentBase.cadl file, which is imported–just like code–when we model each individual document type. For example, here’s the first part of the Food.cadl file:

import "./DocumentBase.cadl";

@doc("Nutritional values of food")
model Food extends DocumentBase {
@doc("food, quickpick")
food?: string;

@doc("Name for a group of related records")
category?: string;

Define a reusable interface

The next step was to address the main issue of getting rid of the discriminator in the path. In the original OpenAPI specification, the result is defined as oneOf a specific type, for example, Food. The {collection} discriminator in the path determines which set of documents to query and, as a result, the type that is returned. Essentially, we need to remove the ambiguity in the path and change /{collection}/{identifier} to /food/{identifier}. If we look closely, we realize the reason a discriminator in the path could be used is because the operations on each collection are identical. In TypeSpec, we can group operations into an interface, then reuse it across multiple endpoints. Being able to specify the exact shape of multiple endpoints in a single definition, and then being able to apply that definition to multiple endpoints, is a powerful technique for driving standardization and consistency across a broad API surface area. In the Nightscout example, the operations on collections, are captured in the ./spec/documentCollection.cadl file. Here, a single DocumentCollection interface is defined that contains all the CRUD operations on collections. The following example shows how PATCH is expressed. The <T> represents a template, and is replaced with a specific model type when the interface is used.

interface DocumentCollection<T> {
...
   @summary("PATCH: Partially updates document in the collection")

   @patch 
   op patch(
           @header("If-Modified-Since") ifModifiedSince: string;
           @path identifier: string,
       ): createdDocument | updatedDocument | BadRequestFailedResponse | UnauthenticatedFailedResponse | UnauthorizedFailedResponse | NotFoundFailedResponse | GoneFailedResponse | StatusResponse<412> | StatusResponse<422>; 

All of the endpoints return a JSON object that contains a status property that duplicates the HTTP status code. The actual response body is in the results property. How responses are modeled is captured in the ./spec/responses.cadl file. In similar fashion, a common model is defined StatusResponse<Status>, and then “instances” of those models are created, which can have additional properties. In the previous example, the PATCH operation (op patch), returns one of these declared responses, for example, createdDocument OR (|), BadRequestFailedResponse, OR StatusResponse<422>.

Not only can we pass in a specific HTTP return code if necessary, but we’re able to use different response types to accurately model service behavior. When attempting to create a document, if it exists, the service returns a different response body. A developer must carefully read the OpenAPI document to understand they must evaluate the response code, 200 or 201 to determine if a document is created or updated. In TypeSpec, it’s easier to indicate creation versus update, and is modeled as follows:

model createdDocument is StatusResponse<201> {
   @header("Location") location: string;
   identifier: string; 
   lastModified: int64;
}

model updatedDocument is StatusResponse<200> {
   identifier: string; 
   isDeduplication?: boolean;
   deduplicatedIdentifier?: string;
}

It’s fairly uncommon for APIs to describe the 200 and 201 response as two distinct response bodies, however, it’s a perfectly valid API design. Kiota doesn’t have a great solution for this particular scenario at the moment, but with the use of the AdditionalData property, all of the returned information can be accessed.

Now, we have our models, a common interface, and a standard set of responses. All we need to do is declare the endpoints in our API. We use the @route decorator to establish the path segment. We get the operations by declaring our route is “decorating” new interface that extends our common DocumentCollection. Instead of a discriminator, the kind of document collection accessed is explicitly expressed through the template parameter, <T> . For example, here’s the endpoint for Food:

       @route("food")
       interface foodDocument extends DocumentCollection<Food> {}

Work with multiple versions of an API

We also had some other interesting discoveries, one of which was that not all of the capability that we need is in the V3 API. We were easily able to include select operations from V2, and keeping them isolated in their own namespace. In TypeSpec, namespaces work much like they do in code, and provide the same organization and isolation mechanism for APIs. When we generate client code using Kiota, the result is a single library that includes operations from both versions of the API. Having a single library that works with both versions API makes is easy for developers to write code that uses the service.

@service({
   title: "Nightscout API",
   version: "3.0.3"
})
@useAuth(AccessToken | JwtToken)
@server("/api","default endpoint")
namespace nightscout {
   @route("v2")
   namespace v2 {
       @route("properties")
       op getProperties(): Properties;

       @route("properties/{properties}")
       op getSelectedProperties(@path properties: string[]): Properties;
   }
   @route("v3")
   namespace v3 {
       @route("devicestatus")

Separate concerns using “sidecars”

The original OpenAPI description contains lots of usage documentation about the API and its operations–which is fantastic. However, it’s common for many people to work on an API. For example, a developer creates the operation definitions, while a product manager might write the documentation. TypeSpec, through a concept called sidecars, facilitates a clean separation of concerns. In our example, we factored out the documentation into a distinct file, ./spec/docs.cadl. Using the @@ construct, we were able to index into another Cadl file. In our example, we added documentation to the read operations of our common interface:

@@doc(DocumentCollection.read, """
Basically this operation looks for a document matching the `identifier` field returning 200 or 404 HTTP status code...

Generate OpenAPI and client libraries

When complete, our main.cadl file is a concise 66 lines of code, making it easy for a developer to quickly understand the entirety of an API. Taking into account the models, common interfaces, and the documentation, the entire TypeSpec totals around 500 lines of code. When rendered as OpenAPI, the resulting specification is over 5,000 lines of code! By generating the OpenAPI from TypeSpec, we get a specification that conforms to our practices and guidelines. Using TypeSpec to codify guidelines, practices, and patterns to generate cleaner, more consistent specifications, is exactly what the Microsoft Graph and Azure SDK teams are doing!

Now that we have a newly constructed OpenAPI, creating a client library is a single command away with Kiota. You can install the Kiota command line tool using the instructions at https://aka.ms/get/kiota. Using Kiota developers can generate client libraries in C#, Go, Java, TypeScript, Python, and Ruby.

kiota generate -l <language> -o <output path> -d https://raw.githubusercontent.com/apidescriptions/nightscout/main/spec/cadl-output/%40cadl-lang/openapi3/openapi.yaml

With this client library, you get a strongly typed experience for accessing the API with all the capabilities we built to make Microsoft 365 applications resilient and efficient.

A final shout out

As you can see from our previous TypeSpec blog posts (“Moostodon” and “Building a Reusable Library”), we’ve been having fun with TypeSpec and Kiota. The combination of the two is proving to be a powerful and elegant way to bring the best developer experience to the authoring OpenAPI specifications and quickly generating client code. We know you want to be “herd”, so let us know your thoughts and take a moment to try out TypeSpec. Check out the code in the APIPatterns organization, and become part of the “movement!” Sorry, we miss ‘Cadl’ because we just can’t resist bad cow puns.

And finally, there were many other folks that contributed to this demo and blog who deserve recognition: Vincent Biret, Mike Kistler, Sébastien Levert, and Rabeb Othmani. And of course, Scott Hanselman, who came up with some of the original ideas and prototype. Thanks for all your help!