Transit format: An interactive tutorial - custom types and caching (part 2)

Sep 22, 2016 • Yehonathan Sharvit

What is Transit?

In our previous article, we introduced Transit.

Basically, Transit is a format and set of libraries for conveying values between applications written in different programming languages.

Transit provides:

• a set of basic elements
• a set of extension elements for representing typed values.

In this article, we are going to show:

• how to write readers and writers to deal with custom types
• how Transit supports compression via caching of repeated elements, e.g., map keys, that can significantly reduce payload size and decoding time, as well as memory in the resulting application representation.

Read more about Transit.

Code examples please

All the above is very very abstract. Let’s code some examples. As usual, the code snippets are interactive: feel free to modify the code in order to get a better feeling of the concepts.

In this article, we are going to use transit-cljs to illustrate Transit format.

First, let’s require transit:

xxxxxxxxxx

(ns my.transit

  (:require [cognitect.transit :as t]))

the evaluation will appear here (soon)...

One of the biggest drawbacks of JSON as a data format is the lack of extensibility. Transit allows users to customize both reading and writing.

Let’s say that we have a type Rational for rational numbers. Something like this:

xxxxxxxxxx

(defrecord Rational [numerator denominator])

xxxxxxxxxx

the evaluation will appear here (soon)...

Writer - customization

Now, let’s see how to write a Transit writer that serializes the Rational values:

xxxxxxxxxx

(deftype ^:no-doc RationalHandler []

  Object

  (tag [_ v] "rational")

  (rep [_ v] #js [(:numerator v) (:denominator v)])

  (stringRep [_ v] nil))

​

(def rational-handler (RationalHandler.))

(def rational-writer (t/writer :json {:handlers {Rational rational-handler}}))

xxxxxxxxxx

the evaluation will appear here (soon)...

And here is the represantation of a Rational:

xxxxxxxxxx

(t/write rational-writer [(Rational. 2 3)])

xxxxxxxxxx

the evaluation will appear here (soon)...

Reader - customization

Now, let’s see how to write a Transit reader for Rational values:

xxxxxxxxxx

(def rational-reader (t/reader :json

                               {:handlers

                                {:rational (fn [[a b]] (->Rational a b))}}))

xxxxxxxxxx

the evaluation will appear here (soon)...

Let’s read an array of Rational values:

xxxxxxxxxx

(t/read rational-reader "[ [\"~#rational\", [3,4]], [\"~#rational\", [2,5]]]")

xxxxxxxxxx

the evaluation will appear here (soon)...

Now, let’s check that we have closed the loop properly:

xxxxxxxxxx

(def y [(Rational. 2 3)])

(= y (t/read rational-reader (t/write rational-writer y)))

xxxxxxxxxx

the evaluation will appear here (soon)...

And the other direction:

xxxxxxxxxx

(def x "[\"~#rational\",[3,4]]")

(= x (t/write rational-writer (t/read rational-reader x)))

xxxxxxxxxx

the evaluation will appear here (soon)...

Caching

In Transit, repeated keys in a map are cached. Therefore the payload doesn’t depend much on the length of the keys.

Let’s create a JSON reader and writer:

xxxxxxxxxx

(def r (t/reader :json))

(def w (t/writer :json))

xxxxxxxxxx

the evaluation will appear here (soon)...

Let’s see how repeated keys in maps are encoded:

xxxxxxxxxx

(def some-ids [{:id 1} {:id 2} {:id 3}])

​

(t/write w some-ids)

xxxxxxxxxx

the evaluation will appear here (soon)...

Obvioulsy, we can read it back:

xxxxxxxxxx

(t/read r (t/write w some-ids))

xxxxxxxxxx

the evaluation will appear here (soon)...

The length of the encoded string almost doesn’t increase, when the length of the keys increases.

First with a short key:

xxxxxxxxxx

(def n 100)

(def key :abc)

(def many-ids (for [x (range n)] {key n}))

​

(count (t/write w many-ids))

xxxxxxxxxx

the evaluation will appear here (soon)...

And now with a long key:

xxxxxxxxxx

(def n 100)

(def key :abcdefghijklmnop)

(def many-ids (for [x (range n)] {key n}))

​

(count (t/write w many-ids))

xxxxxxxxxx

the evaluation will appear here (soon)...

Increase the length of the key, and see how the length of the encoded string stays almost the same.

Pretty cool. Right?

Clojure[script] rocks!

to stay up-to-date with the coolest interactive articles around the world.

Discover more cool interactive articles about javascript, clojure[script], python, ruby, scheme, c++ and even brainfuck!

Give Klipse a Github star to express how much you appreciate Code Interactivity.

Feel free to email me [email protected] for getting practical tips and tricks in writing your first interactive blog post.