Spec of the Week: RFC 6920: Naming Things with Hashes

Spec of the Week: RFC 6920: Naming Things with Hashes

2023-08-22

In this age of endless data creation, an egalitarian form of addressing has become a must; not because all data is equally good, but because all data is equally bad.

For this, we need a format that not only describes the data, but describes its representation of the description of said data. (And optionally, how to aquire said data too.)

Many options exist in this recursive hellscape:

• Magnet links the famous authority-less specification of yester-decade.
• IPFS CIDs made from the most simple concepts but with more layers of abstraction than can be counted by any mortal.
• RFC 6920, a clear reference to 69 and 420, this approach combines easiness with the painful complexity of an RFC's wording and references.

One of my most favorite RFCs as of late, RFC 6920 enables me, you, anyone, to be able to represent any data not just with a Uniform Resource Identifier, but a Universal one! (Which is no longer uniform!)

What is RFC 6920

In the 1960s, content-addressable storage was a new exciting concept in the field of storing data. In the 2010s, RFC 6920 was made, combining the address with its store. Probably certainly not the first, but it is quite a nice implementation of the idea.

Using the power of URIs, we can package the hash of the data, the content type, and a server that might store and provide said data

ni://example.com/sha-256-32;f4OxZQ?ct=text/plain

To resolve it, https://example.com/.well-known/ni/sha-256-32/f4OxZQ?ct=text/plain, and then make sure the hash prefix, after base64 decoding, matches what you get!

Simple!

I'm not going into the binary and human speak-able versions. I don't want to suffer. But both are great for their uses, binary for storing in a pre-existing system that expects specific sized hashes, or human speak-able form for slowly reading to your grandma over the phone so she can finally play that cracked copy of starcraft that you've been nagging her to look at!

Why not a URN?

Uniform Resource Names are neat for naming things, however they are not neat for resolving things. If you have urn:isbn:978-1645679158, what do you do with that?

That being said, maybe a pairing would have been appropriate, a urn:example:ni:sha-256-32:f4OxZQ perhaps. Would a truncated form be okay in that case? Would the fact that multiple files can have the same hash also be okay? It's not a perfect mapping after all, just a statistically comfortable one.

Why not a CID?

CIDs like as used by Bittorrent and IPFS have a vastly different concern: How best to break a file down and reconstitute it at the other end. IPFS (as far as I know) is relatively consistent in how its breaking/unbreaking process is implemented, Bittorrent however is not so consistent, with files able to span multiple chunks, and chunks able to span multiple files. This makes an implementation go from 5 lines of work, to an entire easy afternoon of muddling through specifications.

Why not RFC 6920?

However, with prior-mentioned CIDs, you gain the ability to cryptographically address semi-consistent parts of the whole, making replication of such data over a network much easier compared to RFC 6920, which depends on its transport.

And while CDNs do support caching subsets of a whole object, there is no easy way for a RFC 6920 client to check whether their partial content is valid or not, as it uses a hash over the whole data, and not over a tree of parts.

Implementations in the wild

Hah! None so far! So watch this space. If I find any still in use, I'll update this article with them.

Hello World!

To illustrate how one could generate a ni: for a given data:

rfc6920 () {
    printf 'ni:///sha-256;'
    sha256sum -b |
    tr -cd '[:xdigit:]' |
    hex2bin |  # bring your own implementation!
    base64 |
    tr '/+' '_-' | 
    tr -d =
}

echo -n "Hello World!" | rfc6920 
# ni:///sha-256;f4OxZX_x_FO5LcGBSKHWXfwtSx-j1ncoSt3SABJtkGk

See Also

• RFC 6920

Discuss This Post

#spec-of-the-week

Made with Bear ʕ•ᴥ•ʔ