5

Better JSON Encoding

 2 years ago
source link: https://bbengfort.github.io/2016/01/better-json-encoding/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Better JSON Encoding

January 19, 2016 · 2 min · Benjamin Bengfort

The topic of the day is a simple one: JSON serialization. Here is my question, if you have a data structure like this:

import json
import datetime

data = {
    "now": datetime.datetime.now(),
    "range": xrange(42),
}

Why can’t you do something as simple as: print json.dumps(data)? These are simple Python datetypes from the standard library. Granted serializing a datetime might have some complications, but JSON does have a datetime specification. Moreover, a generator is just an iterable, which can be put into memory as a list, which is exactly the kind of thing that JSON likes to serialize. It feels like this should just work. Luckily, there is a solution to the problem as shown in the Gist below:

import json import datetime

JSON_DATETIME = "%Y-%m-%dT%H:%M:%S.%fZ"

class Encoder(json.JSONEncoder): """ More advancded JSON encoding utility. Attempts to leverage an object's serialize method if it has one, otherwise looks for an encode_type method on the encoder itself that specifies how to encode. """

def encode_generator(self, obj): """ Converts a generator into a list """ return list(obj)

def encode_datetime(self, obj): """ Converts a datetime object into epoch time. """ # Handle timezone aware datetime objects if obj.tzinfo is not None and obj.utcoffset() is not None: obj = obj.replace(tzinfo=None) - obj.utcoffset() return obj.strftime(JSON_DATETIME)

def default(self, obj): """ Perform encoding of complex objects. """ try: return super(SmoakEncoder, self).default(obj) except TypeError: # If object has a serialize method, return that. if hasattr(obj, 'serialize'): return obj.serialize()

# Look for an encoding method on the Encoder method = "encode_%s" % obj.__class__.__name__ if hasattr(self, method): method = getattr(self, method) return method(obj)

# Not sure what is going on if the above two methods didn't work raise SerializationError( "Could not encode type '{0}' using {1}\n" "Either add a serialze method to the object, or add an " "encode_{0} method to {1}".format( obj.__class__.__name__, self.__class__.__name__ ) )

if __name__ == "__main__": # Usage of Encoder print json.dumps(data, cls=Encoder)

Ok, so basically this encoder replaces the default encoding mechanism by trying first, and if that doesn’t work follows the following strategy:

  1. Check if the object has a serialize method; if so, return the call to that.
  2. Check if the encoder has a encode_type method, where “type” is the type of the object, and if so, return a call to that. Note that this encoder already has two special encodings - one for datetime, and the other for a generator.
  3. Wave the white flag; encoding isn’t possible but it will tell you exactly how to remedy the situation and not just yell at you for trying to encode something impossible.

So how do you use this? Well you can create complex objects like:


class Student(object):

    def __init__(self, name, enrolled):
        self.name = name         # Should be a string
        self.enrolled = enrolled # Should be a datetime

    def serialize(self):
        return {
            "name": self.name,
            "enrolled": self.enrolled,
        }


class Course(object):

    def __init__(self, students):
        self.students = students # Should be a list of students

    def serialize(self):
        for student in self.students:
            yield student

And boom, you can now serialize them with the JSON encoder — json.dumps(course, cls=Encoder)! If you have other types that you don’t have direct access to, for example, UUID (part of the Python standard library), then simply extend the encoder and add a encode_UUID method.

Note that extending the json.JSONDecoder is a bit more complicated, but you could do it along the same lines as the encoder methodology.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK