In this post, I will share how to post complex data and decode it in the server side in Python.

The problem

To post some simple Python dict to the server, we may use the below code:

payload = {'name': 'john smith', 'age': 20}

r = requests.post("https://httpbin.org/post", data=payload)

By default, when we use requests.post(url, data=payload) to post payload to the server. We can check the header of the HTTP request via the following command:

print(r.request.header)

The Content-Type of this HTTP request will be application/x-www-form-urlencoded by default. When the data structure is simple, it is fine to use the default options. Once there are nested structure in your Python dictionary, the server can not decode the message properly if the client is using this Content-Type, for example, when we have nested dictionary (a 2-D list):

payload = {'matrix': [[1, 2, 3], [4, 5, 6]], 'msg': 'hello'}

Solution

There are two solutions here.

Serialize data as json string

First, we can encode the complex dictionary into string using json:

# Note that payload must be json-serializable, or you will meet an error.
payload = json.dumps(payload)

r = requests.post(url, data=payload)

On the server side (suppose that we are using Flask), we can decode the string to get the original dict:

import json

from flask import request

user_request = request.form.to_dict()

# decodes the string into original dictionary
my_matrix = json.loads(user_request['matrix'])

Use application/json as Content-Type

Second, we can directly tell the server that we are sending data in json format using requests:

payload = {'matrix': [[1, 2, 3], [4, 5, 6]], 'msg': 'hello'}

r = requests.post(url, json=payload)

In this case, the request header will be something like the following:

{'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate',
'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2705',
'Content-Type': 'application/json'}

The Content-Type field will be application/json, in which we tell the server that we are sending JSON data.

On the server side (suppose you are using flask), we can retrieve that data using the following script:

from flask import request


# directly decodes the request body as JSON.
user_request = request.get_json()

Author jdhao

LastMod 2021-04-09

License CC BY-NC-ND 4.0