JSON (stands for "JavaScript Object Notation") is a text-based format which facilitates data interchange between diverse applications. For example, an application written in C++ running on Windows can easily exchange JSON data with an application written in python and running on Linux. Its simplicity and flexibility has led to widespread usage in recent years, especially in preference to earlier XML-based formats.

There are libraries and toolkits available for parsing and generating JSON from almost any language and environment. This article concentrates on methods and issues arising from processing JSON using python.

Some JSON Samples

The most common JSON entity that you will encounter is an object: a set of key-value mappings in the format shown below.

person.json:

        {
  "firstName": "Alice",
  "lastName": "Hall",
  "age": 35
}

Here is how you can represent an array of objects. In this representation, each item of the array is an object. The following is a sample of salaries of baseball players.

salaries.json:

        [ {
  "year" : 1985,
  "teamId" : "ATL",
  "leagueId" : "NL",
  "playerId" : "barkele01",
  "salary" : 870000
}, {
  "year" : 1985,
  "teamId" : "ATL",
  "leagueId" : "NL",
  "playerId" : "bedrost01",
  "salary" : 550000
} ]

Of course, you can represent an array of scalars too. It looks like this:

        [
    "hello",
    "world",
    35
]

Parsing JSON in Python

Python provides the json module which can be used to both parse JSON, as well as generate JSON from python objects and lists.

The following code snippet shows how to open a JSON file and load the data into a variable.

        import json

with open('sample.json', 'r') as fp:
    obj = json.load(fp)

When you have a string containing the JSON data, you can convert it to a python object (or list) with the following:

        obj = json.loads("""{
  "firstName": "Alice",
  "lastName": "Hall",
  "age": 35
}""")

To parse a JSON URL, you can create a URL object using urllib2 and use json.load() as before.

        import urllib2, json
url = urllib2.urlopen('http://site.com/sample.json')
obj = json.load(url)

Handling Errors

When the JSON has errors, you will get a ValueError. You can handle it and take corrective action if required.

        try:
    obj = json.loads("""{
    "firstName": "Alice",
    "lastName: "Hall",
    "age": 35
    }""")
except ValueError:
    print("error loading JSON")

Parsing JSON From the Command Line

Sometimes, it is useful to parse JSON using the python command line, perhaps to check for errors or to obtain nicely indented output.

        cat glossary.json
# prints
{"glossary": {"GlossDiv": {"GlossList": {"GlossEntry": {"GlossDef": {"GlossSeeAlso": ["GML", "XML"], "para": "A meta-markup language, used to create markup languages such as DocBook."}, "GlossSee": "markup", "Acronym": "SGML", "GlossTerm": "Standard Generalized Markup Language", "Abbrev": "ISO 8879:1986", "SortAs": "SGML", "ID": "SGML"}}, "title": "S"}, "title": "example glossary"}}

To obtain indented output from the above JSON file, you can do the following:

        python -mjson.tool glossary.json
# prints
{
    "glossary": {
        "GlossDiv": {
            "GlossList": {
                "GlossEntry": {
                    "Abbrev": "ISO 8879:1986",
                    "Acronym": "SGML",
                    "GlossDef": {
                        "GlossSeeAlso": [
                            "GML",
                            "XML"
                        ],
                        "para": "A meta-markup language, used to create markup languages such as DocBook."
                    },
                    "GlossSee": "markup",
                    "GlossTerm": "Standard Generalized Markup Language",
                    "ID": "SGML",
                    "SortAs": "SGML"
                }
            },
            "title": "S"
        },
        "title": "example glossary"
    }
}

And here is how you can load the JSON object into python and extract only what you need.

        python -c 'import json; fp = open("glossary.json", "r"); obj = json.load(fp); fp.close(); print(obj["glossary"]["title"]')
# prints
example glossary

Accessing the Data

Once you have loaded the JSON data into a python variable, you can access the data as you would any python dict (or list as the case may be). For example, the above JSON data can be accessed as follows:

        firstName = obj["firstName"]
lastName = obj["Hall"]
age = obj["age"]

Data Types

Data types are automatically determined from the data. Note that age is parsed as an integer.

        print(type(obj["firstName"]), type(obj["lastName"]), type(obj["age"]))
# prints
<type 'unicode'> <type 'unicode'> <type 'int'>

The following conversion table is used to convert from JSON to python.

JSON to Python Type Mapping

Parsing JSON Using a Custom Class

By default, a JSON object is parsed into a python dict. Sometimes you may have the need to automatically create an object of your own class from the JSON data. You can do that by specifying an object_hook function which handles the conversion. The following example shows how.

Here is a custom class representing a Person.

        class Person:
    def __init__(self, firstName, lastName, age):
        self.firstName = firstName
        self.lastName = lastName
        self.age = age

    def __str__(self):
        return '{{"firstName" = "{0}","lastName" = "{1}", "age" = {2}}}'.format(self.firstName, self.lastName, self.age)

An instance of this class is created by passing the required arguments as follows:

        person = Person("Crystal", "Newell", 27)

To use this class to create instances when parsing JSON, you need an object_hook function defined as follows: The function receives a python dict and returns an object of the correct class.

        def obj_creator(d):
    return Person(d['firstName'], d['lastName'], d['age'])

You can now use this object_hook function when invoking the JSON parser.

        with open('sample.json', 'r') as fp:
    obj = json.load(fp, object_hook = obj_creator)

print(obj)
# prints
{"firstName" = "Alice","lastName" = "Hall", "age" = 35}

Examples of JSON Usage

JSON is extremely popular nowadays. Many websites and SaaS (Software As A Service) applications offer JSON output which can be consumed directly by applications. Some of the publicly available ones include:

  • StackOverflow/StackExchange. Here is a URL which returns a list of questions in JSON format.
  • GitHub offers a JSON api at https://developer.github.com/v3/.
  • And here is the Flickr API: https://developer.yahoo.com/flickr/.

If you're looking for more examples on how to put it to good use, check out this guide to building a social media bot using Python.

Are you using JSON to consume or provide services? And are you using python in your technology stack? Do explain in the comments below.