Python json.loads shows ValueError: Extra data

Python json.loads shows ValueError: Extra data

As you can see in the following example, json.loads (and json.load) does not decode multiple json object.

>>> json.loads({})
{}
>>> json.loads({}{}) # == json.loads(json.dumps({}) + json.dumps({}))
Traceback (most recent call last):
  File <stdin>, line 1, in <module>
  File C:Python27libjson__init__.py, line 338, in loads
    return _default_decoder.decode(s)
  File C:Python27libjsondecoder.py, line 368, in decode
    raise ValueError(errmsg(Extra data, s, end, len(s)))
ValueError: Extra data: line 1 column 3 - line 1 column 5 (char 2 - 4)

If you want to dump multiple dictionaries, wrap them in a list, dump the list (instead of dumping dictionaries multiple times)

>>> dict1 = {}
>>> dict2 = {}
>>> json.dumps([dict1, dict2])
[{}, {}]
>>> json.loads(json.dumps([dict1, dict2]))
[{}, {}]

You can just read from a file, jsonifying each line as you go:

tweets = []
for line in open(tweets.json, r):
    tweets.append(json.loads(line))

This avoids storing intermediate python objects. As long as your write one full tweet per append() call, this should work.

Python json.loads shows ValueError: Extra data

I came across this because I was trying to load a JSON file dumped from MongoDB. It was giving me an error

JSONDecodeError: Extra data: line 2 column 1

The MongoDB JSON dump has one object per line, so what worked for me is:

import json
data = [json.loads(line) for line in open(data.json, r)]