Sunday 15 April 2012

json - MemoryError opening 5GB file using Python -



json - MemoryError opening 5GB file using Python -

i getting memoryerror after reading 5gb json file line line in python.here code, using job. new python solution code:

with open("d:/2.5gb.json",'r') json_file: line in json_file: ret= line.find('435355ebe9f768da84e60340266c18cexx') if ret<>-1: print "found" exit()

answer http://ikigomu.com/?p=1

with open("my_large_file.txt", "r") f: line in f: (code here)

the way you're doing exhausting memory limit of python trying load whole thing @ once. works because loads 1 line in @ time.

json python-2.7

No comments:

Post a Comment