I am trying to read a 59 GB file and break it to a number of new files according to some id found in the begging of each line. I am running the code below which breaks with a memory error after producing 45GB files. The system memory remains in very low level during all time and suddenly creates a after the code runs for about 2 hours. I have 16GB ram. Am I using buffering wrongly? Any idea? I am trying to read a 59 GB file and break it t