Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #29361

Re: Python garbage collector/memory manager behaving strangely

Show all headers | View raw

On Mon, 17 Sep 2012 10:28:34 +0800, "Jadhav, Alok"
<alok.jadhav@credit-suisse.com> declaimed the following in
gmane.comp.python.general:


> - As you have seen, the line separator is not '\n' but its '|\n'.
> Sometimes the data itself has '\n' characters in the middle of the line
> and only way to find true end of the line is that previous character
> should be a bar '|'. I was not able specify end of line using
> readlines() function, but I could do it using split() function.
> (One hack would be to readlines and combine them until I find '|\n'. is
> there a cleaner way to do this?)

	Then chunk the file... 

{PSEUDOCODE}

last = ""
while True:
	chunk = fin.read(500000)		#read 1/2 MB max
									#and attach to left-overs
	if not chunk:		#EOF, process any left overs
		processLine(last)
		break			#and exit
	
	lines = (last + chunk).split("|\n")
	last = lines[-1]					#save "partial" line

	for ln in lines[:-1]:				#loop over rest
		processLine(ln)

fin.close()
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Python garbage collector/memory manager behaving strangely Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2012-09-17 00:15 -0400

csiph-web