Not able to store data to dictionary because of memory limitation

Path	csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<ramp99@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.004
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'memory.': 0.05; 'python': 0.08; 'manipulate': 0.09; 'received:209.85.160.174': 0.09; 'received:mail-gy0-f174.google.com': 0.09; 'splitting': 0.09; 'storing': 0.09; 'subject:Not': 0.09; 'cursor': 0.16; 'dictionaries': 0.16; 'dictionary,': 0.16; 'subject:limitation': 0.16; 'subject:memory': 0.16; 'alternate': 0.19; 'stick': 0.19; 'issue.': 0.19; 'memory': 0.21; 'subject:data': 0.22; 'parse': 0.23; 'all,': 0.28; 'message-id:@mail.gmail.com': 0.28; 'problem': 0.29; 'efficiently': 0.30; 'thanks': 0.31; 'remaining': 0.32; 'to:addr:python-list': 0.34; 'instead': 0.34; 'received:209.85.160': 0.34; 'there': 0.34; 'stopped': 0.35; 'option': 0.37; 'disk': 0.37; 'facing': 0.37; 'some': 0.37; 'received:google.com': 0.38; 'received:209.85': 0.38; 'data': 0.39; 'to:addr:python.org': 0.39; 'received:209': 0.40; 'your': 0.60; 'share': 0.68; '1.99': 0.84; 'subject:able': 0.84; 'subject:store': 0.84
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=8LuU9yyI7fpbAo+Hrpegnycl8WTMDdtyCKCTEP8pRV8=; b=fCt6hTAV8QjfiPwG52+F/XrSuaan3GQ76vD6Vj+FZd6HYSEWkFDFvPF2rKPNVWkcGq iuuhJUMRoaoCDrM3ci6E1RjZUZrLK+kdckrqtkSte4iCUth/ps7u5MEEaw/N6Ay3FTFm Z02SJSkj6MiLRtlvp558T+bTKHbrU3T/CJgi4=
MIME-Version	1.0
Date	Wed, 6 Jul 2011 13:19:36 +0530
Subject	Not able to store data to dictionary because of memory limitation
From	Rama Rao Polneni <ramp99@gmail.com>
To	python-list@python.org
Content-Type	text/plain; charset=ISO-8859-1
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.12
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.689.1309938580.1164.python-list@python.org> (permalink)
Lines	22
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1309938580 news.xs4all.nl 21746 [2001:888:2000:d::a6]:42531
X-Complaints-To	abuse@xs4all.nl
Xref	x330-a1.tempe.blueboxinc.net comp.lang.python:8914

Show key headers only | View raw

Hi All,

I am facing a problem when I am storing cursor fetched(Oracle 10G)
data in to a dictionary.
As I don't have option to manipulate data in oracle10G, I had to stick
to python to parse the data for some metrics.

After storing 1.99 GB data in to the dictionary, python stopped to
store the remaining data in to dictionary.

Memory utilization is 26 GB/34GB. That means, still lot memory is left
as unutilized.

Can please share your experices/ideas to resolve this issue.
Is this prople mbecasue of large memory utlization.
Is there any alternate solution to resolve this issue. Like splitting
the dictionaries or writing the data to hard disk instead of writing
to memory.
How efficiently we can use  memory when we are going for dictionaries.

Thanks in advacne,
Rama

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Not able to store data to dictionary because of memory limitation Rama Rao Polneni <ramp99@gmail.com> - 2011-07-06 13:19 +0530
  Re: Not able to store data to dictionary because of memory limitation Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2011-07-06 10:35 +0200
    Re: Not able to store data to dictionary because of memory limitation Rama Rao Polneni <ramp99@gmail.com> - 2011-07-31 14:08 +0530
    Re: Not able to store data to dictionary because of memory limitation Chris Angelico <rosuav@gmail.com> - 2011-07-31 09:46 +0100

csiph-web