Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!newsgate.cistron.nl!newsgate.news.xs4all.nl!194.109.133.85.MISMATCH!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <ckaynor@zindagigames.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'subject:Python': 0.05; 'memory.': 0.05; '(of': 0.07; 'space.': 0.07; 'url:msdn': 0.07; 'python': 0.09; '(it': 0.09; '32-bit': 0.09; 'available)': 0.09; 'cc:addr:python-list': 0.10; 'itself.': 0.11; 'subject:not': 0.11; 'crashes': 0.16; 'oct': 0.16; 'set,': 0.16; 'url:85)': 0.16; 'settings': 0.16; 'wrote:': 0.17; 'instance': 0.17; 'thanks,': 0.18; 'memory': 0.18; 'windows': 0.19; 'software.': 0.21; 'default,': 0.22; 'errors': 0.23; 'originally': 0.23; 'cc:2**1': 0.24; 'tried': 0.25; 'least': 0.25; 'cc:addr:python.org': 0.25; 'header:In-Reply-To:1': 0.25; 'setting': 0.26; 'compiled': 0.27; 'older': 0.27; 'message-id:@mail.gmail.com': 0.27; 'options': 0.27; 'noticed': 0.28; 'restricted': 0.29; 'starts': 0.29; 'that.': 0.30; 'fri,': 0.30; 'error': 0.30; 'file': 0.32; 'cases,': 0.33; 'entry': 0.33; 'received:google.com': 0.34; 'doing': 0.35; 'pm,': 0.35; 'received:209.85': 0.35; 'there': 0.35; 'but': 0.36; 'url:library': 0.36; 'received:209': 0.37; 'subject:: ': 0.38; 'some': 0.38; 'performance': 0.39; 'takes': 0.39; 'space': 0.39; 'url:microsoft': 0.39; 'application': 0.40; 'header:Received:5': 0.40; 'enable': 0.60; 'your': 0.60; 'url:aspx': 0.60; 'save': 0.61; 'maximum': 0.63; 'more': 0.63; 'gave': 0.65; 'limit': 0.65; 'url:en-us': 0.65; 'believe': 0.69; 'physical': 0.69; 'subject:available': 0.96
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding:x-gm-message-state; bh=yr0B4C/2auVwE9BY61339k+jZMtzmnr+zbJ66LCq8Ic=; b=ZBdXQw1tGYedp2XBijXPYjW4xu2h6jfpBmvn+mDBv4XZE7RJeBZjRK5mjs96JSeA+D /LO7/v8/0BsN0OSgBqAN0u1wWcBsIXy/7DCKwyoLIVF0oothepUWdP1WjwchzJCZHnVd uQUZwXWyU7QLlJf8NIQCvlPP1jZrA4NkVi4P5JWWZCGPq4k2VO51QB3/YIVd1+XARcx6 J0m3Jnk3DnckIkqLfPy3otCmxbaQiaIzzn9xN2N1quVHdJc8UDayDdkh3EMrMdNhmKwA rwOBJZL1bxFrbTkyVA09zTDFLCZT+F0TdTXFvhSAeKKjzVZCoMbfKSp3UshehTo0fTek X6AQ==
MIME-Version: 1.0
In-Reply-To: <76D03718A3233B4C8CC236C169B535B5A23E063EB5@AUSP01VMBX08.collaborationhost.net>
References: <76D03718A3233B4C8CC236C169B535B5A23E063E45@AUSP01VMBX08.collaborationhost.net> <k5s743$6jl$1@ger.gmane.org> <76D03718A3233B4C8CC236C169B535B5A23E063EB5@AUSP01VMBX08.collaborationhost.net>
From: Chris Kaynor <ckaynor@zindagigames.com>
Date: Fri, 19 Oct 2012 12:18:26 -0700
Subject: Re: Python does not take up available physical memory
To: Pradipto Banerjee <pradipto.banerjee@adainvestments.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Gm-Message-State: ALoCoQlpMMs+jA5v2Guu/ECcGJsQHkA9L67wPFjwHL0Kd4HQDrdRvJ5UQOu4FEF4m6fRF3qBmNJ2
Cc: "python-list@python.org" <python-list@python.org>, Emile van Sebille <emile@fenx.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2527.1350674329.27098.python-list@python.org>
Lines: 27
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1350674329 news.xs4all.nl 6904 [2001:888:2000:d::a6]:52912
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:31762

On Fri, Oct 19, 2012 at 12:03 PM, Pradipto Banerjee
<pradipto.banerjee@adainvestments.com> wrote:
>
> Thanks, I tried that. Still got MemoryError, but at least this time pytho=
n tried to use the physical memory. What I noticed is that before it gave m=
e the error it used up to 1.5GB (of the 2.23 GB originally showed as availa=
ble) - so in general, python takes up more memory than the size of the file=
 itself.
>

By default, 32-bit processes under Windows are restricted to only
using a maximum of 2GB of user-space memory, with the other 2GB
reserved for kernel usage. There are settings for Windows and the
application when compiled to up the limit to 3GB.

You can follow the MSDN entry at
http://msdn.microsoft.com/en-us/library/windows/desktop/bb613473(v=3Dvs.85)=
.aspx
to enable 3GB of space on your Windows box (it requires a reboot),
however I believe the standard Python installations do not have
IMAGE_FILE_LARGE_ADDRESS_AWARE set, so you will also have to compile a
custom Python instance to get Python to be able to use 3GB of space.
Note that doing so can result in instability and other side-effects:
the Windows options can reduce performance in some cases, and the
process setting can cause crashes or other errors if the application
presumes that user memory always starts with a 0-bit, often used to
save memory in older software.