Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <a5662c8c-67c3-47e6-a8b7-2eddbc272657@googlegroups.com>
References: <0909ee56-c304-4b9d-8be9-97c7a3261d27@googlegroups.com> <j4l4t9-skt.ln1@satorlaser.homedns.org> <mailman.880.1358935694.2939.python-list@python.org> <8daf86cf-386c-44ae-83d5-25fb04572c00@googlegroups.com> <mailman.898.1358952056.2939.python-list@python.org> <a5662c8c-67c3-47e6-a8b7-2eddbc272657@googlegroups.com>
Date: Wed, 23 Jan 2013 16:51:43 +0000
Subject: Re: Memory error with quadratic interpolation
From: Oscar Benjamin <oscar.j.benjamin@gmail.com>
To: Isaac Won <winefrog@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Cc: Python List <python-list@python.org>, Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.910.1358959906.2939.python-list@python.org>
Lines: 55
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:37491

On 23 January 2013 14:57, Isaac Won <winefrog@gmail.com> wrote:
> On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
>> On 23 January 2013 14:28, Isaac Won <winefrog@gmail.com> wrote:
>>
[SNIP]
>
> Following is full error message after I adjusted following Ulich's advice:
>
> interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
> File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__
>      self._spline = splmake(x,oriented_y,order=order)
>   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake
>      coefs = func(xk, yk, order, conds, B)
>   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest
>      u,s,vh = np.dual.svd(B)
>   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd
>      full_matrices=full_matrices, overwrite_a = overwrite_a)
> MemoryError

Where is the new code? You should show full working code (with the
import statements) and the full error that is generated by exactly
that code. If possible you should also write code that someone else
could run even without having access to your data files. If you did
that in your first post, you'd probably have an answer to your problem
by now.

Here is a version of your code that many people on this list can test
straight away:

import numpy as np
from scipy.interpolate import interp1d
x = np.array(31747 * [0.0], float)
indices = np.arange(len(x))
interp = interp1d(indices, x, kind='quadratic')

Running this gives the following error:

~$ python tmp.py
Traceback (most recent call last):
  File "tmp.py", line 5, in <module>
    interp = interp1d(indices, x, kind='quadratic')
  File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
line 308, in __init__
    self._spline = splmake(x,oriented_y,order=order)
  File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
line 805, in splmake
    B = _fitpack._bsplmat(order, xk)
MemoryError

Unless I've misunderstood how this function is supposed to be used, it
just doesn't really seem to work for arrays of much more than a few
hundred elements.


Oscar