Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!newsfeed.hal-mli.net!feeder1.hal-mli.net!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Newsgroups: comp.lang.python
Date: Wed, 23 Jan 2013 09:33:31 -0800 (PST)
In-Reply-To: <mailman.910.1358959906.2939.python-list@python.org>
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=129.118.224.160; posting-account=DOyQBwoAAADIEpwWr89vk1x7RGFM33oc
References: <0909ee56-c304-4b9d-8be9-97c7a3261d27@googlegroups.com> <j4l4t9-skt.ln1@satorlaser.homedns.org> <mailman.880.1358935694.2939.python-list@python.org> <8daf86cf-386c-44ae-83d5-25fb04572c00@googlegroups.com> <mailman.898.1358952056.2939.python-list@python.org> <a5662c8c-67c3-47e6-a8b7-2eddbc272657@googlegroups.com> <mailman.910.1358959906.2939.python-list@python.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Subject: Re: Memory error with quadratic interpolation
From: Isaac Won <winefrog@gmail.com>
To: comp.lang.python@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Cc: Isaac Won <winefrog@gmail.com>, Python List <python-list@python.org>, Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com>
Precedence: list
Message-ID: <mailman.914.1358962421.2939.python-list@python.org>
Lines: 169
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:37496

On Wednesday, January 23, 2013 10:51:43 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 14:57, Isaac Won <winefrog@gmail.com> wrote:
> 
> > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
> 
> >> On 23 January 2013 14:28, Isaac Won <winefrog@gmail.com> wrote:
> 
> >>
> 
> [SNIP]
> 
> >
> 
> > Following is full error message after I adjusted following Ulich's advice:
> 
> >
> 
> > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
> 
> > File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__
> 
> >      self._spline = splmake(x,oriented_y,order=order)
> 
> >   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake
> 
> >      coefs = func(xk, yk, order, conds, B)
> 
> >   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest
> 
> >      u,s,vh = np.dual.svd(B)
> 
> >   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd
> 
> >      full_matrices=full_matrices, overwrite_a = overwrite_a)
> 
> > MemoryError
> 
> 
> 
> Where is the new code? You should show full working code (with the
> 
> import statements) and the full error that is generated by exactly
> 
> that code. If possible you should also write code that someone else
> 
> could run even without having access to your data files. If you did
> 
> that in your first post, you'd probably have an answer to your problem
> 
> by now.
> 
> 
> 
> Here is a version of your code that many people on this list can test
> 
> straight away:
> 
> 
> 
> import numpy as np
> 
> from scipy.interpolate import interp1d
> 
> x = np.array(31747 * [0.0], float)
> 
> indices = np.arange(len(x))
> 
> interp = interp1d(indices, x, kind='quadratic')
> 
> 
> 
> Running this gives the following error:
> 
> 
> 
> ~$ python tmp.py
> 
> Traceback (most recent call last):
> 
>   File "tmp.py", line 5, in <module>
> 
>     interp = interp1d(indices, x, kind='quadratic')
> 
>   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
> 
> line 308, in __init__
> 
>     self._spline = splmake(x,oriented_y,order=order)
> 
>   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
> 
> line 805, in splmake
> 
>     B = _fitpack._bsplmat(order, xk)
> 
> MemoryError
> 
> 
> 
> Unless I've misunderstood how this function is supposed to be used, it
> 
> just doesn't really seem to work for arrays of much more than a few
> 
> hundred elements.
> 
> 
> 
> 
> 
> Oscar

Thank you Oscar for your help and advice.

I agree with you. So, I tried to find the way to solve this problem.

My full code adjusted is:
from scipy.interpolate import interp1d

import numpy as np
import matplotlib.pyplot as plt



with open(filin, "r") as f:

        for line in f:
                columns = line.strip().split()

                a.append(columns[5])
                x = np.array(a, float)


not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')

p = interp(indices)




k = np.arange(31747)

plt.subplot(211)
plt.plot(k, p)
plt.xlabel('Quadratic interpolation')
plt.subplot(212)
plt.plot(k, x)

plt.show()
-----------------------------------------------------------------
Whole error message was:

Traceback (most recent call last):
  File "QI1.py", line 22, in <module>
    interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__
    self._spline = splmake(x,oriented_y,order=order)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake
    coefs = func(xk, yk, order, conds, B)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest
    u,s,vh = np.dual.svd(B)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd
    full_matrices=full_matrices, overwrite_a = overwrite_a)
MemoryError
----------------------------------------------------------------------
Thank you again Oscar,

Isaac