Path: csiph.com!newsfeed.hal-mli.net!feeder3.hal-mli.net!news.stack.nl!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: UNSURE 0.217 X-Spam-Level: ** X-Spam-Evidence: '*H*': 0.58; '*S*': 0.01; 'column': 0.07; 'oh,': 0.09; 'subject:question': 0.10; 'def': 0.12; 'advance!': 0.16; 'thoughts?': 0.16; 'valueerror': 0.16; 'index': 0.16; 'trying': 0.19; 'code,': 0.22; 'import': 0.22; 'conjunction': 0.24; 'looks': 0.24; 'tried': 0.27; 'function': 0.29; 'gives': 0.31; 'header:Received:9': 0.33; 'something': 0.35; 'but': 0.35; 'data,': 0.36; 'date.': 0.36; 'done': 0.36; 'thanks': 0.36; 'possible': 0.36; 'hi,': 0.36; 'system,': 0.38; 'to:addr:python- list': 0.38; '\xa0\xa0\xa0': 0.39; 'to:addr:python.org': 0.39; 'skip:p 20': 0.39; 'how': 0.40; 'remove': 0.60; 'received:98.137': 0.60; 'lower': 0.61; 'new': 0.61; 'rates': 0.61; 'day.': 0.63; 'interest': 0.64; 'within': 0.65; 'header:Reply-To:1': 0.67; 'apart': 0.72; 'records': 0.73; 'upper': 0.74; 'nice,': 0.84; 'received:98.138.229': 0.84; 'us?': 0.84; 'medicine,': 0.91; 'education,': 0.96 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 123730.57538.bm@omp1022.mail.gq1.yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1401997250; bh=D9c002YtypbaaLiYvEJZgk+e22vuRg9gN1EM9WtOIzA=; h=Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=YX0aDOYe94RCMG89GjEHjzMewZ0nk29geMrS3bbUK1aqaNHiI+P4oOyh0PwhBzFgKPxSn474rqCQnCuVchRoraspffwcQYiOCnIaOlH66Ed4VX+QGRRgOSqvjNGiLJBiAhsb1V822GGGm1QjhgUUDorVDR45EvDb5BaUXxqGwoM= X-YMail-OSG: QEduA9wVM1kV3NhWy56WfoAUYPHmMr3OwdpATRiRi2SkVNU avSHQhIBzJa.RfiMEMZuXqOscuTMlTx0mv6BswX6gKMJUFlhTQl6k4qUsNxJ xGxeWjQtVSE53QMSImvswzwCxytbN2LfyhDUnkQNR6qVy_fhz6EYLk1IMwnK UQZ0cQGeX31LNIIrb2rvNPnBB82KJbYMwAGBWc2gTIzHikKcC9qrU5_q1Sp9 BW14FwuOegn0tGFrYiu2lSWdEQWkfTouCM_pIJIe4iuzXy1xgpNvi4HzAV2q 2tCrNWOSn5kmS2YKDKG86LZu3MCA2w65h6WbYZqmTbOOMSPBP7XqqPUMyGMN 2UZPJARNXnZ2kXpXizK6ajFFlfp8HRkw4SAds_ejganITqtTcqebJ_oPI3IK IvD.x8UhCod8Ffuk_WUQ7CjwdyVtJzhtuMTQdL8pj3pBhBhyhyBBVj7HeUui WNub4JSO57fcuLNlREwc5wKC93XubFxF_ZeXTmXq8zBewfFV9Re6Ttawf.n6 OsbMXFfMDAARQMH3xO3L5GB_3J62PCXPWUxVt7eqmDfcJgX1_Cn.LtgDGHw- - X-Rocket-MIMEInfo: 002.001, SGksCgpJIGFtIG5ldyB0byBQYW5kYXMuIEkgYW0gdHJ5aW5nIHRvIHJlbW92ZSB0aGUgbG93ZXIgYW5kIHVwcGVyIDE1IHBlcmNlbnRpbGVzIG9mIGludGVyZXN0IHJhdGVzIHdpdGhpbiBhIGRheS4gVGhlIGluZGV4IGNvbHVtbiBpcyB0aGUgZGF0ZS4gQmVsb3cgaXMgc29tZSBjb2RlLCBidXQgaG93IGRvIEkgYXBwbHkgdGhlIHRyaW0gZnVuY3Rpb24gZGF5LWJ5LWRheT8gSSB0cmllZCB1c2luZyBncm91cGVkKCkgaW4gY29uanVuY3Rpb24gd2l0aCBhcHBseSgpLCBidXQgdGhhdCB0dXJuZWQgb3V0IHRvIGIBMAEBAQE- X-Mailer: YahooMailWebService/0.8.190.668 Date: Thu, 5 Jun 2014 12:40:49 -0700 (PDT) From: Albert-Jan Roskam Subject: Pandas question To: Python MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 05 Jun 2014 22:21:36 +0200 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Albert-Jan Roskam List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 19 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1401999697 news.xs4all.nl 2975 [2001:888:2000:d::a6]:56118 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:72759 Hi,=0A=0AI am new to Pandas. I am trying to remove the lower and upper 15 p= ercentiles of interest rates within a day. The index column is the date. Be= low is some code, but how do I apply the trim function day-by-day? I tried = using grouped() in conjunction with apply(), but that turned out to be an e= fficient way to slow my computer down and choke it. Any thoughts?=0A=0A=0Ai= mport pandas as pd=0A=0Arecords =3D pd.read_table("blah.csv", sep=3D";", pa= rse_dates=3D[2], index_col=3D2, low_memory=3DFalse)=0A=0A=0Adef trim(df, co= lname, boundaries=3D(0.15, 0.85)):=0A=A0=A0=A0 lo =3D df[colname] >=3D df[c= olname].quantile(boundaries[0])=0A=A0=A0=A0 hi =3D df[colname] <=3D df[coln= ame].quantile(boundaries[1])=0A=A0=A0=A0 return df[lo & hi]=0A=0Atrimmed = =3D trim(records, 'pct_12m') # this trims across all data, not day-by-day, = which I want.=0A=0AOh, and is something like the following possible instead= of df[lo & hi]?=0A=0Adf[lo <=3D df[colname] <=3D hi]=A0 # looks nice, but = gives ValueError=0A=0A=A0=0AThanks in advance!=0A=0A=0ARegards,=0A=0AAlbert= -Jan=0A=0A=0A=0A=0A~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~= ~~~~~~~~~~~~~~=0A=0AAll right, but apart from the sanitation, the medicine,= education, wine, public order, irrigation, roads, a =0A=0Afresh water syst= em, and public health, what have the Romans ever done for us?=0A=0A~~~~~~~~= ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~