Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #72759

Pandas question

Show key headers only | View raw

Hi,

I am new to Pandas. I am trying to remove the lower and upper 15 percentiles of interest rates within a day. The index column is the date. Below is some code, but how do I apply the trim function day-by-day? I tried using grouped() in conjunction with apply(), but that turned out to be an efficient way to slow my computer down and choke it. Any thoughts?


import pandas as pd

records = pd.read_table("blah.csv", sep=";", parse_dates=[2], index_col=2, low_memory=False)


def trim(df, colname, boundaries=(0.15, 0.85)):
    lo = df[colname] >= df[colname].quantile(boundaries[0])
    hi = df[colname] <= df[colname].quantile(boundaries[1])
    return df[lo & hi]

trimmed = trim(records, 'pct_12m') # this trims across all data, not day-by-day, which I want.

Oh, and is something like the following possible instead of df[lo & hi]?

df[lo <= df[colname] <= hi]  # looks nice, but gives ValueError

 
Thanks in advance!


Regards,

Albert-Jan




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a 

fresh water system, and public health, what have the Romans ever done for us?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Pandas question Albert-Jan Roskam <fomcl@yahoo.com> - 2014-06-05 12:40 -0700

csiph-web