Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #72759 > unrolled thread

Pandas question

Back to article view | Back to comp.lang.python

Contents

  Pandas question Albert-Jan Roskam <fomcl@yahoo.com> - 2014-06-05 12:40 -0700

#72759 — Pandas question

Hi,

I am new to Pandas. I am trying to remove the lower and upper 15 percentiles of interest rates within a day. The index column is the date. Below is some code, but how do I apply the trim function day-by-day? I tried using grouped() in conjunction with apply(), but that turned out to be an efficient way to slow my computer down and choke it. Any thoughts?


import pandas as pd

records = pd.read_table("blah.csv", sep=";", parse_dates=[2], index_col=2, low_memory=False)


def trim(df, colname, boundaries=(0.15, 0.85)):
    lo = df[colname] >= df[colname].quantile(boundaries[0])
    hi = df[colname] <= df[colname].quantile(boundaries[1])
    return df[lo & hi]

trimmed = trim(records, 'pct_12m') # this trims across all data, not day-by-day, which I want.

Oh, and is something like the following possible instead of df[lo & hi]?

df[lo <= df[colname] <= hi]  # looks nice, but gives ValueError

 
Thanks in advance!


Regards,

Albert-Jan




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a 

fresh water system, and public health, what have the Romans ever done for us?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 

[toc] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web