Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #12378 > unrolled thread

Removing rows from a table

Started byCitzen90210 <peter_bodon@hotmail.com>
First post2012-11-18 22:15 +0000
Last post2012-11-22 00:51 +0000
Articles 3 — 2 participants

Back to article view | Back to comp.soft-sys.math.mathematica


Contents

  Removing rows from a table Citzen90210 <peter_bodon@hotmail.com> - 2012-11-18 22:15 +0000
    Re: Removing rows from a table Ray Koopman <koopman@sfu.ca> - 2012-11-20 00:17 +0000
    Re: Removing rows from a table Citzen90210 <peter_bodon@hotmail.com> - 2012-11-22 00:51 +0000

#12378 — Removing rows from a table

FromCitzen90210 <peter_bodon@hotmail.com>
Date2012-11-18 22:15 +0000
SubjectRemoving rows from a table
Message-ID<k8bmlv$dgr$1@smc.vnet.net>
Total nubie here and struggling!

I have a dataset of approximately 100,000 records of data points recorded at 15 minute intervals.  Each row looks something like this:

{"01/01/2010 06:15", 0.04375, 4.96188, 1.00885, 0, 0, 0}

I've worked out how to parse the date column so that seems to be under control but now I have a list of date ranges that I need to remove from the dataset.  The date ranges to be removed look something like this:

{{2010, 2, 1, 12, 0, 0.0}, {2010, 2, 15, 12, 0, 0.0}},
{{2010, 7, 1, 12, 0, 0.0}, {2010, 7, 15, 12, 0, 0.0}}

with the first column representing the start date of the block to be removed and the second column representing the end date of the block, multiple rows represent multiple blocks to be removed and the total number of blocks is variable.

Ideally I'd like to be able to use Drop or Delete to remove the offending blocks of data but I want to be able to somehow feed the command with the date range table and have it remove the ranges all in one go.

I'm not having much luck working this out for myself and would love to get a few pointers.

[toc] | [next] | [standalone]


#12393

FromRay Koopman <koopman@sfu.ca>
Date2012-11-20 00:17 +0000
Message-ID<k8ei7i$mgb$1@smc.vnet.net>
In reply to#12378
On Nov 18, 2:17 pm, Citzen90210 <peter_bo...@hotmail.com> wrote:
> Total nubie here and struggling!
>
> I have a dataset of approximately 100,000 records of data points recorded at 15 minute intervals.  Each row looks something like this:
>
> {"01/01/2010 06:15", 0.04375, 4.96188, 1.00885, 0, 0, 0}
>
> I've worked out how to parse the date column so that seems to be under control but now I have a list of date ranges that I need to remove from the dataset.  The date ranges to be removed look something like this:
>
> {{2010, 2, 1, 12, 0, 0.0}, {2010, 2, 15, 12, 0, 0.0}},
> {{2010, 7, 1, 12, 0, 0.0}, {2010, 7, 15, 12, 0, 0.0}}
>
> with the first column representing the start date of the block to be removed and the second column representing the end date of the block, multiple rows represent multiple blocks to be removed and the total number of blocks is variable.
>
> Ideally I'd like to be able to use Drop or Delete to remove the offending blocks of data but I want to be able to somehow feed the command with the date range table and have it remove the ranges all in one go.
>
> I'm not having much luck working this out for myself and would love to get a few pointers.

exclude = {
{{2010, 2, 1, 12, 0, 0.}, {2010, 2,15, 12, 0, 0.}},
{{2010, 7, 1, 12, 0, 0.}, {2010, 7,15, 12, 0, 0.}}};

(* Assume that each term at level 2 of 'exclude' is
   {y, m, d, h, m, s} for AbsoluteTime. *)

x = Interval @@ Map[AbsoluteTime, exclude, {2}];

(* Assume that the first term in each row of 'data' is
   {y, m, d, h, m, s} for AbsoluteTime. *)

Select[data, Not@IntervalMemberQ[x, AbsoluteTime@#[[1]]]& ]

[toc] | [prev] | [next] | [standalone]


#12411

FromCitzen90210 <peter_bodon@hotmail.com>
Date2012-11-22 00:51 +0000
Message-ID<k8jsuf$5p4$1@smc.vnet.net>
In reply to#12378
On Monday, 19 November 2012 09:17:44 UTC+11, Citzen90210  wrote:


Thank you Gent's, some nice options there for me mull over and try to understand.

I knew it would be easy if you knew what you were doing.

Many thanks.

PB

[toc] | [prev] | [standalone]


Back to top | Article view | comp.soft-sys.math.mathematica


csiph-web