Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #50010 > unrolled thread

analyzing time

Back to article view | Back to comp.lang.python

Contents

  analyzing time noydb <jenn.duerr@gmail.com> - 2013-07-05 12:18 -0700
    Re: analyzing time Neil Cerutti <neilc@norwich.edu> - 2013-07-05 19:35 +0000
    Re: analyzing time Skip Montanaro <skip@pobox.com> - 2013-07-05 14:43 -0500
    Re: analyzing time Gary Herron <gherron@digipen.edu> - 2013-07-05 12:47 -0700
    Re: analyzing time Terry Reedy <tjreedy@udel.edu> - 2013-07-05 15:54 -0400

#50010 — analyzing time

Hello All,

I have a table with a column of type date, with dates and time combined (like '1/6/2013 3:52:69PM'), that spans many months.  How would I pull out records that are the first and last entries per day?

Also, if I wanted to find time clusters per day (or per week) -- like if an entry is made every day around 11am -- is there a way to get at that temporal statistical cluster?

Python 2.7, Windows 7.

Any guidance would be greatly appreciated!  Time seems tricky...

Thanks,

N

[toc] | [next] | [standalone]

#50011

On 2013-07-05, noydb <jenn.duerr@gmail.com> wrote:
> Hello All,
>
> I have a table with a column of type date, with dates and time
> combined (like '1/6/2013 3:52:69PM'), that spans many months.
> How would I pull out records that are the first and last
> entries per day?
>
> Also, if I wanted to find time clusters per day (or per week)
> -- like if an entry is made every day around 11am -- is there a
> way to get at that temporal statistical cluster?
>
> Python 2.7, Windows 7.
>
> Any guidance would be greatly appreciated!  Time seems tricky...

Time *is* really tricky, but that's because we humans created a
tricky system. If can ignore issues of timespampts, timezones and
daylight savings time, then time handling in Python can be
simple.

datetime.datetime.strptime can translate the time format above
into datetime.datetime objects, which provide all the methods you
will need.

To find clusters and min and max values you will likely need to
put the datetime objects in a list, and use some Python builtins
and list methods.

-- 
Neil Cerutti

[toc] | [prev] | [next] | [standalone]

#50012

> I have a table with a column of type date, with dates and time combined (like '1/6/2013 3:52:69PM'), that spans many months.  How would I pull out records that are the first and last entries per day?

You mentioned "table" and "column", which leads me to think you are
dealing with data in a SQL database.  If so, that would likely change
the problem solution significantly.

If you have something like lists of string data in Python though, you
might want to make sure your timestamps are in a normalized form
first, so you can accurately sort by the timestamps.  For that, my
weapon of choice is the dateutil package, and in particular, the
dateutil.parser module:

>>> x = dateutil.parser.parse("1/6/2013 3:52:59PM")
>>> x
datetime.datetime(2013, 1, 6, 15, 52, 59)
>>> print x
2013-01-06 15:52:59

Once your timestamps are represented as Python datetime objects, the
problem gets a bit easier, especially if you want to find the
beginning and ending timestamps of a bunch of dates.  Sort, then throw
some itertools.groupby pixie dust at it. My ancient, reptilian brain
has never quite grokked all that iterator stuff, so I won't hazard a
guess how to spell the exact solution.

Skip

[toc] | [prev] | [next] | [standalone]

#50014

On 07/05/2013 12:18 PM, noydb wrote:
> Hello All,
>
> I have a table with a column of type date, with dates and time combined (like '1/6/2013 3:52:69PM'), that spans many months.  How would I pull out records that are the first and last entries per day?
>
> Also, if I wanted to find time clusters per day (or per week) -- like if an entry is made every day around 11am -- is there a way to get at that temporal statistical cluster?
>
> Python 2.7, Windows 7.
>
> Any guidance would be greatly appreciated!  Time seems tricky...
>
> Thanks,
>
> N

Are you asking a Python question, like how to turn a string "1/6/2013 
3:52:69PM" into an internal representation of time, or are you asking a 
data analysis and statistical question?

If the former, then look at datetime.strptime from the datetime module.

If the later, then you may get an answer here, but I'd suggest trying 
somewhere that discusses statistics and analysis.

Gary Herron

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

[toc] | [prev] | [next] | [standalone]

#50015

On 7/5/2013 3:18 PM, noydb wrote:

> I have a table with a column of type date, with dates and time

This is a datetime in Python parlance.

> combined (like '1/6/2013 3:52:69PM'), that spans many months.  How
> would I pull out records that are the first and last entries per
> day?

Sort on that column. Look at pairs of rows. If the days differ, you have 
the last of the first and the first of the second. One way:

it = <table iterator>
dt1 = next(it)
d1 = date(dt1)  # whatever that looks like

for row in it:
   dt2 = row
   d2 = date(dt2)
   if d1 != d2:
     do_whatever(dt1, dt2)
   dt1, d1 = dt2, d2

> Also, if I wanted to find time clusters per day (or per week) -- like
> if an entry is made every day around 11am -- is there a way to get at
> that temporal statistical cluster?

Make a histogram of time, ignoring date.

> Python 2.7, Windows 7.
>
> Any guidance would be greatly appreciated!  Time seems tricky...

Yes

-- 
Terry Jan Reedy

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web