Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #61317 > unrolled thread

Re: Eliminate "extra" variable

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Eliminate "extra" variable Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-12-08 19:10 +0000
    Re: Eliminate "extra" variable Roy Smith <roy@panix.com> - 2013-12-08 15:07 -0500
      Fwd: Eliminate "extra" variable Igor Korot <ikorot01@gmail.com> - 2013-12-08 12:58 -0800
      Re: Eliminate "extra" variable Tim Chase <python.list@tim.thechases.com> - 2013-12-08 16:18 -0600
      Re: Fwd: Eliminate "extra" variable Dave Angel <davea@davea.name> - 2013-12-08 17:36 -0500
      Re: Eliminate "extra" variable Igor Korot <ikorot01@gmail.com> - 2013-12-14 23:49 -0800
      Re: Eliminate "extra" variable Tim Chase <python.list@tim.thechases.com> - 2013-12-15 06:17 -0600
      Re: Eliminate "extra" variable Tim Chase <python.list@tim.thechases.com> - 2013-12-15 06:29 -0600
      Re: Eliminate "extra" variable Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2013-12-15 12:08 -0500
      Re: Eliminate "extra" variable Igor Korot <ikorot01@gmail.com> - 2013-12-15 14:46 -0800
      Re: Eliminate "extra" variable MRAB <python@mrabarnett.plus.com> - 2013-12-16 00:58 +0000
      Re: Eliminate "extra" variable Igor Korot <ikorot01@gmail.com> - 2013-12-15 18:43 -0800
      Re: Eliminate "extra" variable Dave Angel <davea@davea.name> - 2013-12-15 21:58 -0500
      Re: Eliminate "extra" variable Chris Angelico <rosuav@gmail.com> - 2013-12-16 13:57 +1100
      Re: Eliminate "extra" variable Igor Korot <ikorot01@gmail.com> - 2013-12-15 20:24 -0800

#61317 — Re: Eliminate "extra" variable

On 08/12/2013 18:58, Tim Chase wrote:
> On 2013-12-07 23:14, Igor Korot wrote:

[big snip]

>

Whenever I need date manipulations I always reach out to this 
http://labix.org/python-dateutil

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

[toc] | [next] | [standalone]

#61324

In article <mailman.3738.1386529877.18130.python-list@python.org>,
 Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:

> On 08/12/2013 18:58, Tim Chase wrote:
> > On 2013-12-07 23:14, Igor Korot wrote:
> 
> [big snip]
> 
> >
> 
> Whenever I need date manipulations I always reach out to this 
> http://labix.org/python-dateutil

The problem with dateutil is it's dog slow.  Sure, I use it too, when 
convenience is more important than performance, but have you ever looked 
at the code for dateutil.parser.parse()?  It's worth reading, just for 
fun.

[toc] | [prev] | [next] | [standalone]

#61329

---------- Forwarded message ----------
From: Igor Korot <ikorot01@gmail.com>
Date: Sun, Dec 8, 2013 at 12:57 PM
Subject: Re: Eliminate "extra" variable
To: Roy Smith <roy@panix.com>


Hi, guys,
Thank you for all those valuable suggestions.
2Tim Chase:
I guess you missed this: "My originalData comes from the database
query" and so the checking of the data quality is a DB burden. ;-)
As to the function: the function purpose is to process the data and
give out the list of dates and the dictionary of date:frequency.
It's input is the query result, so there is no looping when the
function is called. It is called only once.

Also, the data comes from either SQLite or mySQL and so to eliminate
the difference between those engines dates are processed as strings
and converted to dates for the calculation purposes only.
Maybe I will need to refactor SQLite processing to get the dates as
dates and not a string, but that's probably for the future. so that
dates will be kept as the datetime type until the end of the function.
As I wrote the dates will be used as the text for the plotting window
axis labels and as the labels they should come out as strings, hence
the conversion.

Thank you.


On Sun, Dec 8, 2013 at 12:07 PM, Roy Smith <roy@panix.com> wrote:
> In article <mailman.3738.1386529877.18130.python-list@python.org>,
>  Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>
>> On 08/12/2013 18:58, Tim Chase wrote:
>> > On 2013-12-07 23:14, Igor Korot wrote:
>>
>> [big snip]
>>
>> >
>>
>> Whenever I need date manipulations I always reach out to this
>> http://labix.org/python-dateutil
>
> The problem with dateutil is it's dog slow.  Sure, I use it too, when
> convenience is more important than performance, but have you ever looked
> at the code for dateutil.parser.parse()?  It's worth reading, just for
> fun.
> --
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#61330

On 2013-12-08 12:58, Igor Korot wrote:
> Also, the data comes from either SQLite or mySQL and so to eliminate
> the difference between those engines dates are processed as strings
> and converted to dates for the calculation purposes only.
> Maybe I will need to refactor SQLite processing to get the dates as
> dates and not a string, but that's probably for the future. so that
> dates will be kept as the datetime type until the end of the
> function. As I wrote the dates will be used as the text for the
> plotting window axis labels and as the labels they should come out
> as strings, hence the conversion.

Sqlite can do this automatically if you tell it to upon connecting:
    
>>> import sqlite3
>>> conn = sqlite3.connect('x.sqlite',
...    detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>> cur.execute("create table foo (s date);")
<sqlite3.Cursor object at 0x7f7f31665570>
>>> import datetime
>>> today = datetime.date.today()
>>> cur.execute("insert into foo(s) values (?)", (today,))
<sqlite3.Cursor object at 0x7f7f31665570>
>>> cur.execute("select * from foo")
<sqlite3.Cursor object at 0x7f7f31665570>
>>> r = cur.fetchone()
>>> print r
(datetime.date(2013, 12, 8),)


Note that it returns a datetime.date, the same as it was defined.

-tkc

[toc] | [prev] | [next] | [standalone]

#61331

On Sun, 8 Dec 2013 12:58:18 -0800, Igor Korot <ikorot01@gmail.com> 
wrote:
> It's input is the query result, so there is no looping when the
> function is called. It is called only once.

Then why save part of the result in an instance attribute? Just 
return all of the results as a tuple.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#61939

Tim,

On Sun, Dec 8, 2013 at 2:18 PM, Tim Chase <python.list@tim.thechases.com> wrote:
> On 2013-12-08 12:58, Igor Korot wrote:
>> Also, the data comes from either SQLite or mySQL and so to eliminate
>> the difference between those engines dates are processed as strings
>> and converted to dates for the calculation purposes only.
>> Maybe I will need to refactor SQLite processing to get the dates as
>> dates and not a string, but that's probably for the future. so that
>> dates will be kept as the datetime type until the end of the
>> function. As I wrote the dates will be used as the text for the
>> plotting window axis labels and as the labels they should come out
>> as strings, hence the conversion.
>
> Sqlite can do this automatically if you tell it to upon connecting:
>
>>>> import sqlite3
>>>> conn = sqlite3.connect('x.sqlite',
> ...    detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>>> cur.execute("create table foo (s date);")
> <sqlite3.Cursor object at 0x7f7f31665570>
>>>> import datetime
>>>> today = datetime.date.today()
>>>> cur.execute("insert into foo(s) values (?)", (today,))
> <sqlite3.Cursor object at 0x7f7f31665570>
>>>> cur.execute("select * from foo")
> <sqlite3.Cursor object at 0x7f7f31665570>
>>>> r = cur.fetchone()
>>>> print r
> (datetime.date(2013, 12, 8),)

Interesting.
I'm using datetime rather than the date type for CREATE TABLE() command.
And when running SELECT I still see the b'1998-08-05 23:12:12' string
representation when running my program under debugger.

And it is confirmed by running this:

>>> cur.execute("CREATE TABLE foo(bar datetime);")
<sqlite3.Cursor object at 0x00E1E6A0>
>>> import datetime
>>> today = datetime.date.today()
>>> cur.execute("insert into foo(bar) values (?)", (today,))
<sqlite3.Cursor object at 0x00E1E6A0>
>>> cur.execute("select * from foo")
<sqlite3.Cursor object at 0x00E1E6A0>
>>> res = cur.fetchall()
>>> print res
[(u'2013-12-14',)]
>>>

So, I guess this is a bug in the sqlite3 python module as datetime is
legal data type on the DB engine.

Thank you.

>
>
> Note that it returns a datetime.date, the same as it was defined.
>
> -tkc
>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#61942

On 2013-12-14 23:49, Igor Korot wrote:
> Tim,
> 
> On Sun, Dec 8, 2013 at 2:18 PM, Tim Chase wrote:
>>>>> conn = sqlite3.connect('x.sqlite',
>>... detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)

Your example code omitted this one crucial line.  Do you specify the
detect_types parameter to connect()?

-tkc

[toc] | [prev] | [next] | [standalone]

#61943

On 2013-12-15 06:17, Tim Chase wrote:
>>>>>> conn = sqlite3.connect('x.sqlite',
>>>... detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
> 
> Your example code omitted this one crucial line.  Do you specify the
> detect_types parameter to connect()?

It's really the PARSE_DECLTYPES that is important.

http://docs.python.org/2/library/sqlite3.html#sqlite3.PARSE_DECLTYPES

-tkc

[toc] | [prev] | [next] | [standalone]

#61956

On Sat, 14 Dec 2013 23:49:58 -0800, Igor Korot <ikorot01@gmail.com>
declaimed the following:


>
>So, I guess this is a bug in the sqlite3 python module as datetime is
>legal data type on the DB engine.
>
	Note though, SQLite3 only has four native data types: integer, float,
character, blob. It accepts almost anything for a type declaration, and
will map that to the closest internal format (moreover, it will accept any
actual datatype in any actual field -- you can store strings in an integer
field unless the string content IS a representation of an integer in which
case it is turned to binary).

	Regardless of how Python stores a date/datetime/time (well, except for
the floating point C date/time types), it will mostly be kept as a string
representation within SQLite3. 

	It then becomes the responsibility of the converter modules invoked by
the parsing option to identify and format the data. I suspect date and
datetime both parse into the same converter. I'm not sure how it behaves if
one defined converters for date and datetime separately, as I think it uses
substring matching to invoke the converter: a type of date and a type of
datetime would both match a converter for "date" if the date converter is
first in the lookup list.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#61970

Tim,

On Sun, Dec 15, 2013 at 4:29 AM, Tim Chase
<python.list@tim.thechases.com> wrote:
> On 2013-12-15 06:17, Tim Chase wrote:
>>>>>>> conn = sqlite3.connect('x.sqlite',
>>>>... detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>
>> Your example code omitted this one crucial line.  Do you specify the
>> detect_types parameter to connect()?

Yes, I did.
This is the beginning of the session:

>>> import sqlite3
>>> conn = sqlite3.connect('c:\Documents and Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types = sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)

Also please note that you session was missing the cursor creation command. ;-)

Thank you.

>
> It's really the PARSE_DECLTYPES that is important.
>
> http://docs.python.org/2/library/sqlite3.html#sqlite3.PARSE_DECLTYPES
>
> -tkc
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#61975

On 15/12/2013 22:46, Igor Korot wrote:
> Tim,
>
> On Sun, Dec 15, 2013 at 4:29 AM, Tim Chase
> <python.list@tim.thechases.com> wrote:
>> On 2013-12-15 06:17, Tim Chase wrote:
>>>>>>>> conn = sqlite3.connect('x.sqlite',
>>>>>... detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>>
>>> Your example code omitted this one crucial line.  Do you specify the
>>> detect_types parameter to connect()?
>
> Yes, I did.
> This is the beginning of the session:
>
>>>> import sqlite3
>>>> conn = sqlite3.connect('c:\Documents and Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types = sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>
When writing paths on Windows, it's a good idea to use raw string
literals or slashes instead of backslashes:

     conn = sqlite3.connect(r'c:\Documents and 
Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types = 
sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)

or:

     conn = sqlite3.connect('c:/Documents and 
Settings/Igor.FORDANWORK/Desktop/mydb.db', detect_types = 
sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)

> Also please note that you session was missing the cursor creation command. ;-)
>
> Thank you.
>
>>
>> It's really the PARSE_DECLTYPES that is important.
>>
>> http://docs.python.org/2/library/sqlite3.html#sqlite3.PARSE_DECLTYPES
>>

[toc] | [prev] | [next] | [standalone]

#61985

Hi,

On Sun, Dec 15, 2013 at 4:58 PM, MRAB <python@mrabarnett.plus.com> wrote:
> On 15/12/2013 22:46, Igor Korot wrote:
>>
>> Tim,
>>
>> On Sun, Dec 15, 2013 at 4:29 AM, Tim Chase
>> <python.list@tim.thechases.com> wrote:
>>>
>>> On 2013-12-15 06:17, Tim Chase wrote:
>>>>>>>>>
>>>>>>>>> conn = sqlite3.connect('x.sqlite',
>>>>>>
>>>>>> ... detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>>>
>>>>
>>>> Your example code omitted this one crucial line.  Do you specify the
>>>> detect_types parameter to connect()?
>>
>>
>> Yes, I did.
>> This is the beginning of the session:
>>
>>>>> import sqlite3
>>>>> conn = sqlite3.connect('c:\Documents and
>>>>> Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types =
>>>>> sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>
>>
> When writing paths on Windows, it's a good idea to use raw string
> literals or slashes instead of backslashes:
>
>     conn = sqlite3.connect(r'c:\Documents and
> Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types =
> sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>
> or:
>
>     conn = sqlite3.connect('c:/Documents and
> Settings/Igor.FORDANWORK/Desktop/mydb.db', detect_types =
> sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)

So, how do I convert my string to one of those?
I realized I can just do replace '/' to '\', but is there a better alternative?

Thank you.


>
>
>> Also please note that you session was missing the cursor creation command.
>> ;-)
>>
>> Thank you.
>>
>>>
>>> It's really the PARSE_DECLTYPES that is important.
>>>
>>> http://docs.python.org/2/library/sqlite3.html#sqlite3.PARSE_DECLTYPES
>>>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [next] | [standalone]

#61987

On Sun, 15 Dec 2013 18:43:53 -0800, Igor Korot <ikorot01@gmail.com> 
wrote:
> On Sun, Dec 15, 2013 at 4:58 PM, MRAB <python@mrabarnett.plus.com> 
wrote:




> > When writing paths on Windows, it's a good idea to use raw string
> > literals or slashes instead of backslashes:
> >
> >     conn = sqlite3.connect(r'c:\Documents and
> > Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types =
> > sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
> >
> > or:
> >
> >     conn = sqlite3.connect('c:/Documents and
> > Settings/Igor.FORDANWORK/Desktop/mydb.db', detect_types =
> > sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)

> So, how do I convert my string to one of those?
> I realized I can just do replace '/' to '\', but is there a better 
alternative?

The "conversion" is done with a text editor.  The string literals you 
had were just wrong. By the time they have been turned into strings 
it's too late to algorithmically recover your intended string values.

-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#61988

On Mon, Dec 16, 2013 at 1:43 PM, Igor Korot <ikorot01@gmail.com> wrote:
> So, how do I convert my string to one of those?
> I realized I can just do replace '/' to '\', but is there a better alternative?

The path is exactly the same, whether you use forward slashes or
backslashes, on Windows. Most of the world uses backslashes for
Windows paths, but forward slashes do work fine (and that's not a
Python thing - the underlying APIs work with / just fine too). So you
don't have to actually replace anything.

>>> len(open("c:/Festival2013.txt").read())
79137

ChrisA

[toc] | [prev] | [next] | [standalone]

#61993

Hi,
I have the same result even with:

sqlite3.connect(r'...')

Any other alternatives?

Thank you.


On Sun, Dec 15, 2013 at 4:58 PM, MRAB <python@mrabarnett.plus.com> wrote:
> On 15/12/2013 22:46, Igor Korot wrote:
>>
>> Tim,
>>
>> On Sun, Dec 15, 2013 at 4:29 AM, Tim Chase
>> <python.list@tim.thechases.com> wrote:
>>>
>>> On 2013-12-15 06:17, Tim Chase wrote:
>>>>>>>>>
>>>>>>>>> conn = sqlite3.connect('x.sqlite',
>>>>>>
>>>>>> ... detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>>>
>>>>
>>>> Your example code omitted this one crucial line.  Do you specify the
>>>> detect_types parameter to connect()?
>>
>>
>> Yes, I did.
>> This is the beginning of the session:
>>
>>>>> import sqlite3
>>>>> conn = sqlite3.connect('c:\Documents and
>>>>> Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types =
>>>>> sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>>
>>
> When writing paths on Windows, it's a good idea to use raw string
> literals or slashes instead of backslashes:
>
>     conn = sqlite3.connect(r'c:\Documents and
> Settings\Igor.FORDANWORK\Desktop\mydb.db', detect_types =
> sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>
> or:
>
>     conn = sqlite3.connect('c:/Documents and
> Settings/Igor.FORDANWORK/Desktop/mydb.db', detect_types =
> sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
>
>
>> Also please note that you session was missing the cursor creation command.
>> ;-)
>>
>> Thank you.
>>
>>>
>>> It's really the PARSE_DECLTYPES that is important.
>>>
>>> http://docs.python.org/2/library/sqlite3.html#sqlite3.PARSE_DECLTYPES
>>>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web