Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #35531 > unrolled thread

How to get time.strptime()?

Back to article view | Back to comp.lang.python

Contents

  How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-26 08:33 -0800
    Re: How to get time.strptime()? Chris Rebert <clp2@rebertia.com> - 2012-12-26 09:34 -0800
    Re: How to get time.strptime()? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-12-26 17:43 +0000
      Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-26 11:18 -0800
    RE: How to get time.strptime()? Nick Cash <nick.cash@npcinternational.com> - 2012-12-26 21:06 +0000
      Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-26 19:09 -0800
        Re: How to get time.strptime()? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-12-27 04:40 +0000
        Re: How to get time.strptime()? Roy Smith <roy@panix.com> - 2012-12-26 23:53 -0500
        Re: How to get time.strptime()? Mitya Sirenef <msirenef@lightbird.net> - 2012-12-27 00:06 -0500
        Re: How to get time.strptime()? Chris Angelico <rosuav@gmail.com> - 2012-12-27 16:41 +1100
          Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-26 22:19 -0800
          Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-26 22:19 -0800
        Re: How to get time.strptime()? Ben Finney <ben+python@benfinney.id.au> - 2012-12-27 17:24 +1100
      Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-26 19:09 -0800
    Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-27 16:33 -0800
      Re: How to get time.strptime()? Ian Kelly <ian.g.kelly@gmail.com> - 2012-12-27 19:47 -0700
        Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-27 19:48 -0800
        Re: How to get time.strptime()? Gnarlodious <gnarlodious@gmail.com> - 2012-12-27 19:48 -0800

#35531 — How to get time.strptime()?

Error: AttributeError: 'module' object has no attribute '_strptime'

This problem is driving me crazy. It only happens in Python 3.3.0, while on my server running 3.1.3 it behaves as expected. When I try to access time.strptime() it errors with

AttributeError: 'module' object has no attribute '_strptime'.

This error only occurs under mod_wsgi, when running as a one-shot webapp it behaves normally. All other functionalities of the time module are normal.

If anyone could explain why it thinks I want an underscored name maybe it would help. 

Thanks.

-- Gnarlie

[toc] | [next] | [standalone]

#35532

[Multipart message — attachments visible in raw view] — view raw

On Wednesday, December 26, 2012, Gnarlodious wrote:

> Error: AttributeError: 'module' object has no attribute '_strptime'
>

 Please include the full Traceback, not just the final error message.

This problem is driving me crazy. It only happens in Python 3.3.0, while on
> my server running 3.1.3 it behaves as expected. When I try to access
> time.strptime() it errors with
>
> AttributeError: 'module' object has no attribute '_strptime'.


Might be the wrong `time` module. What's `print time.__file__` output?
Did you perchance create a time.py file in your project?

This error only occurs under mod_wsgi, when running as a one-shot webapp it
> behaves normally. All other functionalities of the time module are normal.
>
> If anyone could explain why it thinks I want an underscored name maybe it
> would help.
>
> Thanks.
>
> -- Gnarlie
> --
> http://mail.python.org/mailman/listinfo/python-list
>


-- 
Cheers,
Chris
--
http://rebertia.com

[toc] | [prev] | [next] | [standalone]

#35533

On Wed, 26 Dec 2012 08:33:42 -0800, Gnarlodious wrote:

> Error: AttributeError: 'module' object has no attribute '_strptime'
> 
> This problem is driving me crazy. It only happens in Python 3.3.0, while
> on my server running 3.1.3 it behaves as expected. When I try to access
> time.strptime() it errors with
> 
> AttributeError: 'module' object has no attribute '_strptime'.
[...]
> If anyone could explain why it thinks I want an underscored name maybe
> it would help.

Perhaps if you were to read the ENTIRE traceback, not just the last line, 
you will see something that hints at an explanation. Here is a similar 
error:

py> time.strptime(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/_strptime.py", line 494, in 
_strptime_time
    tt = _strptime(data_string, format)[0]
  File "/usr/local/lib/python3.3/_strptime.py", line 306, in _strptime
    raise TypeError(msg.format(index, type(arg)))
TypeError: strptime() argument 0 must be str, not <class 'NoneType'>


After reading the full traceback, I conclude that the time.strptime 
function calls _strptime._strptime. After importing _strptime and looking 
at the __file__ attribute, I see that it is a pure-python module. So, to 
answer your immediate question:

"explain why it [the time.strptime function] thinks I want an underscored 
name"

the answer is, it thinks you want an underscored name because you *do* 
need an underscored name. It is part of the implementation of 
time.strptime that it delegates some of the work to a private module 
_strptime.

As for why that is not available under mod_wsgi module, but is available 
when running under the normal Python environment, I have looked at the 
_strptime.py source code and nothing stands out to explain why the 
_strptime function might not be defined. So that seems like a question 
about mod_wsgi. I suggest you start here:

http://code.google.com/p/modwsgi/wiki/WhereToGetHelp

If you do get an answer, please post it here as well, so others in the 
future who stumble across this thread will see the answer.

Good luck!



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#35536

Thanks for the help.
This error only occurs on my devbox which is running Py 3.3.0:

print(time.__file__, file=sys.stderr)
/usr/local/python-3.3.0/frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload/time.so
 
which looks normal to me.
The server box, which is running Py 3.1.3, says this:
/usr/local/python-3.1.3/frameworks/Python.framework/Versions/3.1/lib/python3.1/lib-dynload/time.so

Also of possible relevance is that my server is running mod_wsgi 3.3, while my devbox is running mod_wsgi 3.4. However I had a discussion with Dumpleton who decided it is a Python problem:
https://groups.google.com/forum/#!topic/modwsgi/9MF2ixebvfg

I don't know how to get any sort of traceback out of mod_wsgi. The script runs normally as a single-page CGI script but the error only occurs when imported as a module. 

Exactly  what I am saying is this:
import time
print(time.strptime("00", "%y"), file=sys.stderr)
--> No module named 'time.strptime'

But this works normally:
print(time.time(), file=sys.stderr)
--> 1356549188.368575

I can say this and see 'strptime' in the list of functions:
print(dir(time), file=sys.stderr)
--> ['_STRUCT_TM_ITEMS', '__doc__', '__file__', '__loader__', '__name__', '__package__', 'altzone', 'asctime', 'clock', 'ctime', 'daylight', 'get_clock_info', 'gmtime', 'localtime', 'mktime', 'monotonic', 'perf_counter', 'process_time', 'sleep', 'strftime', 'strptime', 'struct_time', 'time', 'timezone', 'tzname', 'tzset']

So I am stumped.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35542

> Error: AttributeError: 'module' object has no attribute '_strptime'
> 
> This problem is driving me crazy. It only happens in Python 3.3.0,
> while on my server running 3.1.3 it behaves as expected. When I try to
> access time.strptime() it errors with
> 
> AttributeError: 'module' object has no attribute '_strptime'.
> 
> This error only occurs under mod_wsgi, when running as a one-shot
> webapp it behaves normally. All other functionalities of the time
> module are normal.

I've run into this issue in the past with Python 2.6 (or maybe 2.7) and mod_python. My problem was that datetime.strptime imported _strptime at runtime, and multiple threads trying to import it at the same time broke an internal import lock.

I was able to work around this by simply importing _strptime myself at server startup time. This may or may not work for you.

See also: http://code.google.com/p/modwsgi/issues/detail?id=177 and http://www.mail-archive.com/python-list@python.org/msg248664.html 

[toc] | [prev] | [next] | [standalone]

#35568

Nick Cash wrote:

> I was able to work around this by simply importing _strptime myself at server startup time.

THANK YOU! That fixed it, I simply put 

import _strptime

in my *.wsgi script. It feels like a kludgy solution, but it works.

I should also mention that I had a similar problem with the "threading" module that I fixed in the same way. This problem occurred inside of sqlite3 because evidently threadlocking has gone standard as of Py 3.3.x

This is problem that has unduly vexed me. When you start learning Python they don't tell you about these sharp edges. Someone needs to explain.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35572

On Wed, 26 Dec 2012 19:09:51 -0800, Gnarlodious wrote:

> This is problem that has unduly vexed me. When you start learning Python
> they don't tell you about these sharp edges.


"Hi, I've just started learning Python today, and I have a question. How 
do I print a list?"

"Before I answer that question, let me explain to you how to fix a 
problem with time.strptime that only occurs when running under mod_wsgi. 
Now, I know that you're still struggling with print, haven't even got to 
importing modules yet, have no idea what strptime is, and that 99.99% of 
Python programmers will never use mod_wsgi in their life, but if we don't 
tell you about this right now, in five years time you will complain that 
nobody told you about the sharp edges when you started..."

*wink*

A serious question: when should somebody have told you about this sharp 
edge, and who should have been responsible to do so?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#35574

In article <0e36e143-5103-4b4b-aaa9-b7b1b768631b@googlegroups.com>,
 Gnarlodious <gnarlodious@gmail.com> wrote:

> This is problem that has unduly vexed me. When you start learning Python they 
> don't tell you about these sharp edges. Someone needs to explain.

The same thing happens when you're born.  You spend your entire life 
figuring out the sharp edges.  And then you die.

[toc] | [prev] | [next] | [standalone]

#35576

On 12/26/2012 10:09 PM, Gnarlodious wrote:
> This is problem that has unduly vexed me. When you start learning 
> Python they don't tell you about these sharp edges. Someone needs to 
> explain. -- Gnarlie 

In fact, if there were no bugs, there would be no need to explain nor to 
pick the
right time / place for explanations. Why doesn't Python warn that it's not
100% perfect? Are people just supposed to "know" this, magically?

-- 
Lark's Tongue Guide to Python: http://lightbird.net/larks/

[toc] | [prev] | [next] | [standalone]

#35578

On Thu, Dec 27, 2012 at 4:06 PM, Mitya Sirenef <msirenef@lightbird.net> wrote:
> On 12/26/2012 10:09 PM, Gnarlodious wrote:
>>
>> This is problem that has unduly vexed me. When you start learning Python
>> they don't tell you about these sharp edges. Someone needs to explain. --
>> Gnarlie
>
>
> In fact, if there were no bugs, there would be no need to explain nor to
> pick the
> right time / place for explanations. Why doesn't Python warn that it's not
> 100% perfect? Are people just supposed to "know" this, magically?

People are supposed to know that Python is not 100% perfect because
there's no such thing as a 100% perfect language. The problem here is
that Python doesn't have any magical way to deal with messy imports in
multiple threads; the solution is to sort out the import before the
multiple threads get going (meaning that when each thread attempts the
import, it's resolved trivially by simply binding the existing module
to a new name).

Threads are fairly messy in the best of times. The only thing I could
imagine Python doing here would be a brutal global lock - as soon as
one thread starts any import, all other threads get locked out of
execution (or at least of other imports). And I'm not sure that'd
solve everything - plus it brings the risk that one thread can stall
the entire system with a failed or faulty import. If you can think of
of any way to improve this, I'm sure python-ideas will happily discuss
it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#35581

Chris Angelico wrote:
 
> The problem here is that Python
> doesn't have any magical way to deal with messy imports in
> multiple threads
But couldn't Py 3.3.1 at least raise an error mentioning threading as a possible cause? Because "No module named  _strptime" is pretty cryptic.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35582

Chris Angelico wrote:
 
> The problem here is that Python
> doesn't have any magical way to deal with messy imports in
> multiple threads
But couldn't Py 3.3.1 at least raise an error mentioning threading as a possible cause? Because "No module named  _strptime" is pretty cryptic.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35583

Mitya Sirenef <msirenef@lightbird.net> writes:

> Why doesn't Python warn that it's not 100% perfect? Are people just
> supposed to "know" this, magically?

+1 QotW

-- 
 \       “[T]he speed of response of the internet will re-introduce us |
  `\    to that from which our political systems have separated us for |
_o__)    so long, the consequences of our own actions.” —Douglas Adams |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#35569

Nick Cash wrote:

> I was able to work around this by simply importing _strptime myself at server startup time.

THANK YOU! That fixed it, I simply put 

import _strptime

in my *.wsgi script. It feels like a kludgy solution, but it works.

I should also mention that I had a similar problem with the "threading" module that I fixed in the same way. This problem occurred inside of sqlite3 because evidently threadlocking has gone standard as of Py 3.3.x

This is problem that has unduly vexed me. When you start learning Python they don't tell you about these sharp edges. Someone needs to explain.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35664

Graham Dumpleton has informed me that the the relevant bug reports are:

http://bugs.python.org/issue8098
http://bugs.python.org/issue9260

To quote:

All the problems derive from a stupid function in Python internals called PyImport_ImportModuleNoBlock(). It was designed to avoid lockups of the import machinery but just causes other problems because a module import can fail if the import lock is held by a different thread. This faulty magic fairy dust was sprinkled on the time.strptime() function which internally loads _strptime module. If you access time.strptime() when another thread holds the import lock, the call will fail because of a failed import.

So there. As more people update to Python 3.3.0 hopefully this magic fairy dust workaround will see some daylight.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35673

On Thu, Dec 27, 2012 at 5:33 PM, Gnarlodious <gnarlodious@gmail.com> wrote:
> Graham Dumpleton has informed me that the the relevant bug reports are:
>
> http://bugs.python.org/issue8098
> http://bugs.python.org/issue9260
>
> To quote:
>
> All the problems derive from a stupid function in Python internals called PyImport_ImportModuleNoBlock(). It was designed to avoid lockups of the import machinery but just causes other problems because a module import can fail if the import lock is held by a different thread. This faulty magic fairy dust was sprinkled on the time.strptime() function which internally loads _strptime module. If you access time.strptime() when another thread holds the import lock, the call will fail because of a failed import.
>
> So there. As more people update to Python 3.3.0 hopefully this magic fairy dust workaround will see some daylight.

Thanks for the context, but that actually seems to be describing the
situation from Python 3.2.  Python 3.3 introduced module-level import
locking with improved deadlock detection that looks for actual cycles
and is supposed to fix the unnecessary ImportErrors described in
#8098, rendering the PyImport_ImportModuleNoBlock "fairy dust"
unnecessary -- in 3.3, I understand that PyImport_ImportModuleNoBlock
is now just a synonym for PyImport_ImportModule.  But you're reporting
that you're still seeing issues with this in Python 3.3.  Diving into
the code, I think I start to understand why.  Take a look at this
function from Lib/importlib/_bootstrap.py:

def _lock_unlock_module(name):
    """Release the global import lock, and acquires then release the
    module lock for a given module name.
    This is used to ensure a module is completely initialized, in the
    event it is being imported by another thread.

    Should only be called with the import lock taken."""
    lock = _get_module_lock(name)
    _imp.release_lock()
    try:
        lock.acquire()
    except _DeadlockError:
        # Concurrent circular import, we'll accept a partially initialized
        # module object.
        pass
    else:
        lock.release()


Normally, it just acquires and releases the module lock to ensure that
the module has been completely initialized.  If a deadlock is
detected, however, then it skips that step and instead of raising an
ImportError (which would seem to me to be the right thing to do here)
it allows the importing code to simply proceed with a module that may
not be fully imported yet.  Since your error message concerned missing
module-level attributes within the _strptime module, that seems to be
what you're running into.  The time.strptime function attempts to
import _strptime, which is already being imported by another thread;
for some unknown reason it detects a cyclical deadlock as a result of
this; it then white-washes the deadlock and tries to use the _strptime
module anyway, causing a different error to occur.  The question then
is how you're managing to get an import cycle by using strptime.

[toc] | [prev] | [next] | [standalone]

#35676

Thank you for a reasonable discussion of the problem, although I don't really understand what you said. This is a WSGI webapp configured for one-process and one-thread.

I can imagine my unorthodox handling of imported modules is suspect. To explain, my webapp first loads modules from a dict of allowed pages:

__init__
    for module in self.allowedPages:
        try:
            self.Pages[module]=getattr(__import__(module), module)


This gives me a dict of objects like this:
self.Pages
{'Yijing': <Yijing.YijingProto object at 0x1067a4f90>, 'Strand': <Strand.StrandProto object at 0x1067b52d0>, 'Gnomon': <Gnomon.GnomonProto object at 0x10675a450>, 'Lexicon': <Lexicon.LexiconProto object at 0x1067c4a50>, 'Dream': <Dream.DreamProto object at 0x106800a50>, 'Grid': <Grid.GridProto object at 0x1067ac310>}


I can then extract the first query string variable name and get the page content:
__call__
    page='Gnomon'
    xml=self.Pages[page](EnvDict, qList) # sends environ & query list


I don't know how several modules are normally handled in a CGI webapp, but this homegrown system has served me well. Threadlocking, however, may find this sort of referencing problematic. That would be my first suspicion.

-- Gnarlie

[toc] | [prev] | [next] | [standalone]

#35677

Thank you for a reasonable discussion of the problem, although I don't really understand what you said. This is a WSGI webapp configured for one-process and one-thread.

I can imagine my unorthodox handling of imported modules is suspect. To explain, my webapp first loads modules from a dict of allowed pages:

__init__
    for module in self.allowedPages:
        try:
            self.Pages[module]=getattr(__import__(module), module)


This gives me a dict of objects like this:
self.Pages
{'Yijing': <Yijing.YijingProto object at 0x1067a4f90>, 'Strand': <Strand.StrandProto object at 0x1067b52d0>, 'Gnomon': <Gnomon.GnomonProto object at 0x10675a450>, 'Lexicon': <Lexicon.LexiconProto object at 0x1067c4a50>, 'Dream': <Dream.DreamProto object at 0x106800a50>, 'Grid': <Grid.GridProto object at 0x1067ac310>}


I can then extract the first query string variable name and get the page content:
__call__
    page='Gnomon'
    xml=self.Pages[page](EnvDict, qList) # sends environ & query list


I don't know how several modules are normally handled in a CGI webapp, but this homegrown system has served me well. Threadlocking, however, may find this sort of referencing problematic. That would be my first suspicion.

-- Gnarlie

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web