Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #82551 > unrolled thread

Portable code: __import__ demands different string types between 2 and 3

Back to article view | Back to comp.lang.python

Contents

  Portable code: __import__ demands different string types between 2 and 3 Ben Finney <ben+python@benfinney.id.au> - 2014-12-15 18:29 +1100
    Re: Portable code: __import__ demands different string types between 2 and 3 Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-15 23:02 +1100

#82551 — Portable code: __import__ demands different string types between 2 and 3

Howdy all,

What should I do, in a world where all text literals are Unicode by
default, to make ‘__import__’ work in both Python 2 and 3?


I'm slowly learning about making Python code that will run under both
Python 2 (version 2.6 or above) and Python 3 (version 3.2 or above).

This entails, I believe, the admonition to ensure text literals are
Unicode by default::

    from __future__ import unicode_literals

and to specify bytes literals explicitly with ‘b'wibble'’ if needed.


The ‘__import__’ built-in function, though, is tripping up.

A contrived, trivial project layout::

    $ mkdir fooproject/
    $ cd fooproject/

    $ mkdir foo/
    $ printf "" > foo/__init__.py
    $ mkdir foo/bar/
    $ printf "" > foo/bar/__init__.py

Here's a simple ‘fooproject/setup.py’::

    from __future__ import unicode_literals

    main_module_name = 'foo'
    main_module = __import__(main_module_name, fromlist=['bar'])

    assert main_module.bar

That fails under Python 2, but runs fine under Python 3::

    $ python2 ./setup.py
    Traceback (most recent call last):
      File "./setup.py", line 4, in <module>
        main_module = __import__(main_module_name, fromlist=['bar'])
    TypeError: Item in ``from list'' not a string

    $ python3 ./setup.py

We've deliberately made unadorned strings Unicode by default. By “not a
string”, I presume Python 2 means “not a ‘bytes’ object”.


Okay, so we'll explicitly set that to a ‘bytes’ literal::

    from __future__ import unicode_literals

    main_module_name = 'foo'
    main_module = __import__(main_module_name, fromlist=[b'bar'])

    assert main_module.bar

Now Python 2 is satisfied, but Python 3 complains::

    $ python2 ./setup.py

    $ python3 ./setup.py
    Traceback (most recent call last):
      File "./setup.py", line 4, in <module>
        main_module = __import__(main_module_name, fromlist=[b'bar'])
      File "<frozen importlib._bootstrap>", line 2281, in
        _handle_fromlist
    TypeError: hasattr(): attribute name must be string


How can I get that ‘__import__’ call, complete with its ‘fromlist’
parameter, working correctly under both Python 2 and Python 3, keeping
the ‘unicode_literals’ setting?

If some kind of kludge is needed to make it work between versions, is
this a bug which should be fixed so “use Unicode for text” remains
applicable advice?

-- 
 \           “I do not believe in immortality of the individual, and I |
  `\        consider ethics to be an exclusively human concern with no |
_o__)  superhuman authority behind it.” —Albert Einstein, letter, 1953 |
Ben Finney

[toc] | [next] | [standalone]

#82561

Ben Finney wrote:

> Howdy all,
> 
> What should I do, in a world where all text literals are Unicode by
> default, to make ‘__import__’ work in both Python 2 and 3?

One approach I frequently use is a conditional import. Off the top of my
head, I'd do something like this:


try:
    import builtins  # Python 3.x.
except ImportError:
    # We're probably running Python 2.x.
    import __builtin__ as builtins

# Untested, just to give you an idea of what I mean.
try:
    _ = __import__("sys", fromlist=["version"])
except TypeError:
    # Shadow the built-in with a patched version.
    def __import__(*args, **kwargs):
        if "fromlist" in kwargs:
            kwargs["fromlist"] = [str(name) for name in kwargs["fromlist"]]
        return builtins.__import__(*args, **kwargs)

If you're really brave, you can even monkey-patch builtins with your own
version. Obviously you still need to keep the old version somewhere. A
closure would be perfect for that:

# Again, untested.
def patch_import(original_import=__import__):
    def __import__(*args, **kwargs):
        if "fromlist" in kwargs:
            kwargs["fromlist"] = [str(name) for name in kwargs["fromlist"]]
        return original_import(*args, **kwargs)
    builtins.__import__ = __import__

Monkey-patching builtins.__import__ is one of the few
not-completely-frowned-upon uses of monkey-patching.

Perhaps a better approach might be to eschew the use of __import__ and see
whether the functions in imputil (deprecated) or importlib do what you
need.

https://docs.python.org/2/library/imputil.html
https://docs.python.org/3/library/importlib.html

Aside: __import__ is not recommended for user code.

"Direct use of __import__() is also discouraged in favor of
importlib.import_module()."

https://docs.python.org/3/library/functions.html#__import__



-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web