Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #91342 > unrolled thread

Returning a custom file object (Python 3)

Back to article view | Back to comp.lang.python

Contents

  Returning a custom file object (Python 3) Steven D'Aprano <steve@pearwood.info> - 2015-05-28 12:16 +1000
    Re: Returning a custom file object (Python 3) Ben Finney <ben+python@benfinney.id.au> - 2015-05-28 12:40 +1000
      Re: Returning a custom file object (Python 3) Marko Rauhamaa <marko@pacujo.net> - 2015-05-28 08:29 +0300
        Re: Returning a custom file object (Python 3) Chris Angelico <rosuav@gmail.com> - 2015-05-28 15:49 +1000
          Re: Returning a custom file object (Python 3) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-28 17:04 +1000
            Re: Returning a custom file object (Python 3) Chris Angelico <rosuav@gmail.com> - 2015-05-28 19:06 +1000
        Re: Returning a custom file object (Python 3) Gary Herron <gherron@digipen.edu> - 2015-05-27 22:56 -0700
          Re: Returning a custom file object (Python 3) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-28 16:52 +1000
    Re: Returning a custom file object (Python 3) Ben Finney <ben+python@benfinney.id.au> - 2015-05-28 12:34 +1000
    Re: Returning a custom file object (Python 3) Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-05-28 10:29 +0100

#91342 — Returning a custom file object (Python 3)

I'd like to return a custom file object, say my own subclass. I can easily
subclass the file object:


from io import TextIOWrapper
class MyFile(TextIOWrapper):
    pass


but how do I tell open() to use MyFile?


Answers for Python 3, thanks.




-- 
Steven

[toc] | [next] | [standalone]

#91343

Ben Finney <ben+python@benfinney.id.au> writes:

> Steven D'Aprano <steve@pearwood.info> writes:
>
> > but how do I tell open() to use MyFile?
>
> I haven't used it, but does the ‘opener’ parameter do what you want?

No, it doesn't; the ‘opener’ parameter doesn't have any say in the type
of object returned from ‘open’.

It seems the existing ‘open’ implementation doesn't allow you to
override the type of object returned. You may have to resort to
monkey-patching the ‘io’ attributes to put your preferred classes there.

-- 
 \       “Science shows that belief in God is not only obsolete. It is |
  `\                        also incoherent.” —Victor J. Stenger, 2001 |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#91350

Ben Finney <ben+python@benfinney.id.au>:

> It seems the existing ‘open’ implementation doesn't allow you to
> override the type of object returned.

The question is, can you assign to the builtin namespace. I'm guessing
you can't.

Within a module, you can simply do:

   open = MyFile

Also, in other modules, you can:

   from myfile import open


Marko

[toc] | [prev] | [next] | [standalone]

#91351

On Thu, May 28, 2015 at 3:29 PM, Marko Rauhamaa <marko@pacujo.net> wrote:
> Ben Finney <ben+python@benfinney.id.au>:
>
>> It seems the existing ‘open’ implementation doesn't allow you to
>> override the type of object returned.
>
> The question is, can you assign to the builtin namespace. I'm guessing
> you can't.
>
> Within a module, you can simply do:
>
>    open = MyFile
>
> Also, in other modules, you can:
>
>    from myfile import open

Well, you can...

>>> import builtins
>>> builtins.open
<built-in function open>
>>> builtins.open = "not your grandfather's open() function"
>>> open
"not your grandfather's open() function"

... but I don't think replacing all of open() is what Steven has in
mind; it's a function that does a lot of work, most of which is what's
wanted.

Depending on how brutal you want to be, though, you _could_ hack around it.

>>> from io import TextIOWrapper
>>> class MyFile(TextIOWrapper):
...     def demo(self): print("Hello, world!")
...
>>> f = open("/tmp/dummy", "w", encoding="utf-8")
>>> f
<_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>> f.__class__
<class '_io.TextIOWrapper'>
>>> f.__class__ = MyFile
>>> f.demo()
Hello, world!

This does appear to work. Whether or not it's a good idea is a
separate question. And of course, you could replace open() with
something like this:

>>> _orig_open = open
>>> def open(*args, **kw):
...   cls = kw.pop("use_class", None)
...   f = _orig_open(*args, **kw)
...   if cls is not None: f.__class__ = cls
...   return f
...
>>> f = open("/tmp/dummy", "w", encoding="utf-8", use_class=MyFile)
>>> f
<_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>> f.demo()
Hello, world!

But again, I'm really not sure this is a good way to do things.

ChrisA

[toc] | [prev] | [next] | [standalone]

#91354

On Thursday 28 May 2015 15:49, Chris Angelico wrote:

> ... but I don't think replacing all of open() is what Steven has in
> mind; it's a function that does a lot of work, most of which is what's
> wanted.

Correct. If I wanted to replace open(), I would have just shadowed it, or 
monkey-patched it, or just used myopen().

I want open() to return my own subclass instead of the standard one. And I 
think you have the solution I was looking for:


> Depending on how brutal you want to be, though, you _could_ hack around
> it.
> 
>>>> from io import TextIOWrapper
>>>> class MyFile(TextIOWrapper):
> ...     def demo(self): print("Hello, world!")
> ...
>>>> f = open("/tmp/dummy", "w", encoding="utf-8")
>>>> f
> <_io.TextIOWrapper name='/tmp/dummy' mode='w' encoding='utf-8'>
>>>> f.__class__
> <class '_io.TextIOWrapper'>
>>>> f.__class__ = MyFile
>>>> f.demo()
> Hello, world!
> 
> This does appear to work. Whether or not it's a good idea is a
> separate question.


And this is EXACTLY the sort of use-case that having __class__ be writable 
is intended to solve. So this is exactly the solution I was after, thank 
you. Er... except for one little problem... in Python 3.3:

py> f = open("/tmp/a", "r")
py> f.__class__ = MyFile
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __class__ assignment: only for heap types


So it doesn't work for me. What version of Python are you using?



-- 
Steve

[toc] | [prev] | [next] | [standalone]

#91355

On Thu, May 28, 2015 at 5:04 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
>> This does appear to work. Whether or not it's a good idea is a
>> separate question.
>
>
> And this is EXACTLY the sort of use-case that having __class__ be writable
> is intended to solve. So this is exactly the solution I was after, thank
> you. Er... except for one little problem... in Python 3.3:
>
> py> f = open("/tmp/a", "r")
> py> f.__class__ = MyFile
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: __class__ assignment: only for heap types
>
>
> So it doesn't work for me. What version of Python are you using?

Huh, didn't even think to check other versions.

$ python3
Python 3.5.0b1+ (default:7255af1a1c50+, May 26 2015, 00:39:06)
[GCC 4.9.2] on linux

It's a build fresh from source control around about this weekend (some
time after feature freeze, obviously, but beyond that all I know is
what you see there). I guess this is pretty new; trying the same thing
on 3.4.2 doesn't work. But hey! 3.5 will be out soon...

ChrisA

[toc] | [prev] | [next] | [standalone]

#91352

On 05/27/2015 10:29 PM, Marko Rauhamaa wrote:
> Ben Finney <ben+python@benfinney.id.au>:
>
>> It seems the existing ‘open’ implementation doesn't allow you to
>> override the type of object returned.
> The question is, can you assign to the builtin namespace. I'm guessing
> you can't.

Of course you can.

Python2:
 >>> __builtins__.open = "whatever"
 >>>

Python3:
 >>> import builtins
 >>> builtins.open = "whatever"
 >>>

Of course doing so is like shooting yourself in the foot:  Any 
subsequent pain is your own fault and probably well deserved.

Gary Herron



>
> Within a module, you can simply do:
>
>     open = MyFile
>
> Also, in other modules, you can:
>
>     from myfile import open
>
>
> Marko


-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

[toc] | [prev] | [next] | [standalone]

#91353

On Thursday 28 May 2015 15:56, Gary Herron wrote:

> On 05/27/2015 10:29 PM, Marko Rauhamaa wrote:
>> Ben Finney <ben+python@benfinney.id.au>:
>>
>>> It seems the existing ‘open’ implementation doesn't allow you to
>>> override the type of object returned.
>> The question is, can you assign to the builtin namespace. I'm guessing
>> you can't.
> 
> Of course you can.
> 
> Python2:
>  >>> __builtins__.open = "whatever"


Don't use __builtins__ with an "s". That's an implementation of CPython and 
may not exist in the future, and doesn't exist in other implementations.

Instead, you should import __builtin__ with no "s". Confusing? It certainly 
is, which is why Python 3 renamed __builtin__ to builtins.




-- 
Steve

[toc] | [prev] | [next] | [standalone]

#91344

Steven D'Aprano <steve@pearwood.info> writes:

> from io import TextIOWrapper
> class MyFile(TextIOWrapper):
>     pass
>
> but how do I tell open() to use MyFile?

I haven't used it, but does the ‘opener’ parameter do what you want?

    open(file, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None, closefd=True, opener=None) ->
             file object

    […]

    A custom opener can be used by passing a callable as *opener*. The
    underlying file descriptor for the file object is then obtained by
    calling *opener* with (*file*, *flags*). *opener* must return an
    open file descriptor (passing os.open as *opener* results in
    functionality similar to passing None).

-- 
 \          “Our products just aren't engineered for security.” —Brian |
  `\             Valentine, senior vice-president of Microsoft Windows |
_o__)                                                      development |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#91356

On 28 May 2015 at 03:16, Steven D'Aprano <steve@pearwood.info> wrote:
> I'd like to return a custom file object, say my own subclass. I can easily
> subclass the file object:
>
>
> from io import TextIOWrapper
> class MyFile(TextIOWrapper):
>     pass
>
>
> but how do I tell open() to use MyFile?

Does the below do what you want?

#!/usr/bin/env python3

class Wrapper:
    def __init__(self, wrapped):
        self._wrapped = wrapped
    def __getattr__(self, attrname):
        return getattr(self._wrapped, attrname)
    # Special methods are not resolved by __getattr__
    def __iter__(self):
        return self._wrapped.__iter__()
    def __enter__(self):
        return self._wrapped.__enter__()
    def __exit__(self, *args):
        return self._wrapped.__exit__(*args)

class WrapFile(Wrapper):
    def write(self):
        return "Writing..."

f = WrapFile(open('wrap.py'))
print(f.readline())
with f:
    print(len(list(f)))
print(f.write())

>
> Answers for Python 3, thanks.

Tested on 3.2.


--
Oscar

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web