Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #100982 > unrolled thread

how to get names of attributes

Back to article view | Back to comp.lang.python

Contents

  how to get names of attributes "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2015-12-30 11:51 +0000
    Re: how to get names of attributes Chris Angelico <rosuav@gmail.com> - 2015-12-30 22:58 +1100
    Re: how to get names of attributes "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2015-12-30 12:16 +0000
      Re: how to get names of attributes Chris Angelico <rosuav@gmail.com> - 2015-12-30 23:34 +1100
    Re: how to get names of attributes "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2015-12-30 12:40 +0000
      Re: how to get names of attributes Chris Angelico <rosuav@gmail.com> - 2015-12-30 23:50 +1100
        Re: how to get names of attributes "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2015-12-30 13:31 +0000
          Re: how to get names of attributes Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-12-30 14:16 +0000
          Re: how to get names of attributes Chris Angelico <rosuav@gmail.com> - 2015-12-31 00:45 +1100
      Re: how to get names of attributes Random832 <random832@fastmail.com> - 2015-12-30 12:04 -0500
      Re: how to get names of attributes Chris Angelico <rosuav@gmail.com> - 2015-12-31 09:26 +1100
    Re: how to get names of attributes Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-12-30 14:10 +0000
      Re: how to get names of attributes "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2015-12-30 14:50 +0000
    Re: how to get names of attributes Steven D'Aprano <steve@pearwood.info> - 2015-12-31 10:58 +1100
      Re: how to get names of attributes "Charles T. Smith" <cts.private.yahoo@gmail.com> - 2015-12-31 10:46 +0000

#100982 — how to get names of attributes

Hi,

How can I get *all* the names of an object's attributes?  I have legacy 
code with mixed new style classes and old style classes and I need to 
write methods which deal with both.  That's the immediate problem, but 
I'm always running into the need to understand how objects are linked, in 
particular when in pdb.  The answers one always sees on StackOverflow is 
that you don't need to understand, understanding is not the pythonic way 
to do things.

Alternatively, is there are map documented somewhere - more complete than
python/python-2.7.3-docs-html/library/stdtypes.html?
highlight=class#special-attributes

Or, is the code available uncompiled somewhere on my machine?

Does anyone know *why* the __members__ method was deprecated, to be 
replaced by dir(), which doesn't tell the truth (if only it took an 
optional parameter to say: "be truthful")

cts

[toc] | [next] | [standalone]

#100983

On Wed, Dec 30, 2015 at 10:51 PM, Charles T. Smith
<cts.private.yahoo@gmail.com> wrote:
> Does anyone know *why* the __members__ method was deprecated, to be
> replaced by dir(), which doesn't tell the truth (if only it took an
> optional parameter to say: "be truthful")

Does vars() help here? It works on old-style and new-style classes,
and it's doing broadly the same sort of thing as you're talking about.

ChrisA

[toc] | [prev] | [next] | [standalone]

#100984

On Wed, 30 Dec 2015 11:51:19 +0000, Charles T. Smith wrote:

> Hi,
> 
> How can I get *all* the names of an object's attributes?  I have legacy
> code with mixed new style classes and old style classes and I need to
> write methods which deal with both.  That's the immediate problem, but
> I'm always running into the need to understand how objects are linked,
> in particular when in pdb.  The answers one always sees on StackOverflow
> is that you don't need to understand, understanding is not the pythonic
> way to do things.
> 
> Alternatively, is there are map documented somewhere - more complete
> than python/python-2.7.3-docs-html/library/stdtypes.html?
> highlight=class#special-attributes
> 
> Or, is the code available uncompiled somewhere on my machine?
> 
> Does anyone know *why* the __members__ method was deprecated, to be
> replaced by dir(), which doesn't tell the truth (if only it took an
> optional parameter to say: "be truthful")
> 
> cts


For example:

    (PDB)pp dir   (newclass.__class__)
    ['__class__',
     '__delattr__',
     '__dict__',
     '__doc__',
     '__format__',
     '__getattribute__',
     '__hash__',
     '__init__',
     '__module__',
     '__new__',
     '__reduce__',
     '__reduce_ex__',
     '__repr__',
     '__setattr__',
     '__sizeof__',
     '__str__',
     '__subclasshook__',
     '__weakref__',
     'm2']

    (PDB)pp dir   (oldclass.__class__)
    ['__doc__', '__module__', 'm3']

    (PDB)pp    (oldclass.__class__.__name__)
    'C3'

    (PDB)pp    (newclass.__class__.__name__)
    'C2'

Both dir() invocations are lying to me.  The old-style class even ignores 
the pretty-print command.

I'm glad I discovered __mro__(), but how can I do the same thing for old-
style classes?

[toc] | [prev] | [next] | [standalone]

#100985

On Wed, Dec 30, 2015 at 11:16 PM, Charles T. Smith
<cts.private.yahoo@gmail.com> wrote:
> I'm glad I discovered __mro__(), but how can I do the same thing for old-
> style classes?

You should be able to track through __bases__ and use vars() at every level:

>>> class X: pass
...
>>> class Y(X): pass
...
>>> class Z(Y): pass
...
>>> X.x=1
>>> Y.y=2
>>> Z.z=3
>>> inst=Z()
>>> inst.i=4
>>> def class_vars(old_style_class):
...     v = {}
...     for cls in old_style_class.__bases__:
...         v.update(class_vars(cls))
...     v.update(vars(old_style_class))
...     return v
...
>>> def all_vars(old_style_inst):
...     v = class_vars(old_style_inst.__class__)
...     v.update(vars(old_style_inst))
...     return v
...
>>> all_vars(inst)
{'i': 4, '__module__': '__main__', 'y': 2, 'x': 1, 'z': 3, '__doc__': None}

I'm not 100% sure I've matched the MRO here, but if all you want is
the complete set of attribute names, this should work - I think.

ChrisA

[toc] | [prev] | [next] | [standalone]

#100986

On Wed, 30 Dec 2015 11:51:19 +0000, Charles T. Smith wrote:

> Hi,
> 
> How can I get *all* the names of an object's attributes?  I have legacy
> code with mixed new style classes and old style classes and I need to
> write methods which deal with both.  That's the immediate problem, but
> I'm always running into the need to understand how objects are linked,
> in particular when in pdb.  The answers one always sees on StackOverflow
> is that you don't need to understand, understanding is not the pythonic
> way to do things.
> 
> Alternatively, is there are map documented somewhere - more complete
> than python/python-2.7.3-docs-html/library/stdtypes.html?
> highlight=class#special-attributes
> 
> Or, is the code available uncompiled somewhere on my machine?
> 
> Does anyone know *why* the __members__ method was deprecated, to be
> replaced by dir(), which doesn't tell the truth (if only it took an
> optional parameter to say: "be truthful")
> 
> cts


Oh!

Although the referenced doc says:

  "For compatibility reasons, classes are still old-style by default."

is it true that dictionaries are by default always new-style objects?

  (PDB)c6 = { "abc" : 123, "def" : 456}

  (PDB)isinstance (c6, dict)
  True

  (PDB)isinstance (c6, object)
  True

[toc] | [prev] | [next] | [standalone]

#100987

On Wed, Dec 30, 2015 at 11:40 PM, Charles T. Smith
<cts.private.yahoo@gmail.com> wrote:
> Oh!
>
> Although the referenced doc says:
>
>   "For compatibility reasons, classes are still old-style by default."
>
> is it true that dictionaries are by default always new-style objects?
>
>   (PDB)c6 = { "abc" : 123, "def" : 456}
>
>   (PDB)isinstance (c6, dict)
>   True
>
>   (PDB)isinstance (c6, object)
>   True

I believe that's true, yes. The meaning of "by default" there is that
"class X: pass" will make an old-style class. All built-in types are
now new-style classes.

ChrisA

[toc] | [prev] | [next] | [standalone]

#100993

On Wed, 30 Dec 2015 23:50:03 +1100, Chris Angelico wrote:

> On Wed, Dec 30, 2015 at 11:40 PM, Charles T. Smith
> <cts.private.yahoo@gmail.com> wrote:
>> Oh!
>>
>> Although the referenced doc says:
>>
>>   "For compatibility reasons, classes are still old-style by default."
>>
>> is it true that dictionaries are by default always new-style objects?
>>
>>   (PDB)c6 = { "abc" : 123, "def" : 456}
>>
>>   (PDB)isinstance (c6, dict)
>>   True
>>
>>   (PDB)isinstance (c6, object)
>>   True
> 
> I believe that's true, yes. The meaning of "by default" there is that
> "class X: pass" will make an old-style class. All built-in types are now
> new-style classes.
> 
> ChrisA


Okay, thank you.  I'm trying to understand your program.

Unfortunately, I haven't gotten the same output you had, using python 2.6 
or 2.7.  Maybe I haven't been able to restore the indentation correctly 
after having been filtered through pan(1).

I wonder what the difference is between vars() and items()

[toc] | [prev] | [next] | [standalone]

#100995

On 30/12/2015 13:31, Charles T. Smith wrote:
>
> I wonder what the difference is between vars() and items()
>

Not much.  From https://docs.python.org/3/library/functions.html#vars

<quote>
vars([object])

     Return the __dict__ attribute for a module, class, instance, or any 
other object with a __dict__ attribute.

     Objects such as modules and instances have an updateable __dict__ 
attribute; however, other objects may have write restrictions on their 
__dict__ attributes (for example, classes use a dictproxy to prevent 
direct dictionary updates).

     Without an argument, vars() acts like locals(). Note, the locals 
dictionary is only useful for reads since updates to the locals 
dictionary are ignored.
</quote>

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#100997

On Thu, Dec 31, 2015 at 12:31 AM, Charles T. Smith
<cts.private.yahoo@gmail.com> wrote:
> Okay, thank you.  I'm trying to understand your program.
>
> Unfortunately, I haven't gotten the same output you had, using python 2.6
> or 2.7.  Maybe I haven't been able to restore the indentation correctly
> after having been filtered through pan(1).
>
> I wonder what the difference is between vars() and items()

What I sent you was a log of interactive Python, so there are prompts
and continuation prompts. Here's a script version of the same thing
(running under CPython 2.7):

class X: pass

class Y(X): pass

class Z(Y): pass

X.x=1
Y.y=2
Z.z=3
inst=Z()
inst.i=4
def class_vars(old_style_class):
    v = {}
    for cls in old_style_class.__bases__:
        v.update(class_vars(cls))
    v.update(vars(old_style_class))
    return v

def all_vars(old_style_inst):
    v = class_vars(old_style_inst.__class__)
    v.update(vars(old_style_inst))
    return v

print(all_vars(inst))
# {'i': 4, '__module__': '__main__', 'y': 2, 'x': 1, 'z': 3, '__doc__': None}

Does that work better?

ChrisA

[toc] | [prev] | [next] | [standalone]

#101003

On Wed, Dec 30, 2015, at 07:50, Chris Angelico wrote:
> I believe that's true, yes. The meaning of "by default" there is that
> "class X: pass" will make an old-style class. All built-in types are
> now new-style classes.

To be clear, AFAIK, built-in types were never old-style classes - prior
to the introduction of the new type system (i.e. in Python 2.1 and
earlier) they were not classes, and afterwards they were immediately
new-style classes.

[toc] | [prev] | [next] | [standalone]

#101014

On Thu, Dec 31, 2015 at 4:04 AM, Random832 <random832@fastmail.com> wrote:
> On Wed, Dec 30, 2015, at 07:50, Chris Angelico wrote:
>> I believe that's true, yes. The meaning of "by default" there is that
>> "class X: pass" will make an old-style class. All built-in types are
>> now new-style classes.
>
> To be clear, AFAIK, built-in types were never old-style classes - prior
> to the introduction of the new type system (i.e. in Python 2.1 and
> earlier) they were not classes, and afterwards they were immediately
> new-style classes.
> --
> https://mail.python.org/mailman/listinfo/python-list

Thanks Random. I wasn't actively using Python until about 2.5ish, and
wasn't following python-dev until even more recently, so I didn't keep
track of all that. And frankly, old-style classes have just been
something I avoid whereever possible.

ChrisA

[toc] | [prev] | [next] | [standalone]

#100994

On 30/12/2015 11:51, Charles T. Smith wrote:
> Hi,
>
> Does anyone know *why* the __members__ method was deprecated, to be
> replaced by dir(), which doesn't tell the truth (if only it took an
> optional parameter to say: "be truthful")

https://bugs.python.org/issue456420
https://bugs.python.org/issue449989

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#101000

On Wed, 30 Dec 2015 14:10:14 +0000, Mark Lawrence wrote:

> On 30/12/2015 11:51, Charles T. Smith wrote:
>> Hi,
>>
>> Does anyone know *why* the __members__ method was deprecated, to be
>> replaced by dir(), which doesn't tell the truth (if only it took an
>> optional parameter to say: "be truthful")
> 
> https://bugs.python.org/issue456420
> https://bugs.python.org/issue449989

Thank you, that was what I asked for, kinda.

According to the experts, the reason that the __members__() method was
deprecated is that it was an "ugly hack".  *Why* it was an ugly hack
wasn't made clear (to me), unfortunately.

My other postings to this topic show, I think, why the ability to know
what's available (in the way of attributes) is a good thing, as I
try to figure out how to get at my attributes without triggering
__getattr__(), which I'm implementing.

[toc] | [prev] | [next] | [standalone]

#101026

On Wed, 30 Dec 2015 10:51 pm, Charles T. Smith wrote:

> Hi,
> 
> How can I get *all* the names of an object's attributes?  

In the most general case, you cannot.

Classes can define a __getattr__ method (and a __getattribute__ method, for
new-style classes only) which implement dynamic attributes. These can be
*extremely* dynamic and impossible to predict ahead of time. Starting from
the simplest cases to the most horrible:

    def __getattr__(self, name):
        if name == "spam":
            return 1
        elif name == self._attribute_name:
            return 2
        elif name == some_function(10, 20, 30):
            return 3
        elif name.lower() in ("x", "y") or name.startswith("foo"):
            return 4
        elif 1626740500 <= hash(name) <= 1626740600:
            return 5
        elif name == "surprise" and random.random() < 0.5:
            return 6
        raise AttributeError


So you can see that even in principle, there is no way for the Python
interpreter to look inside the __getattr__ method and determine what
attributes exist.

Fortunately, there's a way around that: you can customise the list of
attribute names returned by dir():

py> class X:
...     def __dir__(self):
...             return dir(X) + ["a", "b"]
...     def spam(self):
...             pass
...
py> x = X()
py> dir(x)
['__dir__', '__doc__', '__module__', 'a', 'b', 'spam']


So if you have dynamic attributes generated by __getattr__, the polite thing
to do is to return their names from __dir__.



> I have legacy 
> code with mixed new style classes and old style classes and I need to
> write methods which deal with both.  That's the immediate problem, but
> I'm always running into the need to understand how objects are linked, in
> particular when in pdb.  The answers one always sees on StackOverflow is
> that you don't need to understand, understanding is not the pythonic way
> to do things.

That's why I don't think much of the majority of StackOverflow answers.


> Alternatively, is there are map documented somewhere - more complete than
> python/python-2.7.3-docs-html/library/stdtypes.html?
> highlight=class#special-attributes
> 
> Or, is the code available uncompiled somewhere on my machine?

That depends on how you installed Python. If you installed it from source,
then it will be, unless you deleted it after compiling. But it is easy
enough to get the Python source code:

https://www.python.org/downloads/source/

https://docs.python.org/devguide/setup.html#checkout

https://hg.python.org/cpython/file/tip


> Does anyone know *why* the __members__ method was deprecated, to be
> replaced by dir(), which doesn't tell the truth (if only it took an
> optional parameter to say: "be truthful")

dir() is intentionally meant for use in the interactive interpreter, to
return "interesting" attributes. It isn't really intended for programmatic
use.

For that, you might wish to write your own version of dir(). Here is some
pseudo-code:


def mydir(obj):
    if type(obj) defines __dir__, return the output of __dir__
    names = set()
    if obj is an instance:
        if obj has __slots__, then add each slot to names;
        try:
            add each key from vars(obj) to names;
        except TypeError:
            # no instance __dict__
            pass
        call mydir(type(obj)) and add the names it returns to names;
    else obj must be a class or type:
        add each key from vars(obj) to names;
        if obj is an old-style class:
            do the same for each class in obj.__bases__;
        else obj must be a new-style class:
            do the same for each class in obj.__mro__
    return sorted(names)

But I expect that the differences between this and what the built-in dir()
return will be minimal, and only attributes which are part of the machinery
used to get classes themselves working, not part of your class or instance.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#101047

On Thu, 31 Dec 2015 10:58:17 +1100, Steven D'Aprano wrote:

  (some very good information)

Thank you.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web