Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85753 > unrolled thread

What the Pythons docs means by "container" ?

Back to article view | Back to comp.lang.python

Contents

  What the Pythons docs means by "container" ? candide <c.candide@laposte.net> - 2015-02-17 13:21 -0800
    Re: What the Pythons docs means by "container" ? Cameron Simpson <cs@zip.com.au> - 2015-02-18 08:39 +1100
      Re: What the Pythons docs means by "container" ? candide <c.candide@laposte.net> - 2015-02-17 16:30 -0800
        Re: What the Pythons docs means by "container" ? Chris Angelico <rosuav@gmail.com> - 2015-02-18 11:49 +1100
          Re: What the Pythons docs means by "container" ? candide <pascal.ortiz@gmail.com> - 2015-02-17 18:14 -0800
            Re: What the Pythons docs means by "container" ? MRAB <python@mrabarnett.plus.com> - 2015-02-18 18:04 +0000
            Re: What the Pythons docs means by "container" ? Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-18 11:50 -0700
            Re: What the Pythons docs means by "container" ? Ethan Furman <ethan@stoneleaf.us> - 2015-02-18 11:43 -0800
            Re: What the Pythons docs means by "container" ? Terry Reedy <tjreedy@udel.edu> - 2015-02-18 15:04 -0500
              Re: What the Pythons docs means by "container" ? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-19 10:03 +1100
                Re: What the Pythons docs means by "container" ? Ben Finney <ben+python@benfinney.id.au> - 2015-02-19 11:05 +1100
                Re: What the Pythons docs means by "container" ? Chris Angelico <rosuav@gmail.com> - 2015-02-19 11:33 +1100
    Re: What the Pythons docs means by "container" ? Terry Reedy <tjreedy@udel.edu> - 2015-02-17 19:54 -0500
    Re: What the Pythons docs means by "container" ? perfectican <perfectican@gmail.com> - 2015-02-19 02:59 -0800
      Re: What the Pythons docs means by "container" ? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-19 11:20 +0000
    Re: What the Pythons docs means by "container" ? Rustom Mody <rustompmody@gmail.com> - 2015-02-20 03:37 -0800
      Re: What the Pythons docs means by "container" ? Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-20 11:38 -0700

#85753 — What the Pythons docs means by "container" ?

Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains :

--------------------------
Some objects contain references to other objects; these are called containers.
--------------------------

So containers contain : what a great definition!

To be more precise, the "container" wording suggests a data structure _storing_ items somewhere and that the number of items in the container has a memory footprint. This is exacly the "container" definition given by the ISO C++ standard (cf.  http://www.open-std.org/jtc1/sc22/open/n2356/lib-containers.html) :

--------------------------
Containers are objects that store other objects.
--------------------------

So I was considering a Python range object NOT being a container: indeed, range(10**6) does NOT store 10**6 integers and range(10**6) has a small memory footprint in the contrary to the associated list :

----------------------------------------------
>>> from sys import getsizeof
>>> r = range(10**6)
>>> L = list(r)
>>> getsizeof(r)
24
>>> getsizeof(L)
4500056
>>> 
----------------------------------------------

Then, mining the official docs, I realized that the collections module provides an ABC Container class allowing you to know if a given object is a container or not. And what a surprise, a range object IS a container !

----------------------------------------------
>>> from collections import Container
>>> r = range(10**6)
>>> isinstance(r, Container)                                                            
True                                                                                    
>>>
----------------------------------------------

So, what documentation means by the generic term of "container"? 

I agree with the existence of built-in containers (list, dict and the like) and the existence of special containers provided by the collections module (most of them inheriting from a built-in container) but I can't find a precise definition explaining what is a "container object" or a "container type". 

The docs at :

https://docs.python.org/3.2/reference/datamodel.html#object.__contains__

explains that a container is an object compatible with the membership test (in and not in operators). So a file is a container ? recall a file supports the in operator :

----------------------------------------------
$ touch my_test.txt
$ python3.2
Python 3.2.3 (default, Feb 28 2014, 00:22:33) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 42 in open("my_test.txt")
False
>>> from collections import Container
>>> isinstance(open("my_test.txt"), Container)
False
>>> 
----------------------------------------------




Reference of interest :

http://blog.wachowicz.eu/?p=132
http://stackoverflow.com/questions/11575925/what-exactly-are-containers-in-python-and-what-are-all-the-python-container

[toc] | [next] | [standalone]

#85754

On 17Feb2015 13:21, candide <c.candide@laposte.net> wrote:
>Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains :
>--------------------------
>Some objects contain references to other objects; these are called containers.
>--------------------------

And this is the basic thing, as you understand.

>So containers contain : what a great definition!

Shrug. It is a useful descriptive term when you want to talk about things. You 
might keep in mind that a "container"'s _primary_ purpose is usually its 
containing function. So a list. But not so some arbitrary object, even though 
most objects have references to other objects.

[...]
>So I was considering a Python range object NOT being a container: indeed, range(10**6) does NOT store 10**6 integers and range(10**6) has a small memory footprint
[...]
>Then, mining the official docs, I realized that the collections module provides an ABC Container class allowing you to know if a given object is a container or not. And what a surprise, a range object IS a container !

You might find it duck types like other containers.

[...]
>The docs at :
>https://docs.python.org/3.2/reference/datamodel.html#object.__contains__
>explains that a container is an object compatible with the membership test (in and not in operators).

Duck typing again: this bases the term on behaviour, not memory use.

>So a file is a container ? recall a file supports the in operator :

Yes and no. A file can be iterated. "in" on a file reads the file as a side 
effect of iterating over it. A generator also supports "in", again by 
iteration.

Neither of these things has a .__contains__ method; the "in" operator falls 
back to iteration if there is no .__contains__.

A container usually lets you test "in" in a nondestructive/nonconsuming 
fashion. An iterable can be "in"ed, but it will be consumed. So iterables are 
not, of themselves, containers.

So, range? A range object knows its bounds and state; it can answer "in" 
without consuming the range iteration, so it is effectively a container.

Hoping this helps.

Cheers,
Cameron Simpson <cs@zip.com.au>

[Alain] had been looking at his dashboard, and had not seen me, so I
ran into him. - Jean Alesi on his qualifying prang at Imola '93

[toc] | [prev] | [next] | [standalone]

#85758

> 
> Shrug. It is a useful descriptive term when you want to talk about things. You

Python Language Referenceb (PLR) is expected to give formal and acurate definition, not "descriptive term to talk about things".

> might keep in mind that a "container"'s _primary_ purpose is usually its 
> containing function. So a list. But not so some arbitrary object, even though 

"containing function" ? Does PLR document this "containing function"?


> most objects have references to other objects.

have or contain ;)  ? 


 
> Duck typing again: this bases the term on behaviour, not memory use.

Again this is undocumented. Compare with the definition of duck-typed iterators, cf. https://docs.python.org/3.2/library/stdtypes.html#iterator-types

On the other hand, recall that the new behavior of range since Python 3 is motivated by memory considerations, quoted from the doc : 



--------------------
The advantage of the range type over a regular list or tuple is that a range object will always take the same (small) amount of memory, no matter the size of the range it represents (as it only stores the start, stop and step values, calculating individual items and subranges as needed).
--------------------


> Neither of these things has a .__contains__ method; the "in" operator falls 
> back to iteration if there is no .__contains__.


The documentation is fuzzy and inconsistant here: containers are supposed to implement the membership test, dixit the doc. 


> 
> A container usually lets you test "in" in a nondestructive/nonconsuming

"usually" ? I need something more acurate ;)


> fashion. An iterable can be "in"ed, but it will be consumed. So iterables are 
> not, of themselves, containers.

First this is undocumented. Second, range objects are considered as iterable by the doc. And you are telling us that an iterable is not a container. And range has a __contains__ method.

> 
> So, range? A range object knows its bounds and state; it can answer "in" 
> without consuming the range iteration, so it is effectively a container.

Again, this no-consuming feature is completely undocumented. And unconvincing: do you regard a linked list as not being a container ?

[toc] | [prev] | [next] | [standalone]

#85760

On Wed, Feb 18, 2015 at 11:30 AM, candide <c.candide@laposte.net> wrote:
> Python Language Referenceb (PLR) is expected to give formal and acurate definition, not "descriptive term to talk about things".

In a lot of ways, technically accurate definitions can be useless,
where sloppy but descriptive terms are useful. For instance, we can
distinguish between an integer (doesn't contain things) and a list
(does contain things), and then categorize sets, tuples, and
SimpleNamespace()s with the latter, and exceptions, strings, and
functions with the former. But technically, an exception has a
reference to something else (a traceback), and could thus be said to
contain it, and a function always has a reference to its global
namespace (which itself contains that function), etc, so any specific
and formal definition is likely to be quite useless. Also, a string
can be said to "contain" a whole lot of substrings - "bcd" in "abcde"
is True - even though those substrings don't actually have to exist as
objects, so it's impossible for the string to retain references to
those substrings. Even worse, a string always contains *itself*, which
makes a definition based on the 'in' operator conflict entirely with a
definition based on a real-world analogy involving a box with stuff in
it.

So, what's a container? It's a thing that you put other objects into.
That's all the definition you really need, most of the time. If you
want a better definition, you first need to ask yourself: What aspect
of "containerness" matters here? Why do you need to distinguish
container from non-container? If you want to know whether the 'in'
operator can be used, then ask whether the 'in' operator can be used.
If you want to know whether it's possible for this object to be part
of a reference cycle, ask whether it's possible for this object to be
part of a reference cycle. If you want to know whether this is a
useful way to return multiple results from a function, then ... you
get the idea. They're all very different aspects of containers, and
not all containers have all those aspects. (For instance, a tuple
can't [1] have a reference to itself. You can't get a reference cycle
without having some other type of object, such as a list, involved.)
"Is this a container?" isn't often a useful question to ask.

ChrisA

[1] You can use C code to create a tuple containing itself, but in C,
you're most welcome to shoot yourself in the foot in other ways, too.

[toc] | [prev] | [next] | [standalone]

#85764

Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :

> 
> So, what's a container? It's a thing that you put other objects into.

I agree with this approach. The important point to consider here is the last word in your definition : "into". There is the container and there is the content (the objects into). The so-called built-in containers (list, string, etc) are in conformance with this view. Now, regarding a range object as a container disagrees completely with the definition given in the PLR : there is no contents and hence there is no container. For instance, range(10**6) doesn't hold any kind of object, there are no reference to the int objects 0, 1, 2, ... As the range's docstring explains, range returns a VIRTUAL sequence.

[toc] | [prev] | [next] | [standalone]

#85803

On 2015-02-18 02:14, candide wrote:
> Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :
>
>> So, what's a container? It's a thing that you put other objects
>> into.
>
> I agree with this approach. The important point to consider here is
> the last word in your definition : "into". There is the container and
> there is the content (the objects into). The so-called built-in
> containers (list, string, etc) are in conformance with this view.
> Now, regarding a range object as a container disagrees completely
> with the definition given in the PLR : there is no contents and hence
> there is no container. For instance, range(10**6) doesn't hold any
> kind of object, there are no reference to the int objects 0, 1, 2,
> ... As the range's docstring explains, range returns a VIRTUAL
> sequence.
>
It's a virtual, read-only container that contains integers.

Try comparing range(10) with tuple(range(10)). Both contain integers.
Both have a length. Both can be indexed.

[toc] | [prev] | [next] | [standalone]

#85808

On Wed, Feb 18, 2015 at 11:04 AM, MRAB <python@mrabarnett.plus.com> wrote:
> On 2015-02-18 02:14, candide wrote:
>>
>> Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :
>>
>>> So, what's a container? It's a thing that you put other objects
>>> into.
>>
>>
>> I agree with this approach. The important point to consider here is
>> the last word in your definition : "into". There is the container and
>> there is the content (the objects into). The so-called built-in
>> containers (list, string, etc) are in conformance with this view.
>> Now, regarding a range object as a container disagrees completely
>> with the definition given in the PLR : there is no contents and hence
>> there is no container. For instance, range(10**6) doesn't hold any
>> kind of object, there are no reference to the int objects 0, 1, 2,
>> ... As the range's docstring explains, range returns a VIRTUAL
>> sequence.
>>
> It's a virtual, read-only container that contains integers.
>
> Try comparing range(10) with tuple(range(10)). Both contain integers.
> Both have a length. Both can be indexed.

Another way to look at range is as an optimization of the range from
Python 2, which returned an actual list of integers. Most callers are
only interested in iterating over the container and don't have to be
concerned whether the result is actually a list or just something that
looks enough like one for their purposes. From that point of view, a
range object is still a container.

[toc] | [prev] | [next] | [standalone]

#85812

[Multipart message — attachments visible in raw view] — view raw

On 02/17/2015 06:14 PM, candide wrote:
> Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :
> 
>>
>> So, what's a container? It's a thing that you put other objects into.
> 
> I agree with this approach. The important point to consider here is the last word in your definition : "into".

You are getting bogged down with implementation details; not even dict, list, nor tuple actually have the contained
objects inside their memory space.  What they have is a record of what objects they "contain", and methods to work with
those objects.

Whether a "contained" object exists before it is accessed is irrelevant, is an implementation detail, and is a level of
optimization.

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#85816

On 2/18/2015 2:43 PM, Ethan Furman wrote:

> Whether a "contained" object exists before it is accessed is irrelevant, is an implementation detail, and is a level of
> optimization.

Is in not irrelevant in that virtual collections can be infinite.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#85832

Terry Reedy wrote:

> On 2/18/2015 2:43 PM, Ethan Furman wrote:
> 
>> Whether a "contained" object exists before it is accessed is irrelevant,
>> is an implementation detail, and is a level of optimization.
> 
> Is in not irrelevant in that virtual collections can be infinite.

Hmmm. I'm not sure I believe that. Can you enumerate all of the items and
see if there actually is an infinite number of them? I'll wait... 

*wink*

But seriously... Ethan's comment shouldn't be interpreted as meaning that
virtual containers are exactly the same as concrete ones. Ethan's comment
should be understand in terms that whether something is a container or not
is *not* connected to whether it is a concrete data structure like a list
or an array, or a virtual/lazy/on-demand data structure like (x)range.

In Python ABC terms, I believe that the, or at least a, defining
characteristic of containers is that they implement the "in" operator. If
there is some value X for which `X in OBJ` succeeds, then OBJ may be a
container.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85837

Steven D'Aprano <steve+comp.lang.python@pearwood.info> writes:

> Terry Reedy wrote:
>
> > […] virtual collections can be infinite.
>
> Hmmm. I'm not sure I believe that. Can you enumerate all of the items
> and see if there actually is an infinite number of them? I'll wait...

A mathematician should know better: you don't need to enumerate to
demonstrate a collection is infinite.

Provided, of course, that the collection is *virtual* — so virtual that
it's not actually implemented on hardware :-)

-- 
 \        “If you have the facts on your side, pound the facts. If you |
  `\     have the law on your side, pound the law. If you have neither |
_o__)                       on your side, pound the table.” —anonymous |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#85838

On Thu, Feb 19, 2015 at 11:05 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
>> Hmmm. I'm not sure I believe that. Can you enumerate all of the items
>> and see if there actually is an infinite number of them? I'll wait...
>
> A mathematician should know better: you don't need to enumerate to
> demonstrate a collection is infinite.

But you may have to demonstrate a 1:1 correspondence with counting
numbers, to prove that it's countably infinite. And that's pretty much
what the enumerate() function does, so that's enumeration.... right?

Bizarre pseudo-logic for the win.

ChrisA

[toc] | [prev] | [next] | [standalone]

#85761

On 2/17/2015 4:21 PM, candide wrote:
> Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains :

I use 'collection' rather than 'container'.

> --------------------------
> Some objects contain references to other objects; these are called containers.
> --------------------------

A mathematical set has 'members', like a club.  Objects, like people, 
can be members of 0 to many clubs.  Containing 'references',  and the 
nature of said references is a computer implementation issue.  There are 
no 'references' in math.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#85896

in duck-typing, any objects can acts as Container. but only difference is type that we need to consider as a Container or not.

[toc] | [prev] | [next] | [standalone]

#85898

On 19/02/2015 10:59, perfectican wrote:
> in duck-typing, any objects can acts as Container. but only difference is type that we need to consider as a Container or not.
>

This comment makes no sense to me at all.  Could you please rephrase it, 
preferably supplying some context at the same time.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#85965

On Wednesday, February 18, 2015 at 2:51:34 AM UTC+5:30, candide wrote:
> Official Python documentation very frequently invokes a mysterious *container* data structure. The PLR manual explains :
> 
> --------------------------
> Some objects contain references to other objects; these are called containers.
> --------------------------
> 
> So containers contain : what a great definition!
> 
> To be more precise, the "container" wording suggests a data structure _storing_ items somewhere and that the number of items in the container has a memory footprint. This is exacly the "container" definition given by the ISO C++ standard (cf.  http://www.open-std.org/jtc1/sc22/open/n2356/lib-containers.html) :
> 
> --------------------------
> Containers are objects that store other objects.
> --------------------------
> 
> So I was considering a Python range object NOT being a container: indeed, range(10**6) does NOT store 10**6 integers and range(10**6) has a small memory footprint in the contrary to the associated list :
> 
> ----------------------------------------------
> >>> from sys import getsizeof
> >>> r = range(10**6)
> >>> L = list(r)
> >>> getsizeof(r)
> 24
> >>> getsizeof(L)
> 4500056
> >>> 
> ----------------------------------------------
> 
> Then, mining the official docs, I realized that the collections module provides an ABC Container class allowing you to know if a given object is a container or not. And what a surprise, a range object IS a container !
> 
> ----------------------------------------------
> >>> from collections import Container
> >>> r = range(10**6)
> >>> isinstance(r, Container)                                                            
> True                                                                                    
> >>>
> ----------------------------------------------
> 
> So, what documentation means by the generic term of "container"? 
> 
> I agree with the existence of built-in containers (list, dict and the like) and the existence of special containers provided by the collections module (most of them inheriting from a built-in container) but I can't find a precise definition explaining what is a "container object" or a "container type". 
> 
> The docs at :
> 
> https://docs.python.org/3.2/reference/datamodel.html#object.__contains__
> 
> explains that a container is an object compatible with the membership test (in and not in operators). So a file is a container ? recall a file supports the in operator :
> 
> ----------------------------------------------
> $ touch my_test.txt
> $ python3.2
> Python 3.2.3 (default, Feb 28 2014, 00:22:33) 
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> 42 in open("my_test.txt")
> False
> >>> from collections import Container
> >>> isinstance(open("my_test.txt"), Container)
> False
> >>> 
> ----------------------------------------------

+1

See https://www.python.org/dev/peps/pep-0246/

There Guido says "something much better is about to happen"

Best as I know its not happened yet...

[Personal note: I am currently trying to teach python
and the lack of a standardized place for all protocols (in python's duck typing sense)
is quite a slow-down-er for me and my class
]

[toc] | [prev] | [next] | [standalone]

#85987

On Fri, Feb 20, 2015 at 4:37 AM, Rustom Mody <rustompmody@gmail.com> wrote:
> See https://www.python.org/dev/peps/pep-0246/
>
> There Guido says "something much better is about to happen"
>
> Best as I know its not happened yet...

Well, since that PEP was rejected we now have ABCs. Although PEP 3119
wasn't written until a couple of years later, so who knows if that's
what he had in mind.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web