Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #106254 > unrolled thread

Drowning in a teacup?

Back to article view | Back to comp.lang.python

Contents

  Drowning in a teacup? Fillmore <fillmore_remove@hotmail.com> - 2016-04-01 16:27 -0400
    Re: Drowning in a teacup? Rob Gaddi <rgaddi@highlandtechnology.invalid> - 2016-04-01 20:39 +0000
      Re: Drowning in a teacup? Ian Kelly <ian.g.kelly@gmail.com> - 2016-04-01 14:52 -0600
    Re: Drowning in a teacup? MRAB <python@mrabarnett.plus.com> - 2016-04-01 21:45 +0100
    Re: Drowning in a teacup? Michael Selik <michael.selik@gmail.com> - 2016-04-01 20:46 +0000
    Re: Drowning in a teacup? Steven D'Aprano <steve@pearwood.info> - 2016-04-02 09:46 +1100
    Re: Drowning in a teacup? Fillmore <fillmore_remove@hotmail.com> - 2016-04-02 00:03 -0400
    Re: Drowning in a teacup? Vito De Tullio <vito.detullio@gmail.com> - 2016-04-02 07:45 +0200
    Re: Drowning in a teacup? Michael Selik <michael.selik@gmail.com> - 2016-04-02 05:51 +0000
    Re: Drowning in a teacup? Vito De Tullio <vito.detullio@gmail.com> - 2016-04-02 08:26 +0200
    Re: Drowning in a teacup? Peter Otten <__peter__@web.de> - 2016-04-02 12:31 +0200
    Re: Drowning in a teacup? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2016-04-02 12:53 +0100
    Re: Drowning in a teacup? Michael Selik <michael.selik@gmail.com> - 2016-04-02 14:36 +0000
    Re: Drowning in a teacup? Ned Batchelder <ned@nedbatchelder.com> - 2016-04-02 12:28 -0700
      Re: Drowning in a teacup? Random832 <random832@fastmail.com> - 2016-04-02 15:54 -0400
        Re: Drowning in a teacup? Steven D'Aprano <steve@pearwood.info> - 2016-04-03 12:33 +1000
      Re: Drowning in a teacup? Ethan Furman <ethan@stoneleaf.us> - 2016-04-02 16:15 -0700
      Re: Drowning in a teacup? Random832 <random832@fastmail.com> - 2016-04-02 19:19 -0400

#106254 — Drowning in a teacup?

notorious pass by reference vs pass by value biting me in the backside 
here. Proceeding in order.

I need to scan a list of strings. If one of the elements matches the 
beginning of a search keyword, that element needs to snap to the front 
of the list.
I achieved that this way:


    for i in range(len(mylist)):
         if(mylist[i].startswith(key)):
             mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]

Since I need this code in multiple places, I placed it inside a function

def bringOrderStringToFront(mylist, key):

     for i in range(len(mylist)):
         if(mylist[i].startswith(key)):
             mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]

and called it this way:

  if orderstring:
      bringOrderStringToFront(Tokens, orderstring)

right?
Nope, wrong! contrary to what I thought I had understood about how 
parameters are passed in Python, the function is acting on a copy(!) and 
my original list is unchanged.

I fixed it this way:

def bringOrderStringToFront(mylist, key):

     for i in range(len(mylist)):
         if(mylist[i].startswith(key)):
             mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
     return(mylist)

and:

if orderstring:
    Tokens = bringOrderStringToFront(Tokens, orderstring)

but I'm left with a sour taste of not understanding what I was doing 
wrong. Can anyone elaborate? what's the pythonista way to do it right?

Thanks

[toc] | [next] | [standalone]

#106259

Fillmore wrote:

>
> notorious pass by reference vs pass by value biting me in the backside 
> here. Proceeding in order.
>
> I need to scan a list of strings. If one of the elements matches the 
> beginning of a search keyword, that element needs to snap to the front 
> of the list.
> I achieved that this way:
>
>
>     for i in range(len(mylist)):
>          if(mylist[i].startswith(key)):
>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> Since I need this code in multiple places, I placed it inside a function
>
> def bringOrderStringToFront(mylist, key):
>
>      for i in range(len(mylist)):
>          if(mylist[i].startswith(key)):
>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> and called it this way:
>
>   if orderstring:
>       bringOrderStringToFront(Tokens, orderstring)
>
> right?
> Nope, wrong! contrary to what I thought I had understood about how 
> parameters are passed in Python, the function is acting on a copy(!) and 
> my original list is unchanged.
>

Nope, that's not your problem.  Your problem is the line:

>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]

Which should be read as: "Take mylist[i], all of mylist before i,
and all of mylist after i and create a new list from it. Store that
new list in a variable called mylist, overwriting the previous value."

If instead you were using the .insert and .remove methods, you'd be
manipulating the existing list instead.  But you don't want to do that,
because those methods are O(N) on the list length; manipulating the
middle of lists is slow.

That said, you've got some logic problems too; which come from the fact
that your iteration is shooting at a moving target.

How about:
  newlist = (
    [x for x in mylist if x.startswith(key)] +
    [x for x in mylist if not x.startswith(key)]
  )
  return newlist

Or if you really insist on mutating the original list (which seems less
clean to me, but you do you), then:
  newlist = blahblahblah
  mylist[:] = newlist

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com

Email address domain is currently out of order.  See above to fix.

[toc] | [prev] | [next] | [standalone]

#106263

On Fri, Apr 1, 2016 at 2:39 PM, Rob Gaddi
<rgaddi@highlandtechnology.invalid> wrote:
> Fillmore wrote:
>> Nope, wrong! contrary to what I thought I had understood about how
>> parameters are passed in Python, the function is acting on a copy(!) and
>> my original list is unchanged.
>>
>
> Nope, that's not your problem.  Your problem is the line:
>
>>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> Which should be read as: "Take mylist[i], all of mylist before i,
> and all of mylist after i and create a new list from it. Store that
> new list in a variable called mylist, overwriting the previous value."
>
> If instead you were using the .insert and .remove methods, you'd be
> manipulating the existing list instead.  But you don't want to do that,
> because those methods are O(N) on the list length; manipulating the
> middle of lists is slow.

Or use slice assignment. This should work in place of the above:

    mylist[:] = [mylist[i]] + mylist[:i] + mylist[i+1:]

Still O(n), but so will be any implementation that removes something
from an arbitrary list position and inserts it at the front.

[toc] | [prev] | [next] | [standalone]

#106261

On 2016-04-01 21:27, Fillmore wrote:
>
> notorious pass by reference vs pass by value biting me in the backside
> here. Proceeding in order.
>
> I need to scan a list of strings. If one of the elements matches the
> beginning of a search keyword, that element needs to snap to the front
> of the list.
> I achieved that this way:
>
>
>      for i in range(len(mylist)):
>           if(mylist[i].startswith(key)):
>               mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> Since I need this code in multiple places, I placed it inside a function
>
> def bringOrderStringToFront(mylist, key):
>
>       for i in range(len(mylist)):
>           if(mylist[i].startswith(key)):
>               mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> and called it this way:
>
>    if orderstring:
>        bringOrderStringToFront(Tokens, orderstring)
>
> right?
> Nope, wrong! contrary to what I thought I had understood about how
> parameters are passed in Python, the function is acting on a copy(!) and
> my original list is unchanged.
>
> I fixed it this way:
>
> def bringOrderStringToFront(mylist, key):
>
>       for i in range(len(mylist)):
>           if(mylist[i].startswith(key)):
>               mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>       return(mylist)
>
> and:
>
> if orderstring:
>      Tokens = bringOrderStringToFront(Tokens, orderstring)
>
> but I'm left with a sour taste of not understanding what I was doing
> wrong. Can anyone elaborate? what's the pythonista way to do it right?
>
Python always passes a reference to the object, so the name "mylist" in 
the function is a local name that refers to the list that you passed.

When you say "mylist = something", you're just binding that local name 
to another object (i.e., it'll now refer to that object instead).

What you want to do it mutate the list itself. You can do that by 
replacing its elements with the new list you've created:

     mylist[:] = [mylist[i]] + mylist[:i] + mylist[i+1:]

[toc] | [prev] | [next] | [standalone]

#106262

Give this a shot

    def snap(keyword, words):
        matches = [i for i, s in enumerate(words) if s.startswith(keyword)]
        for i in matches:
            lst.insert(0, lst.pop(i))

Your current implementation is reassigning the local variable ``mylist`` to
a new list inside the function.

On Fri, Apr 1, 2016 at 4:30 PM Fillmore <fillmore_remove@hotmail.com> wrote:

>
> notorious pass by reference vs pass by value biting me in the backside
> here. Proceeding in order.
>
> I need to scan a list of strings. If one of the elements matches the
> beginning of a search keyword, that element needs to snap to the front
> of the list.
> I achieved that this way:
>
>
>     for i in range(len(mylist)):
>          if(mylist[i].startswith(key)):
>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> Since I need this code in multiple places, I placed it inside a function
>
> def bringOrderStringToFront(mylist, key):
>
>      for i in range(len(mylist)):
>          if(mylist[i].startswith(key)):
>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> and called it this way:
>
>   if orderstring:
>       bringOrderStringToFront(Tokens, orderstring)
>
> right?
> Nope, wrong! contrary to what I thought I had understood about how
> parameters are passed in Python, the function is acting on a copy(!) and
> my original list is unchanged.
>
> I fixed it this way:
>
> def bringOrderStringToFront(mylist, key):
>
>      for i in range(len(mylist)):
>          if(mylist[i].startswith(key)):
>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]
>      return(mylist)
>
> and:
>
> if orderstring:
>     Tokens = bringOrderStringToFront(Tokens, orderstring)
>
> but I'm left with a sour taste of not understanding what I was doing
> wrong. Can anyone elaborate? what's the pythonista way to do it right?
>
> Thanks
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [prev] | [next] | [standalone]

#106269

On Sat, 2 Apr 2016 07:27 am, Fillmore wrote:

> 
> notorious pass by reference vs pass by value biting me in the backside
> here. Proceeding in order.

Python is NEITHER pass by reference nor pass by value.

Please read this:

http://import-that.dreamwidth.org/1130.html

before asking any additional questions.


> def bringOrderStringToFront(mylist, key):
> 
>      for i in range(len(mylist)):
>          if(mylist[i].startswith(key)):
>              mylist = [mylist[i]] + mylist[:i] + mylist[i+1:]

[mylist[i]] + mylist[:i] + mylist[i+1:]

creates a new list. It doesn't modify the existing list.

mylist = ...

performs an assignment to the LOCAL VARIABLE mylist, setting it to the new
list just created. The original list is unchanged.

Trick: you can use slicing to write to the original list:

mylist[:] = [mylist[i]] + mylist[:i] + mylist[i+1:]


Or you can modify the list directly in place:

    for i in range(len(mylist)):
        if mylist[i].startswith(key):
            x = mylist[i]
            del mylist[i]
            mylist.insert(0, x)


which is better written as:

    for i, x in enumerate(mylist):
        if x.startswith(key):
            del mylist[i]
            mylist.insert(0, x)

or if you prefer:

    for i, x in enumerate(mylist):
        if x.startswith(key):
            mylist[:] = [x] + mylist[:i] + mylist[i+1:]



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#106271

On 04/01/2016 04:27 PM, Fillmore wrote:
>
> notorious pass by reference vs pass by value biting me in the backside here. Proceeding in order.


Many thanks to all of those who replied!

[toc] | [prev] | [next] | [standalone]

#106274

Fillmore wrote:

> I need to scan a list of strings. If one of the elements matches the
> beginning of a search keyword, that element needs to snap to the front
> of the list.

I know this post regards the function passing, but, on you specific problem, 
can't you just ... sort the list with a custom key?

something like (new list)

    >>> sorted(['no_a', 'yes_c', 'no_b', 'yes_z', 'no_y', 'yes_x'],
    ...         key=lambda e: not e.startswith('yes'))
    ['yes_c', 'yes_z', 'yes_x', 'no_a', 'no_b', 'no_y']

or (in place)

    >>> l = ['no_a', 'yes_c', 'no_b', 'yes_z', 'no_y', 'yes_x']
    >>> l.sort(key=lambda e: not e.startswith('yes'))
    >>> l
    ['yes_c', 'yes_z', 'yes_x', 'no_a', 'no_b', 'no_y']



-- 
By ZeD

[toc] | [prev] | [next] | [standalone]

#106276

On Sat, Apr 2, 2016, 1:46 AM Vito De Tullio <vito.detullio@gmail.com> wrote:

> Fillmore wrote:
>
> > I need to scan a list of strings. If one of the elements matches the
> > beginning of a search keyword, that element needs to snap to the front
> > of the list.
>
> I know this post regards the function passing, but, on you specific
> problem,
> can't you just ... sort the list with a custom key?
>
> something like (new list)
>
>     >>> sorted(['no_a', 'yes_c', 'no_b', 'yes_z', 'no_y', 'yes_x'],
>     ...         key=lambda e: not e.startswith('yes'))
>     ['yes_c', 'yes_z', 'yes_x', 'no_a', 'no_b', 'no_y']
>
> or (in place)
>
>     >>> l = ['no_a', 'yes_c', 'no_b', 'yes_z', 'no_y', 'yes_x']
>     >>> l.sort(key=lambda e: not e.startswith('yes'))
>     >>> l
>     ['yes_c', 'yes_z', 'yes_x', 'no_a', 'no_b', 'no_y']
>

If the number of matches is small relative to the size of the list, I'd
expect the sort would be slower than most of the other suggestions.

>

[toc] | [prev] | [next] | [standalone]

#106277

Michael Selik wrote:

>> > I need to scan a list of strings. If one of the elements matches the
>> > beginning of a search keyword, that element needs to snap to the front
>> > of the list.
>>
>> I know this post regards the function passing, but, on you specific
>> problem,
>> can't you just ... sort the list with a custom key?
>
> If the number of matches is small relative to the size of the list, I'd
> expect the sort would be slower than most of the other suggestions.

umm... I take the liberty to set up a little test

    $ cat test_sort_prefix.py
    #!/usr/bin/env python3

    from sys import argv
    from timeit import timeit


    def sort_in_place(l):
        def key(e):
            return not e.startswith('yes')
        l.sort(key=key)


    def original_solution(mylist):
        for i in range(len(mylist)):
            if(mylist[i].startswith('yes')):
                mylist[:] = [mylist[i]] + mylist[:i] + mylist[i+1:]


    def main():
        if argv[1] == 'sort_in_place':
            f = sort_in_place
        elif argv[1] == 'original_solution':
            f = original_solution
        else:
            raise Exception()

        nomatches = int(argv[2])
        matches = int(argv[3])

        l = ["no_" for _ in range(nomatches)] + ["yes_" for _ in 
range(matches)]

        print(timeit("f(l)", number=100, globals={"f": f, "l": l}))

    if __name__ == '__main__':
        main()
    $ ./test_sort_prefix.py sort_in_place 1000000 3
    33.260575089996564
    $ ./test_sort_prefix.py original_solution 1000000 3
    35.93009542999789
    $


in my tests there is no sensible difference between the two solutions... and 
the number of matches is risible :)

-- 
By ZeD

[toc] | [prev] | [next] | [standalone]

#106283

Vito De Tullio wrote:

> Michael Selik wrote:
> 
>>> > I need to scan a list of strings. If one of the elements matches the
>>> > beginning of a search keyword, that element needs to snap to the front
>>> > of the list.
>>>
>>> I know this post regards the function passing, but, on you specific
>>> problem,
>>> can't you just ... sort the list with a custom key?
>>
>> If the number of matches is small relative to the size of the list, I'd
>> expect the sort would be slower than most of the other suggestions.
> 
> umm... I take the liberty to set up a little test
> 
>     $ cat test_sort_prefix.py
>     #!/usr/bin/env python3
> 
>     from sys import argv
>     from timeit import timeit
> 
> 
>     def sort_in_place(l):
>         def key(e):
>             return not e.startswith('yes')
>         l.sort(key=key)
> 
> 
>     def original_solution(mylist):
>         for i in range(len(mylist)):
>             if(mylist[i].startswith('yes')):
>                 mylist[:] = [mylist[i]] + mylist[:i] + mylist[i+1:]

original_solution() reverses the order of the matches while sort_in_place() 
doesn't. If the order is not relevant I'd try something like

def exchange(items):
    pos = 0
    for index, item in enumerate(items):
        if item.startswith("yes"):
            items[pos], items[index] = item, items[pos]
            pos += 1

>     def main():
>         if argv[1] == 'sort_in_place':
>             f = sort_in_place
>         elif argv[1] == 'original_solution':
>             f = original_solution
>         else:
>             raise Exception()
> 
>         nomatches = int(argv[2])
>         matches = int(argv[3])
> 
>         l = ["no_" for _ in range(nomatches)] + ["yes_" for _ in
> range(matches)]

Python's timsort knows how to deal with (partially) sorted lists. I suggest 
that you add

random.seed(42) # same list for all script invocations
random.shuffle(l)

here to spread the matches somewhat over the list.

>         print(timeit("f(l)", number=100, globals={"f": f, "l": l}))

Remember that f(l) modifies l, so all but the first invocation will see a 
list where all matches are at the beginning. In other words: in 99 out of 
100 runs the list is already sorted.

While adding the overhead of copying the list I would expect the results of

timeit("f(l[:])", ...)

to be more realistic.

>     if __name__ == '__main__':
>         main()
>     $ ./test_sort_prefix.py sort_in_place 1000000 3
>     33.260575089996564
>     $ ./test_sort_prefix.py original_solution 1000000 3
>     35.93009542999789
>     $
> 
> 
> in my tests there is no sensible difference between the two solutions...
> and the number of matches is risible :)

Indeed, as the number of matches rises your sort() approach will likely fare 
better.

Your conclusions may be correct, but the benchmark to support them is 
flawed.

[toc] | [prev] | [next] | [standalone]

#106285

On 02/04/2016 06:51, Michael Selik wrote:
> On Sat, Apr 2, 2016, 1:46 AM Vito De Tullio <vito.detullio@gmail.com> wrote:
>
>> Fillmore wrote:
>>
>>> I need to scan a list of strings. If one of the elements matches the
>>> beginning of a search keyword, that element needs to snap to the front
>>> of the list.
>>
>> I know this post regards the function passing, but, on you specific
>> problem,
>> can't you just ... sort the list with a custom key?
>>
>> something like (new list)
>>
>>      >>> sorted(['no_a', 'yes_c', 'no_b', 'yes_z', 'no_y', 'yes_x'],
>>      ...         key=lambda e: not e.startswith('yes'))
>>      ['yes_c', 'yes_z', 'yes_x', 'no_a', 'no_b', 'no_y']
>>
>> or (in place)
>>
>>      >>> l = ['no_a', 'yes_c', 'no_b', 'yes_z', 'no_y', 'yes_x']
>>      >>> l.sort(key=lambda e: not e.startswith('yes'))
>>      >>> l
>>      ['yes_c', 'yes_z', 'yes_x', 'no_a', 'no_b', 'no_y']
>>
>
> If the number of matches is small relative to the size of the list, I'd
> expect the sort would be slower than most of the other suggestions.
>

My gut feeling about how fast something is in Python has never once been 
right in 15 years.  There is only way way to find out, measure it with 
things like https://docs.python.org/3/library/profile.html and 
https://docs.python.org/3/library/timeit.html.  IIRC Steven D'Aprano has 
a wrapper for the latter called Stopwatch.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#106286

On Sat, Apr 2, 2016 at 6:32 AM Peter Otten <__peter__@web.de> wrote:

> Vito De Tullio wrote:
>
> > Michael Selik wrote:
> >
> >>> > I need to scan a list of strings. If one of the elements matches the
> >>> > beginning of a search keyword, that element needs to snap to the
> front
> >>> > of the list.
> >>>
> >>> I know this post regards the function passing, but, on you specific
> >>> problem,
> >>> can't you just ... sort the list with a custom key?
> >>
> >> If the number of matches is small relative to the size of the list, I'd
> >> expect the sort would be slower than most of the other suggestions.
> >
> > umm... I take the liberty to set up a little test
> >
> >     $ cat test_sort_prefix.py
> >     #!/usr/bin/env python3
> >
> >     from sys import argv
> >     from timeit import timeit
> >
> >
> >     def sort_in_place(l):
> >         def key(e):
> >             return not e.startswith('yes')
> >         l.sort(key=key)
> >
> >
> >     def original_solution(mylist):
> >         for i in range(len(mylist)):
> >             if(mylist[i].startswith('yes')):
> >                 mylist[:] = [mylist[i]] + mylist[:i] + mylist[i+1:]
>
> original_solution() reverses the order of the matches while sort_in_place()
> doesn't. If the order is not relevant I'd try something like
>
> def exchange(items):
>     pos = 0
>     for index, item in enumerate(items):
>         if item.startswith("yes"):
>             items[pos], items[index] = item, items[pos]
>             pos += 1
>
> >     def main():
> >         if argv[1] == 'sort_in_place':
> >             f = sort_in_place
> >         elif argv[1] == 'original_solution':
> >             f = original_solution
> >         else:
> >             raise Exception()
> >
> >         nomatches = int(argv[2])
> >         matches = int(argv[3])
> >
> >         l = ["no_" for _ in range(nomatches)] + ["yes_" for _ in
> > range(matches)]
>
> Python's timsort knows how to deal with (partially) sorted lists. I suggest
> that you add
>
> random.seed(42) # same list for all script invocations
> random.shuffle(l)
>
> here to spread the matches somewhat over the list.
>
> >         print(timeit("f(l)", number=100, globals={"f": f, "l": l}))
>
> Remember that f(l) modifies l, so all but the first invocation will see a
> list where all matches are at the beginning. In other words: in 99 out of
> 100 runs the list is already sorted.
>
> While adding the overhead of copying the list I would expect the results of
>
> timeit("f(l[:])", ...)
>
> to be more realistic.
>
> >     if __name__ == '__main__':
> >         main()
> >     $ ./test_sort_prefix.py sort_in_place 1000000 3
> >     33.260575089996564
> >     $ ./test_sort_prefix.py original_solution 1000000 3
> >     35.93009542999789
> >     $
> >
> >
> > in my tests there is no sensible difference between the two solutions...
> > and the number of matches is risible :)
>
> Indeed, as the number of matches rises your sort() approach will likely
> fare
> better.
>
> Your conclusions may be correct, but the benchmark to support them is
> flawed.
>
>
Option 1: scan and match, pop, insert
Scan and match is O(n)
pop is O(n) ... gotta shift everything over one after deleting
insert is O(n) on a list, better use deque.appendleft for O(1)

Option 2: map keyfunc, sort
Mapping the keyfunction is O(n)
(tim)Sorting is O(n log n) worst case, O(n) best case.

Option 1 using a deque will be O(n).
Option 2 will be O(n log n) worst case and O(n) best case.

On small datasets the constant factors matter, but then it's small, so who
cares about speed? (partially joking)

[toc] | [prev] | [next] | [standalone]

#106306

On Friday, April 1, 2016 at 4:27:30 PM UTC-4, Fillmore wrote:
> notorious pass by reference vs pass by value biting me in the backside 
> here. Proceeding in order.

As others have pointed out, this is false dichotomy.  There are other
possibilities than pass by reference and pass by value.  Python (and
many other languages) use something called pass by object or pass by
sharing.

This talk has details of how Python's names and values work:
http://nedbatchelder.com/text/names1.html

--Ned.

[toc] | [prev] | [next] | [standalone]

#106309

On Sat, Apr 2, 2016, at 15:28, Ned Batchelder wrote:
> On Friday, April 1, 2016 at 4:27:30 PM UTC-4, Fillmore wrote:
> > notorious pass by reference vs pass by value biting me in the backside 
> > here. Proceeding in order.
> 
> As others have pointed out, this is false dichotomy.  There are other
> possibilities than pass by reference and pass by value.  Python (and
> many other languages) use something called pass by object or pass by
> sharing.

I think that this arises from a confusion as to what a "value" is in
"pass by value".

The point of it being pass by value is that there is no statement you
can execute in the function that has the effect of an assignment of the
expression that was passed in from the caller. This holds no matter what
kind of object is passed in or what mutable properties it has or does
not have.

[toc] | [prev] | [next] | [standalone]

#106337

On Sun, 3 Apr 2016 05:54 am, Random832 wrote:

> On Sat, Apr 2, 2016, at 15:28, Ned Batchelder wrote:
>> On Friday, April 1, 2016 at 4:27:30 PM UTC-4, Fillmore wrote:
>> > notorious pass by reference vs pass by value biting me in the backside
>> > here. Proceeding in order.
>> 
>> As others have pointed out, this is false dichotomy.  There are other
>> possibilities than pass by reference and pass by value.  Python (and
>> many other languages) use something called pass by object or pass by
>> sharing.
> 
> I think that this arises from a confusion as to what a "value" is in
> "pass by value".

I don't understand what you mean by this. Are you defending the idea that
Python is pass-by-value, or saying that Fillmore is confused by what
pass-by-value means, or something else? What precisely are you trying to
say here?



> The point of it being pass by value is that there is no statement you
> can execute in the function that has the effect of an assignment of the
> expression that was passed in from the caller. 

No, that alone is insufficient. There are other calling conventions that
have the same effect. Pass-by-value also requires that the value being
passed is copied.

Unfortunately we've been plagued by a bunch of comp sci graduates who suffer
from the "little bit of learning" problem. Having learned that there are
only two calling conventions, pass by value and pass by reference, and
correctly recognising that their language of choice is not pass by
reference, they then jump through some remarkable intellectual contortions
to prove that it is therefore pass by value.

To do this, they normally redefine "value" to mean something other than what
everyone else thinks of as the value being passed to a function.

The Java community is especially prone to this:

http://javadude.com/articles/passbyvalue.htm

Note the author's definition of pass-by-value:

[quote]
Pass-by-value

The actual parameter (or argument expression) is fully evaluated and the
resulting value is copied into a location being used to hold the formal
parameter's value during method/function execution.
[end quote]


That the value is *copied* is fundamental to pass by value semantics.

The author, Scott Stanchfield, then claims that Java is pass by value. 

(Note: for this entire discussion, I'm referring to boxed values, or
objects, not primitives. There's no dispute that Java is pass by value for
primitives.)

But to make this work, he has to use a bit of slight of hand: what gets
copied in Java function calls is not the value of the argument, but some
abstract reference to that value. There's this invisible abstract reference
is the value of the variable.

He doesn't come right out and say this, but if you read his description of
what Java does, it should be obvious that he thinks of Java variables as
being bound to abstract pointers or references rather than being bound to
objects themselves. Consequently, if you bind a value to a variable name:

String str = "Hello";

then to Stanchfield, the value of the string variable str is presumably
not "Hello", but some unknown and unknowable abstract reference like
0x832fe50.

This is the only way to make sense of his argument. Objects are not copied
when you pass them to a function in Java, but the invisible reference to
the object may be. Fredrik Lundh (the Effbot) quite rightly holds this view
in scorn:


[quote]
I guess you can, in theory, value an artificial number assigned
to an object as much as the object itself.

    "Joe, I think our son might be lost in the woods"
    "Don't worry, I have his social security number"
[end quote]

http://effbot.org/zone/call-by-object.htm


Stanchfield's view of invisible references to objects being passed by value
is, I have no doubt, right from the perspective of the Java implementation.
Some would argue that he is "technically correct". But he is answering the
wrong question. Calling Java (or Python) "pass by value" gives us no
insight into how the language works. It is reductionism gone mad.

Nobody, except maybe compiler designers, cares how the Java virtual machine
implements argument passing. What Java developers care about is the
behaviour of the Java language. Java's so-called "pass by value (where the
value is some invisible reference to the value you actually care about)"
behaves quite differently from pass by value in C, Pascal and others. Java
doesn't copy objects when you pass them to functions. Nor does it pass
references in pass by reference sense.

Java objects are passed using the exact same semantics as Python objects,
and neither pass by reference nor pass by value match that actual
behaviour. Both languages provide an abstraction layer between the internal
implementation details of the interpreter or virtual machine, and the
high-level behaviour of the language.


Stanchfield makes the same mistake with his claim that "Java has pointers",
since of course it does not. The Java language gives you no way of creating
such pointers to arbitrary chunks of memory (or to primitives), no way to
dereference such a pointer if you had one (which you don't), and especially
no way of advancing such (non-existent) pointers to the next record in
memory. To put it another way, not only are pointers not first-class values
in Java, they aren't values at all. There's no way to declare a variable of
type "Pointer to Foo", or equivalent.

So Stanchfield's claim that "Java has pointers" is, again, based on his
conflating of values in the Java language compared to implementation
details of the Java virtual machine. The JVM manages objects via indirect
references to the object, and for that indirection, it may use some form
of "thing which points to another thing":

- C pointers (memory addresses);
- some form of managed reference to a memory location;
- classic Mac OS "handles" (managed pointers-to-pointers);
- or even integer indexes into a giant array, like early 
  versions of Fortran. 

But that hardly means that pointers are a feature of the Java language.



> This holds no matter what 
> kind of object is passed in or what mutable properties it has or does
> not have.

I'm still not getting your point.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#106325

On 04/02/2016 12:54 PM, Random832 wrote:
> On Sat, Apr 2, 2016, at 15:28, Ned Batchelder wrote:
>> On Friday, April 1, 2016 at 4:27:30 PM UTC-4, Fillmore wrote:
>>> notorious pass by reference vs pass by value biting me in the backside
>>> here. Proceeding in order.
>>
>> As others have pointed out, this is false dichotomy.  There are other
>> possibilities than pass by reference and pass by value.  Python (and
>> many other languages) use something called pass by object or pass by
>> sharing.
>
> I think that this arises from a confusion as to what a "value" is in
> "pass by value".
>
> The point of it being pass by value is that there is no statement you
> can execute in the function that has the effect of an assignment of the
> expression that was passed in from the caller. This holds no matter what
> kind of object is passed in or what mutable properties it has or does
> not have.

Also, if "pass-by-value" is being used, even mutation of the passed 
object will not show up in the caller.

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#106326

On Sat, Apr 2, 2016, at 19:15, Ethan Furman wrote:
> Also, if "pass-by-value" is being used, even mutation of the passed 
> object will not show up in the caller.

I disagree. I don't think the definition of pass-by-value implies this.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web