Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #82751 > unrolled thread

List Comprehensions

Back to article view | Back to comp.lang.python

Contents

  List Comprehensions Ganesh Pal <ganesh1pal@gmail.com> - 2014-12-22 11:12 +0530
    Re: List Comprehensions Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-22 20:21 +1100
      Re: List Comprehensions Chris Angelico <rosuav@gmail.com> - 2014-12-22 20:45 +1100
        Re: List Comprehensions Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-12-23 02:18 +1100
          Re: List Comprehensions Chris Angelico <rosuav@gmail.com> - 2014-12-23 04:10 +1100
          Re: List Comprehensions Terry Reedy <tjreedy@udel.edu> - 2014-12-22 15:28 -0500
      Re: List Comprehensions Ian Kelly <ian.g.kelly@gmail.com> - 2014-12-22 08:12 -0700

#82751 — List Comprehensions

[Multipart message — attachments visible in raw view] — view raw

Hi ,


(a)  I was trying to reduce the below piece of code using List
comprehension ? Any suggestion please let me know


     for opt in options:
          opt['result'] = Queue.Queue()
          tmp_thread = pause.Thread(opt)
          threads.append(tmp_thread)
           tmp_thread.start()

(b) *   Is there anything that I need to consider while using list
comprehension with threads ?*


Regards,

Ganesh

[toc] | [next] | [standalone]

#82764

Ganesh Pal wrote:

> Hi ,
> 
> 
> (a)  I was trying to reduce the below piece of code using List
> comprehension ? Any suggestion please let me know
> 
> 
>      for opt in options:
>           opt['result'] = Queue.Queue()
>           tmp_thread = pause.Thread(opt)
>           threads.append(tmp_thread)
>            tmp_thread.start()

Why do you think a list comprehension is appropriate here?

A list comprehension is appropriate when you are building a list of results,
with no side-effects. Something like this:

results = []  # We want these results.
for item in items:
    tmp = function(item)  # Should have NO side-effects.
    results.append(tmp)  # We only care about the result of each call.

can be re-written as a list comp:

results = [function(item) for item in items]

If the called function has side-effects, a list comp is not a good solution.

If you don't care about the results, a list comp is not a good solution.


List comps were not invented as a way to squash arbitrarily complicated
for-loops into a single line. This is Python, not Perl.



> (b) *   Is there anything that I need to consider while using list
> comprehension with threads ?*

That depends on what you mean by "using list comprehension with threads".

If you mean "use a list comprehension inside a thread", then no, you don't
need to be concerned. So long as your functions are pure functions with no
side-effects, running a list comp inside one thread cannot affect any other
thread.

If you mean, "running threads inside the list comprehension", then
absolutely you need to be concerned. You need to take the exact same
precautions to run threaded code safely, regardless of whether the threads
are running in a for-loop or a list comp.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82766

On Mon, Dec 22, 2014 at 8:21 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> If the called function has side-effects, a list comp is not a good solution.

Hmm. I'm not so sure about that. Side effects are fine in a list comp,
as long as you're making use of the return values. For instance, if
you have a function that inserts a record to a database and returns
the new ID, you could reasonably use a list comp to save a bunch of
records and get a list of IDs for subsequent use.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82778

Chris Angelico wrote:

> On Mon, Dec 22, 2014 at 8:21 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> If the called function has side-effects, a list comp is not a good
>> solution.
> 
> Hmm. I'm not so sure about that. Side effects are fine in a list comp,
> as long as you're making use of the return values. For instance, if
> you have a function that inserts a record to a database and returns
> the new ID, you could reasonably use a list comp to save a bunch of
> records and get a list of IDs for subsequent use.

I hear what you are saying, but a list comprehension is a functional idiom.
To mix functional and procedural idioms in the one expression is rather
icky. Better to use one or the other but not both simultaneously.

I'll accept that this is a weak recommendation though.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82791

On Tue, Dec 23, 2014 at 2:18 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Chris Angelico wrote:
>
>> On Mon, Dec 22, 2014 at 8:21 PM, Steven D'Aprano
>> <steve+comp.lang.python@pearwood.info> wrote:
>>> If the called function has side-effects, a list comp is not a good
>>> solution.
>>
>> Hmm. I'm not so sure about that. Side effects are fine in a list comp,
>> as long as you're making use of the return values. For instance, if
>> you have a function that inserts a record to a database and returns
>> the new ID, you could reasonably use a list comp to save a bunch of
>> records and get a list of IDs for subsequent use.
>
> I hear what you are saying, but a list comprehension is a functional idiom.
> To mix functional and procedural idioms in the one expression is rather
> icky. Better to use one or the other but not both simultaneously.
>
> I'll accept that this is a weak recommendation though.

In my opinion, trying to separate functional and procedural idioms is
like trying to separate 'for' loops and recursion; they're two tools
that can be used separately or together, in whatever way makes the
most sense. Given that side-effecting functions are a mess in
functional programming anyway, of course they cause problems for
functional idioms; but if it's okay to have side effects at all, it
ought to be okay to have side effects of a list comp.

A list comp should take an (or several) input iterable(s) and produce
an output list. My personal expectation is that it should step through
the entire iterable, and every element of it should be "eyeballed": if
there's an 'if'' clause, they might not all produce output elements,
but they all at least had a chance to. The input iterable and its
contents shouldn't be changed in the process, other than being
consumed (if it's a one-shot iterator/generator). Side effects are
fine, just as long as the output list is important to the code.

Of course, that's just *my* expectation. As we've seen before,
expectations can differ; several of us believe that "while x:" implies
that x is able to become false (either through mutation or through
rebinding) during the loop, and others see it as equally viable for it
to mean "if x: while True:". I suspect our various backgrounds
influence our Python styles fairly strongly. And that's a good thing -
we aren't all forced into one style.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82804

On 12/22/2014 12:10 PM, Chris Angelico wrote:
> On Tue, Dec 23, 2014 at 2:18 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> Chris Angelico wrote:
>>
>>> On Mon, Dec 22, 2014 at 8:21 PM, Steven D'Aprano
>>> <steve+comp.lang.python@pearwood.info> wrote:
>>>> If the called function has side-effects, a list comp is not a good
>>>> solution.
>>>
>>> Hmm. I'm not so sure about that. Side effects are fine in a list comp,
>>> as long as you're making use of the return values. For instance, if
>>> you have a function that inserts a record to a database and returns
>>> the new ID, you could reasonably use a list comp to save a bunch of
>>> records and get a list of IDs for subsequent use.
>>
>> I hear what you are saying, but a list comprehension is a functional idiom.

In particular, it is a way to hide the mutational, non-functional, 
.append that is a necessary part of constructing an array of references 
in contiguous memory on a physical machine.

>> To mix functional and procedural idioms in the one expression is rather
>> icky.

A list comp with side-effects in the target expression is definitely a 
subversion of the original intent.  Whether Python programmers should 
respect that intent is another matter.  Limiting the 
opinion/recommendation to single expressions makes it more defensible 
than something broader.

>> Better to use one or the other but not both simultaneously.
>> I'll accept that this is a weak recommendation though.


> In my opinion, trying to separate functional and procedural idioms is
> like trying to separate 'for' loops and recursion; they're two tools

A for loop with a recursive call in its body is not 'one expression'. 
Moreover, it is easy to claim that driving multiple recursion with a for 
loop is clearer than using the recursive equivalent of a for loop to 
drive the 'horizontal' repetition.  Untested example for preorder 
traversal of a tree where nodes have ordered children:

def preorder(node):
   yield node
   for child in node.children():  # horizontal repetition
        yield from preorder(child)  # vertically nested repetition

I do not see any such gain over Ganesh Pal's original for-loop code.

> that can be used separately or together, in whatever way makes the
> most sense. Given that side-effecting functions are a mess in
> functional programming anyway, of course they cause problems for
> functional idioms; but if it's okay to have side effects at all, it
> ought to be okay to have side effects of a list comp.

I see this as equivalent to "If it is okay to have separated side 
effects, it ought to be okay to have mixed-in side effects."

I am not claiming that I would *never* use side effects in a list comp, 
but I am sympathetic to the objection.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#82777

[Multipart message — attachments visible in raw view] — view raw

On Mon, Dec 22, 2014 at 2:21 AM, Steven D'Aprano <
steve+comp.lang.python@pearwood.info> wrote:
> > (b) *   Is there anything that I need to consider while using list
> > comprehension with threads ?*
>
> That depends on what you mean by "using list comprehension with threads".
>
> If you mean "use a list comprehension inside a thread", then no, you don't
> need to be concerned. So long as your functions are pure functions with no
> side-effects, running a list comp inside one thread cannot affect any
other
> thread.

It could, however, be itself affected by other threads modifying the same
data if one isn't careful.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web