Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #30503 > unrolled thread

Slicing iterables in sub-generators without loosing elements

Back to article view | Back to comp.lang.python

Contents

  Slicing iterables in sub-generators without loosing elements Thomas Bach <thbach@students.uni-mainz.de> - 2012-09-29 18:14 +0200
    Re: Slicing iterables in sub-generators without loosing elements Paul Rubin <no.email@nospam.invalid> - 2012-09-29 09:26 -0700
      Re: Slicing iterables in sub-generators without loosing elements Thomas Bach <thbach@students.uni-mainz.de> - 2012-09-29 18:44 +0200
    Re: Slicing iterables in sub-generators without loosing elements 88888 Dihedral <dihedral88888@googlemail.com> - 2012-09-30 17:58 -0700
      Re: Slicing iterables in sub-generators without loosing elements Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-10-01 09:19 +0100
        Re: Slicing iterables in sub-generators without loosing elements Ramchandra Apte <maniandram01@gmail.com> - 2012-10-02 09:12 -0700
          Re: Slicing iterables in sub-generators without loosing elements Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-10-02 17:44 +0100
            Re: Slicing iterables in sub-generators without loosing elements Ramchandra Apte <maniandram01@gmail.com> - 2012-10-02 20:21 -0700
            Re: Slicing iterables in sub-generators without loosing elements Ramchandra Apte <maniandram01@gmail.com> - 2012-10-02 20:21 -0700
          Re: Slicing iterables in sub-generators without loosing elements Chris Angelico <rosuav@gmail.com> - 2012-10-03 03:58 +1000
            Re: Slicing iterables in sub-generators without loosing elements Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-10-03 00:57 +0000
              Re: Slicing iterables in sub-generators without loosing elements 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-02 18:11 -0700
                Re: Slicing iterables in sub-generators without loosing elements Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-10-03 01:24 +0000
                  Re: Slicing iterables in sub-generators without loosing elements 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-02 18:42 -0700
          Re: Slicing iterables in sub-generators without loosing elements Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-10-02 19:35 +0100
          Re: Slicing iterables in sub-generators without loosing elements Terry Reedy <tjreedy@udel.edu> - 2012-10-02 14:59 -0400
        Re: Slicing iterables in sub-generators without loosing elements Ramchandra Apte <maniandram01@gmail.com> - 2012-10-02 09:12 -0700
        Re: Slicing iterables in sub-generators without loosing elements 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-02 13:54 -0700
        Re: Slicing iterables in sub-generators without loosing elements 88888 Dihedral <dihedral88888@googlemail.com> - 2012-10-02 13:54 -0700
    Re: Slicing iterables in sub-generators without loosing elements 88888 Dihedral <dihedral88888@googlemail.com> - 2012-09-30 17:58 -0700

#30503 — Slicing iterables in sub-generators without loosing elements

Hi,

say we have the following:

>>> data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]

is there a way to code a function iter_in_blocks such that

>>> result = [ list(block) for block in iter_in_blocks(data) ]

evaluates to

>>> result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]

by _only_ _iterating_ over the list (caching all the elements sharing
the same first element doesn't count)?

I came up with the following

def iter_in_blocks(iterable):
    my_iter = iter(iterable)
    while True:
        first = next(my_iter)
        pred = lambda entry: entry[0] == first[0]
        def block_iter():
            yield first
            for entry in itertools.takewhile(pred, my_iter):
                yield entry
        yield block_iter()

which does not work as itertools.takewhile consumes the first entry
not fulfilling the pred.

I currently have the intuition that the problem is not solvable
without using e.g. a global to pass something back to iter_in_blocks
from block_iter. Any other suggestions?

Regards,
	Thomas Bach.

[toc] | [next] | [standalone]

#30504

Thomas Bach <thbach@students.uni-mainz.de> writes:
>>>> result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]
> by _only_ _iterating_ over the list (caching all the elements sharing
> the same first element doesn't count)?

itertools.groupby(data, lambda (x,y) : x)

is basically what you want.

[toc] | [prev] | [next] | [standalone]

#30507

On Sat, Sep 29, 2012 at 09:26:00AM -0700, Paul Rubin wrote:
> Thomas Bach <thbach@students.uni-mainz.de> writes:
> 
> itertools.groupby(data, lambda (x,y) : x)
> 
> is basically what you want.

True!

Thanks,
       Thomas Bach

[toc] | [prev] | [next] | [standalone]

#30577

On Sunday, September 30, 2012 12:15:57 AM UTC+8, Thomas Bach wrote:
> Hi,
> 
> 
> 
> say we have the following:
> 
> 
> 
> >>> data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]
> 
> 
> 
> is there a way to code a function iter_in_blocks such that
> 
> 
> 
> >>> result = [ list(block) for block in iter_in_blocks(data) ]
> 
> 
> 
> evaluates to
> 
> 
> 
> >>> result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]
> 
> 
> 
> by _only_ _iterating_ over the list (caching all the elements sharing
> 
> the same first element doesn't count)?
> 
> 
> 
> I came up with the following
> 
> 
> 
> def iter_in_blocks(iterable):
> 
>     my_iter = iter(iterable)
> 
>     while True:
> 
>         first = next(my_iter)
> 
>         pred = lambda entry: entry[0] == first[0]
> 
>         def block_iter():
> 
>             yield first
> 
>             for entry in itertools.takewhile(pred, my_iter):
> 
>                 yield entry
> 
>         yield block_iter()
> 
> 
> 
> which does not work as itertools.takewhile consumes the first entry
> 
> not fulfilling the pred.
> 
> 
> 
> I currently have the intuition that the problem is not solvable
> 
> without using e.g. a global to pass something back to iter_in_blocks
> 
> from block_iter. Any other suggestions?
> 
> 
> 
> Regards,
> 
> 	Thomas Bach.

Your question seems vague to me. If you know you are storing 
only immutable tuples in a list, then the way to iterate is simple. 

For example: 

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)] 
# all tuples 

for item in data: 
    x1=item[0] # first entry in each tuple
    x2=item[1]
    print x1, x2 # or use yield in a function to iterate



 

[toc] | [prev] | [next] | [standalone]

#30599

On 01/10/2012 01:58, 88888 Dihedral wrote:
>
> Your question seems vague to me. If you know you are storing
> only immutable tuples in a list, then the way to iterate is simple.
>

Does Python have a magic method that let's me use mutable tuples?  I'd 
also like immutable lists.  Is it worth raising a feature request on the 
bug tracker?

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#30639

On Monday, 1 October 2012 13:47:50 UTC+5:30, Mark Lawrence  wrote:
> On 01/10/2012 01:58, 88888 Dihedral wrote:
> 
> >
> 
> > Your question seems vague to me. If you know you are storing
> 
> > only immutable tuples in a list, then the way to iterate is simple.
> 
> >
> 
> 
> 
> Does Python have a magic method that let's me use mutable tuples?  I'd 
> 
> also like immutable lists.  Is it worth raising a feature request on the 
> 
> bug tracker?
> 
> 
> 
> -- 
> 
> Cheers.
> 
> 
> 
> Mark Lawrence.

Mark, you are talking to a bot.

[toc] | [prev] | [next] | [standalone]

#30645

On 02/10/2012 17:12, Ramchandra Apte wrote:
> On Monday, 1 October 2012 13:47:50 UTC+5:30, Mark Lawrence  wrote:
>> On 01/10/2012 01:58, 88888 Dihedral wrote:
>>
>>>
>>
>>> Your question seems vague to me. If you know you are storing
>>
>>> only immutable tuples in a list, then the way to iterate is simple.
>>
>>>
>>
>>
>>
>> Does Python have a magic method that let's me use mutable tuples?  I'd
>>
>> also like immutable lists.  Is it worth raising a feature request on the
>>
>> bug tracker?
>>
>>
>>
>> --
>>
>> Cheers.
>>
>>
>>
>> Mark Lawrence.
>
> Mark, you are talking to a bot.
>

What happened to freedom of speech?  If I want to talk to a bot, I'll 
talk to a bot.  Besides I'm not convinced it/he/she is a bot.  Plus if 
you read my post carefully, add in several years experience of Python 
the language and Python the comedy, you might come to the conclusion 
that a certain amount of urine extraction was going on :)

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#30670

On Tuesday, 2 October 2012 22:13:20 UTC+5:30, Mark Lawrence  wrote:
> On 02/10/2012 17:12, Ramchandra Apte wrote:
> 
> > On Monday, 1 October 2012 13:47:50 UTC+5:30, Mark Lawrence  wrote:
> 
> >> On 01/10/2012 01:58, 88888 Dihedral wrote:
> 
> >>
> 
> >>>
> 
> >>
> 
> >>> Your question seems vague to me. If you know you are storing
> 
> >>
> 
> >>> only immutable tuples in a list, then the way to iterate is simple.
> 
> >>
> 
> >>>
> 
> >>
> 
> >>
> 
> >>
> 
> >> Does Python have a magic method that let's me use mutable tuples?  I'd
> 
> >>
> 
> >> also like immutable lists.  Is it worth raising a feature request on the
> 
> >>
> 
> >> bug tracker?
> 
> >>
> 
> >>
> 
> >>
> 
> >> --
> 
> >>
> 
> >> Cheers.
> 
> >>
> 
> >>
> 
> >>
> 
> >> Mark Lawrence.
> 
> >
> 
> > Mark, you are talking to a bot.
> 
> >
> 
> 
> 
> What happened to freedom of speech?  If I want to talk to a bot, I'll 
> 
> talk to a bot.  Besides I'm not convinced it/he/she is a bot.  Plus if 
> 
> you read my post carefully, add in several years experience of Python 
> 
> the language and Python the comedy, you might come to the conclusion 
> 
> that a certain amount of urine extraction was going on :)
> 
> 
> 
> -- 
> 
> Cheers.
> 
> 
> 
> Mark Lawrence.

Never said you can't.

[toc] | [prev] | [next] | [standalone]

#30671

On Tuesday, 2 October 2012 22:13:20 UTC+5:30, Mark Lawrence  wrote:
> On 02/10/2012 17:12, Ramchandra Apte wrote:
> 
> > On Monday, 1 October 2012 13:47:50 UTC+5:30, Mark Lawrence  wrote:
> 
> >> On 01/10/2012 01:58, 88888 Dihedral wrote:
> 
> >>
> 
> >>>
> 
> >>
> 
> >>> Your question seems vague to me. If you know you are storing
> 
> >>
> 
> >>> only immutable tuples in a list, then the way to iterate is simple.
> 
> >>
> 
> >>>
> 
> >>
> 
> >>
> 
> >>
> 
> >> Does Python have a magic method that let's me use mutable tuples?  I'd
> 
> >>
> 
> >> also like immutable lists.  Is it worth raising a feature request on the
> 
> >>
> 
> >> bug tracker?
> 
> >>
> 
> >>
> 
> >>
> 
> >> --
> 
> >>
> 
> >> Cheers.
> 
> >>
> 
> >>
> 
> >>
> 
> >> Mark Lawrence.
> 
> >
> 
> > Mark, you are talking to a bot.
> 
> >
> 
> 
> 
> What happened to freedom of speech?  If I want to talk to a bot, I'll 
> 
> talk to a bot.  Besides I'm not convinced it/he/she is a bot.  Plus if 
> 
> you read my post carefully, add in several years experience of Python 
> 
> the language and Python the comedy, you might come to the conclusion 
> 
> that a certain amount of urine extraction was going on :)
> 
> 
> 
> -- 
> 
> Cheers.
> 
> 
> 
> Mark Lawrence.

Never said you can't.

[toc] | [prev] | [next] | [standalone]

#30649

On Wed, Oct 3, 2012 at 2:44 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> What happened to freedom of speech?  If I want to talk to a bot, I'll talk
> to a bot.  Besides I'm not convinced it/he/she is a bot.  Plus if you read
> my post carefully, add in several years experience of Python the language
> and Python the comedy, you might come to the conclusion that a certain
> amount of urine extraction was going on :)

Coupled with a bit of bot-seeding, which is always fun.

One of these days I'm going to mail Dihedral a whole pile of Gilbert
and Sullivan operetta and see if any of it comes back in his posts...

Dihedral might be a bot and might not. I've come to the conclusion
that it's not worth trying to find out, given that a good bot can
outdo a lot of humans in useful conversation.

ChrisA

[toc] | [prev] | [next] | [standalone]

#30664

On Wed, 03 Oct 2012 03:58:02 +1000, Chris Angelico wrote:

> Dihedral might be a bot and might not. I've come to the conclusion that
> it's not worth trying to find out, given that a good bot can outdo a lot
> of humans in useful conversation.

Oh, I'm convinced that it's a bot.

The fact that Dihedral never responds to conversations about him/it is a 
give away: nearly all people are far to egotistical to let accusations of 
bot-hood go unchallenged. And even though its responses are grammatically 
correct, they often are semantically meaningless. Even the best AIs 
(Evie, CleverBot, Alice) don't come close to passing the Turing test. It 
helps that Dihedral only talks about computer programming, but even so, 
it has well and truly failed my personal Turing test.

Mind you, some human beings fail the Turing test too:

http://northernplanets.blogspot.com.au/2006/06/failing-turing-test.html


Of course, Dihedral's creator wanted to really confound us, the 
occasional human response would probably muddy the waters sufficiently.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#30665

Steven D'Aprano於 2012年10月3日星期三UTC+8上午8時57分20秒寫道：
> On Wed, 03 Oct 2012 03:58:02 +1000, Chris Angelico wrote:
> 
> 
> 
> > Dihedral might be a bot and might not. I've come to the conclusion that
> 
> > it's not worth trying to find out, given that a good bot can outdo a lot
> 
> > of humans in useful conversation.
> 
> 
> 
> Oh, I'm convinced that it's a bot.
> 
> 
> 
> The fact that Dihedral never responds to conversations about him/it is a 
> 
> give away: nearly all people are far to egotistical to let accusations of 
> 
> bot-hood go unchallenged. And even though its responses are grammatically 
> 
> correct, they often are semantically meaningless. Even the best AIs 
> 
> (Evie, CleverBot, Alice) don't come close to passing the Turing test. It 
> 
> helps that Dihedral only talks about computer programming, but even so, 
> 
> it has well and truly failed my personal Turing test.
> 
> 
> 
> Mind you, some human beings fail the Turing test too:
> 
> 
> 
> http://northernplanets.blogspot.com.au/2006/06/failing-turing-test.html
> 
> 
> 
> 
> 
> Of course, Dihedral's creator wanted to really confound us, the 
> 
> occasional human response would probably muddy the waters sufficiently.
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven

Are you still not getting the generator part  in Python?
Do you really test any generator before?

[toc] | [prev] | [next] | [standalone]

#30667

On Tue, 02 Oct 2012 18:11:20 -0700, 88888 Dihedral wrote:

> Steven D'Aprano於 2012年10月3日星期三UTC+8上午8時57分20秒寫道：

>> Oh, I'm convinced that it's a bot.
>> The fact that Dihedral never responds to conversations about him/it is
>> a give away: nearly all people are far to egotistical to let
>> accusations of bot-hood go unchallenged.
[...]
>> Of course, Dihedral's creator wanted to really confound us, the
>> occasional human response would probably muddy the waters sufficiently.

> Are you still not getting the generator part  in Python? Do you really
> test any generator before?

I rest my case :)

I have many problems with generators. Can you tell me more about them? I 
really need your help. Sometimes, when it is very cold, the diesel 
freezes and the generator raises StopIteration at the wrong place.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#30668

Steven D'Aprano於 2012年10月3日星期三UTC+8上午9時24分13秒寫道：
> On Tue, 02 Oct 2012 18:11:20 -0700, 88888 Dihedral wrote:
> 
> 
> 
> > Steven D'Aprano於 2012年10月3日星期三UTC+8上午8時57分20秒寫道：
> 
> 
> 
> >> Oh, I'm convinced that it's a bot.
> 
> >> The fact that Dihedral never responds to conversations about him/it is
> 
> >> a give away: nearly all people are far to egotistical to let
> 
> >> accusations of bot-hood go unchallenged.
> 
> [...]
> 
> >> Of course, Dihedral's creator wanted to really confound us, the
> 
> >> occasional human response would probably muddy the waters sufficiently.
> 
> 
> 
> > Are you still not getting the generator part  in Python? Do you really
> 
> > test any generator before?
> 
> 
> 
> I rest my case :)
> 
> 
> 
> I have many problems with generators. Can you tell me more about them? I 
> 
> really need your help. Sometimes, when it is very cold, the diesel 
> 
> freezes and the generator raises StopIteration at the wrong place.
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven
Please work out your own templates  as I showed in the decorator part.

[toc] | [prev] | [next] | [standalone]

#30650

On 02/10/2012 18:58, Chris Angelico wrote:
>
> Dihedral might be a bot and might not. I've come to the conclusion
> that it's not worth trying to find out, given that a good bot can
> outdo a lot of humans in useful conversation.
>
> ChrisA
>

Try telling that to the newbies on the Python tutor mailing list and 
they simply won't believe ya :)

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#30653

On 10/2/2012 1:58 PM, Chris Angelico wrote:
> On Wed, Oct 3, 2012 at 2:44 AM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>> What happened to freedom of speech?  If I want to talk to a bot, I'll talk
>> to a bot.  Besides I'm not convinced it/he/she is a bot.  Plus if you read
>> my post carefully, add in several years experience of Python the language
>> and Python the comedy, you might come to the conclusion that a certain
>> amount of urine extraction was going on :)
>
> Coupled with a bit of bot-seeding, which is always fun.
>
> One of these days I'm going to mail Dihedral a whole pile of Gilbert
> and Sullivan operetta and see if any of it comes back in his posts...
>
> Dihedral might be a bot and might not. I've come to the conclusion
> that it's not worth trying to find out, given that a good bot can
> outdo a lot of humans in useful conversation.

I just read that bots playing Unreal Tournament with an 'act human' 
module seem more human to at least some humans than humans. Perhaps that 
is because humans have to become bot-like to play UT well.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#30640

On Monday, 1 October 2012 13:47:50 UTC+5:30, Mark Lawrence  wrote:
> On 01/10/2012 01:58, 88888 Dihedral wrote:
> 
> >
> 
> > Your question seems vague to me. If you know you are storing
> 
> > only immutable tuples in a list, then the way to iterate is simple.
> 
> >
> 
> 
> 
> Does Python have a magic method that let's me use mutable tuples?  I'd 
> 
> also like immutable lists.  Is it worth raising a feature request on the 
> 
> bug tracker?
> 
> 
> 
> -- 
> 
> Cheers.
> 
> 
> 
> Mark Lawrence.

Mark, you are talking to a bot.

[toc] | [prev] | [next] | [standalone]

#30657

On Monday, October 1, 2012 4:17:50 PM UTC+8, Mark Lawrence wrote:
> On 01/10/2012 01:58, 88888 Dihedral wrote:
> 
> >
> 
> > Your question seems vague to me. If you know you are storing
> 
> > only immutable tuples in a list, then the way to iterate is simple.
> 
> >
> 
> 
> 
> Does Python have a magic method that let's me use mutable tuples?  I'd 
> 
> also like immutable lists.  Is it worth raising a feature request on the 
> 
> bug tracker?
> 
> 
> 
> -- 
> 
> Cheers.
> 
> 
> 
> Mark Lawrence.

Python resolves objects by names and dynamical types.
This will force the programmer to write structured programs.

[toc] | [prev] | [next] | [standalone]

#30658

On Monday, October 1, 2012 4:17:50 PM UTC+8, Mark Lawrence wrote:
> On 01/10/2012 01:58, 88888 Dihedral wrote:
> 
> >
> 
> > Your question seems vague to me. If you know you are storing
> 
> > only immutable tuples in a list, then the way to iterate is simple.
> 
> >
> 
> 
> 
> Does Python have a magic method that let's me use mutable tuples?  I'd 
> 
> also like immutable lists.  Is it worth raising a feature request on the 
> 
> bug tracker?
> 
> 
> 
> -- 
> 
> Cheers.
> 
> 
> 
> Mark Lawrence.

Python resolves objects by names and dynamical types.
This will force the programmer to write structured programs.

[toc] | [prev] | [next] | [standalone]

#30579

On Sunday, September 30, 2012 12:15:57 AM UTC+8, Thomas Bach wrote:
> Hi,
> 
> 
> 
> say we have the following:
> 
> 
> 
> >>> data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]
> 
> 
> 
> is there a way to code a function iter_in_blocks such that
> 
> 
> 
> >>> result = [ list(block) for block in iter_in_blocks(data) ]
> 
> 
> 
> evaluates to
> 
> 
> 
> >>> result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]
> 
> 
> 
> by _only_ _iterating_ over the list (caching all the elements sharing
> 
> the same first element doesn't count)?
> 
> 
> 
> I came up with the following
> 
> 
> 
> def iter_in_blocks(iterable):
> 
>     my_iter = iter(iterable)
> 
>     while True:
> 
>         first = next(my_iter)
> 
>         pred = lambda entry: entry[0] == first[0]
> 
>         def block_iter():
> 
>             yield first
> 
>             for entry in itertools.takewhile(pred, my_iter):
> 
>                 yield entry
> 
>         yield block_iter()
> 
> 
> 
> which does not work as itertools.takewhile consumes the first entry
> 
> not fulfilling the pred.
> 
> 
> 
> I currently have the intuition that the problem is not solvable
> 
> without using e.g. a global to pass something back to iter_in_blocks
> 
> from block_iter. Any other suggestions?
> 
> 
> 
> Regards,
> 
> 	Thomas Bach.

Your question seems vague to me. If you know you are storing 
only immutable tuples in a list, then the way to iterate is simple. 

For example: 

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)] 
# all tuples 

for item in data: 
    x1=item[0] # first entry in each tuple
    x2=item[1]
    print x1, x2 # or use yield in a function to iterate



 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web