Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #89555 > unrolled thread

Useful module to be written by a newbie

Back to article view | Back to comp.lang.python

Contents

  Useful module to be written by a newbie Cecil Westerhof <Cecil@decebal.nl> - 2015-04-29 16:53 +0200
    Re: Useful module to be written by a newbie Peter Otten <__peter__@web.de> - 2015-04-29 18:27 +0200
      Re: Useful module to be written by a newbie Cecil Westerhof <Cecil@decebal.nl> - 2015-04-29 19:12 +0200
        Re: Useful module to be written by a newbie Peter Otten <__peter__@web.de> - 2015-04-29 21:03 +0200
          Re: Useful module to be written by a newbie Cecil Westerhof <Cecil@decebal.nl> - 2015-04-29 22:47 +0200
            Re: Useful module to be written by a newbie Peter Otten <__peter__@web.de> - 2015-04-30 00:00 +0200
            Re: Useful module to be written by a newbie Michael Welle <mwe012008@gmx.net> - 2015-05-09 08:10 +0200
              Re: Useful module to be written by a newbie Cecil Westerhof <Cecil@decebal.nl> - 2015-05-09 10:28 +0200
                Re: Useful module to be written by a newbie Michael Welle <mwe012008@gmx.net> - 2015-05-09 11:43 +0200
        Re: Useful module to be written by a newbie Ian Kelly <ian.g.kelly@gmail.com> - 2015-04-29 13:40 -0600

#89555 — Useful module to be written by a newbie

I have experience with Python, but it has been some time ago. The best
way to relearn a language (for me) is just make a lot of code with it.
But it would be nice if it was useful at the same time. I started a
Python library on GitHub:
    https://github.com/CecilWesterhof/PythonLibrary

Anyone an idea about functions and classes that would be useful for
the Python community and could be written by me?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [next] | [standalone]

#89558

Cecil Westerhof wrote:

> I have experience with Python, but it has been some time ago. The best
> way to relearn a language (for me) is just make a lot of code with it.
> But it would be nice if it was useful at the same time. I started a
> Python library on GitHub:
>     https://github.com/CecilWesterhof/PythonLibrary
> 
> Anyone an idea about functions and classes that would be useful for
> the Python community and could be written by me?

Realistically a Python coder with a little experience will have a glance at 
your code and run away.

But: a family member, or friend, or even you yourself with a small real-
world problem will not care whether your code is idiomatic Python as long as 
it works as advertised. So I'd look that way.

As you idle along you'll quickly get better and may encounter a library 
where you think "That might be a useful extension" or one where you think 
"That's much too complex, I can make it simpler for 80% of the use cases". 

Voilà your contributions to the community.


PS: Regarding your MovingAverage class, have a look at zip(), generators or 
the __iter__() method, and -- most important -- unit tests.

[toc] | [prev] | [next] | [standalone]

#89560

Op Wednesday 29 Apr 2015 18:27 CEST schreef Peter Otten:

> Cecil Westerhof wrote:
>
>> I have experience with Python, but it has been some time ago. The
>> best way to relearn a language (for me) is just make a lot of code
>> with it. But it would be nice if it was useful at the same time. I
>> started a Python library on GitHub:
>> https://github.com/CecilWesterhof/PythonLibrary
>>
>> Anyone an idea about functions and classes that would be useful for
>> the Python community and could be written by me?
>
> Realistically a Python coder with a little experience will have a
> glance at your code and run away.

Oops, that is not nice to hear. :'-( But can you enlighten me? Then I
can learn from it.


> But: a family member, or friend, or even you yourself with a small
> real- world problem will not care whether your code is idiomatic
> Python as long as it works as advertised. So I'd look that way.
>
> As you idle along you'll quickly get better and may encounter a
> library where you think "That might be a useful extension" or one
> where you think "That's much too complex, I can make it simpler for
> 80% of the use cases".
>
> Voilà your contributions to the community.

Well, I hope that will happen soon.


> PS: Regarding your MovingAverage class, have a look at zip(),
> generators or the __iter__() method, and -- most important -- unit
> tests.

I will do that.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#89569

Cecil Westerhof wrote:

> Op Wednesday 29 Apr 2015 18:27 CEST schreef Peter Otten:
> 
>> Cecil Westerhof wrote:
>>
>>> I have experience with Python, but it has been some time ago. The
>>> best way to relearn a language (for me) is just make a lot of code
>>> with it. But it would be nice if it was useful at the same time. I
>>> started a Python library on GitHub:
>>> https://github.com/CecilWesterhof/PythonLibrary
>>>
>>> Anyone an idea about functions and classes that would be useful for
>>> the Python community and could be written by me?
>>
>> Realistically a Python coder with a little experience will have a
>> glance at your code and run away.
> 
> Oops, that is not nice to hear. :'-( 

Sorry, I did not mean to discourage you or insult you, I just wanted to make 
it clear that your code is not there yet.

> But can you enlighten me? Then I can learn from it.

I was judging from the look of your MovingAverage.

I don't like the interface, it really should take an iterable so that you 
can write

>>> list(moving_average([1,2,3], 2))
[1.5, 2.5]

I don't see how you cope with error accumulation.

Given how generic your code is I don't see why you limit it to just int and 
float, and I don't expect that limitation to find errors caused by my code 
using your moving average. 

And then there's the testing...

Compare that to the beast I found in a web search for moving average python:

<https://github.com/linsomniac/python-movingaverage/blob/master/movingaverage.py>

I'm no numerics expert and it would take me some time to verify that it is a 
good implementation without bugs or corner cases that affect my use case, 
but I can pick any three lines from the code and they just look right. If 
right now I needed an implementation of moving average that is more advanced 
than the naive one I can churn out

>>> def moving_average(values, n):
...     values = iter(values)
...     d = []
...     for i in range(n-1):
...         d.append(next(values))
...     for v in values:
...         d.append(v)
...         yield sum(d) / n
...         del d[0]
... 
>>> list(moving_average([1,2,3], 2))
[1.5, 2.5]

and there is no implementation from a reputable project like numpy I'd 
certainly use that one rather than yours.

To put it into perspective: There are of course many areas where there is 
not much competition and if someone makes some exotic hardware accessible 
via Python then I am grateful and will accept that that person writes Python 
as if it were C and has not yet grokked the exact meaning of the global 
statement. In other words: useful wins over easy-to-use wins over idiomatic.

[toc] | [prev] | [next] | [standalone]

#89577

Op Wednesday 29 Apr 2015 21:03 CEST schreef Peter Otten:

>>> Realistically a Python coder with a little experience will have a
>>> glance at your code and run away.
>>
>> Oops, that is not nice to hear. :'-( 
>
> Sorry, I did not mean to discourage you or insult you, I just wanted
> to make it clear that your code is not there yet.

You did not. Of-course it is not nice to hear, but if it is true, it
is very useful. If there is a lot to be desired, then it is good when
someone point this out.


>> But can you enlighten me? Then I can learn from it.
>
> I was judging from the look of your MovingAverage.
>
> I don't like the interface, it really should take an iterable so
> that you can write
>
>>>> list(moving_average([1,2,3], 2))
> [1.5, 2.5]

I should probably document it better. The idea is that you do not have
the data yet. At certain intervals data is acquired and at that moment
you want the moving average with the new data. But it would not hurt
to have the possibility to feed it a list of values, beside single
value. Something to better the function. Thanks.


> I don't see how you cope with error accumulation.

That is a good one. I have the values themselves (I use them in the
test). I could every so much new values check if there is an error. If
that is the case log it and use the correct value.


> Given how generic your code is I don't see why you limit it to just
> int and float, and I don't expect that limitation to find errors
> caused by my code using your moving average.

Because it gives back a float I thought it better to use only int and
float, but I was contemplating expansion.


> And then there's the testing...
>
> Compare that to the beast I found in a web search for moving average
> python:
>
> <https://github.com/linsomniac/python-movingaverage/blob/master/movingaverage.py>

That is certainly something I can learn from.

I have to implement the following improvements:
    Create a next_values
    Create a internal checking function
    Implement no_rolling_sum?
    Expand testing
    Option to force no_rolling_sum when no_rolling_sum? is True
    - for the current call
    - for the current call and length -1 following

And I should move it to mathDecebal.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#89581

Cecil Westerhof wrote:

> Op Wednesday 29 Apr 2015 21:03 CEST schreef Peter Otten:

>> I was judging from the look of your MovingAverage.
>>
>> I don't like the interface, it really should take an iterable so
>> that you can write
>>
>>>>> list(moving_average([1,2,3], 2))
>> [1.5, 2.5]
> 
> I should probably document it better. The idea is that you do not have
> the data yet. At certain intervals data is acquired and at that moment
> you want the moving average with the new data. But it would not hurt
> to have the possibility to feed it a list of values, beside single
> value. Something to better the function. Thanks.

With that explanation and especially Ian's motivational excursion it makes 
more sense. My conclusion "ah newbie, doesn't know/shuns iterator protocol" 
may be wrong. However, it may be better to follow Ian's example and decouple 
adding a new value and reading the current average.

[toc] | [prev] | [next] | [standalone]

#90213

Hello,

Cecil Westerhof <Cecil@decebal.nl> writes:

> Op Wednesday 29 Apr 2015 21:03 CEST schreef Peter Otten:
>
>>>> Realistically a Python coder with a little experience will have a
>>>> glance at your code and run away.
>>>
>>> Oops, that is not nice to hear. :'-( 
>>
>> Sorry, I did not mean to discourage you or insult you, I just wanted
>> to make it clear that your code is not there yet.
>
> You did not. Of-course it is not nice to hear, but if it is true, it
> is very useful. If there is a lot to be desired, then it is good when
> someone point this out.
>
>
>>> But can you enlighten me? Then I can learn from it.
learning a new language looks like an easy job, in most cases. All the
language's keywords and stuff, you can shuffle that into your head in a
weekend or so. But what it makes it a hard task is all the idioms. It
takes a long time to learn them. I like your approach of hacking random
algorithms, like happynumbers and friends, (everyone does it to learn a
new language I think) and show them for criticism (not everyone does
that).

Regards
hmw

-- 
biff4emacsen - A biff-like tool for (X)Emacs
http://www.c0t0d0s0.de/biff4emacsen/biff4emacsen.html
Flood - Your friendly network packet generator
http://www.c0t0d0s0.de/flood/flood.html

[toc] | [prev] | [next] | [standalone]

#90220

Op Saturday 9 May 2015 08:10 CEST schreef Michael Welle:

> Cecil Westerhof <Cecil@decebal.nl> writes:
>
>> Op Wednesday 29 Apr 2015 21:03 CEST schreef Peter Otten:
>>
>>>>> Realistically a Python coder with a little experience will have
>>>>> a glance at your code and run away.
>>>>
>>>> Oops, that is not nice to hear. :'-( 
>>>
>>> Sorry, I did not mean to discourage you or insult you, I just
>>> wanted to make it clear that your code is not there yet.
>>
>> You did not. Of-course it is not nice to hear, but if it is true,
>> it is very useful. If there is a lot to be desired, then it is good
>> when someone point this out.
>>
>>
>>>> But can you enlighten me? Then I can learn from it.
> learning a new language looks like an easy job, in most cases. All
> the language's keywords and stuff, you can shuffle that into your
> head in a weekend or so. But what it makes it a hard task is all the
> idioms. It takes a long time to learn them. I like your approach of
> hacking random algorithms, like happynumbers and friends, (everyone
> does it to learn a new language I think) and show them for criticism
> (not everyone does that).

Well in my experience the fastest way to learn something is let people
‘burn you down’. Of-course you need to be able to take it. Also
important: “C'est le ton qui fait la musique”. But no problems here.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

[toc] | [prev] | [next] | [standalone]

#90225

Hello,

Cecil Westerhof <Cecil@decebal.nl> writes:

> Op Saturday 9 May 2015 08:10 CEST schreef Michael Welle:
[...]
>> learning a new language looks like an easy job, in most cases. All
>> the language's keywords and stuff, you can shuffle that into your
>> head in a weekend or so. But what it makes it a hard task is all the
>> idioms. It takes a long time to learn them. I like your approach of
>> hacking random algorithms, like happynumbers and friends, (everyone
>> does it to learn a new language I think) and show them for criticism
>> (not everyone does that).
>
> Well in my experience the fastest way to learn something is let people
> ‘burn you down’. Of-course you need to be able to take it. Also
full ack.


> important: “C'est le ton qui fait la musique”. But no problems here.
The only problem is that there often is no tone in textual usenet, only
notes, so to speak. Sometimes one came along a bit harsh (I mean in
general, not referencing this thread here), but it is really not
intended. It's just because the tone (of the voice) or the facial
expressions are missing. Anyways, back to Python ;).

Regards
hmw

-- 
biff4emacsen - A biff-like tool for (X)Emacs
http://www.c0t0d0s0.de/biff4emacsen/biff4emacsen.html
Flood - Your friendly network packet generator
http://www.c0t0d0s0.de/flood/flood.html

[toc] | [prev] | [next] | [standalone]

#89570

On Wed, Apr 29, 2015 at 1:03 PM, Peter Otten <__peter__@web.de> wrote:
> I was judging from the look of your MovingAverage.
>
> I don't like the interface, it really should take an iterable so that you
> can write
>
>>>> list(moving_average([1,2,3], 2))
> [1.5, 2.5]

The problem with this is that many use cases for moving averages need
to access the current average before future items become available.
For example, a download speed indicator: it needs to average the
transfer rate over the last few seconds. There will be more transfer
rate data in the future, but it's not available yet, so you can't add
it to the iterable unless the iterable is actually some sort of
read/write buffer that can be appended to once iteration has already
started.

That seems overly complex, though; for one, you need to be careful not
to exhaust the buffer, since the iteration protocol requires that once
next() raises StopIteration, it will always raise StopIteration. But
also, why add the data to object A so that it can be consumed by
object B if you could just add it to object B directly?

So I think the iterable interface that you're describing really needs
to be a coroutine of some sort:

>>> from collections import deque
>>> def moving_average(length):
...     values = deque([(yield)], maxlen=length)
...     while True:
...         values.append((yield sum(values) / len(values)))
...
>>> mavg = moving_average(5)
>>> next(mavg)
>>> mavg.send(1)
1.0
>>> mavg.send(2)
1.5
>>> mavg.send(3)
2.0
>>> mavg.send(4)
2.5
>>> mavg.send(5)
3.0
>>> mavg.send(6)
4.0
>>> mavg.send(7)
5.0

This works, but I don't really like it. For one, our moving_average
"iterable" isn't really an iterable any more; we need to call send
instead of next, which means we can't just stick it inside a for loop.
And if we can't iterate over it, then what's the point of using a
generator? If we make it a class, then we can give it a more flexible
API.

class MovingAverage(object):

    def __init__(self, length):
        self.values = deque(maxlen=length)

    def append(self, value):
        self.values.append(value)

    def average(self):
        return sum(self.values) / len(self.values)

Which is pretty much back to where we started.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web