Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #51054 > unrolled thread

odd behavoiur seen

Back to article view | Back to comp.lang.python

Contents

  odd behavoiur seen Chris Hinsley <chris.hinsley@gmail.com> - 2013-07-22 19:36 +0100
    Re: odd behavoiur seen Chris Hinsley <chris.hinsley@gmail.com> - 2013-07-22 19:42 +0100
      Re: odd behavoiur seen Peter Otten <__peter__@web.de> - 2013-07-22 21:47 +0200
        Re: odd behavoiur seen Chris Hinsley <chris.hinsley@gmail.com> - 2013-07-22 21:12 +0100

#51054 — odd behavoiur seen

Folks, I have this decorator:

def memoize(maxsize):
    def _memoize(func):
        lru_cache = {}
        lru_list = []

        def memoizer(*args, **kwargs):
            key = str(args) + str(kwargs)
            if key in lru_cache:
                lru_list.remove(key)
                lru_list.append(key)
                return lru_cache[key]
            print len(lru_list),
            if len(lru_list) >= maxsize:
                del(lru_cache[lru_list[0]])
                del(lru_list[0])
            ret = func(*args, **kwargs)
            lru_cache[key] = ret
            lru_list.append(key)
            return ret
        return memoizer
    return _memoize

I didn't used to do the 'len(lru_list) >= maxsize' just '==' and 
noticed it sailing past the max number of entries, so put in the print 
statement, and now I see it ocationally printing a value 1 larger than 
maxsize !!!

So if I use it as '@memoize(64)' I see some 65's in the output ! I'm at 
a loss to explain it, does anyone knows why ? Is it a bug or some 
threading issue ? I'm not useing threads BTW, and I've noticed this in 
both running it with Python or Pypy.

Best Regards

Chris

[toc] | [next] | [standalone]

#51055

On 2013-07-22 18:36:41 +0000, Chris Hinsley said:

> Folks, I have this decorator:
> 
> def memoize(maxsize):
>     def _memoize(func):
>         lru_cache = {}
>         lru_list = []

Other clues, I use it on a recursive function:

@memoize(64)
def next_move(board, colour, alpha, beta, ply):
    if ply <= 0:
        return evaluate(board) * colour
    for new_board in all_moves(board[:], colour):
        score = -next_move(new_board, -colour, -beta, -alpha, ply - 1)
        if score >= beta:
            return score
        if score > alpha:
            alpha = score
    return alpha

And I notice I don't get the strange problem on a non-recursive 
function ! Or at least I don't seam to.

Chris

[toc] | [prev] | [next] | [standalone]

#51057

Chris Hinsley wrote:

> On 2013-07-22 18:36:41 +0000, Chris Hinsley said:
> 
>> Folks, I have this decorator:
>> 
>> def memoize(maxsize):
>>     def _memoize(func):
>>         lru_cache = {}
>>         lru_list = []
> 
> Other clues, I use it on a recursive function:
> 
> @memoize(64)
> def next_move(board, colour, alpha, beta, ply):
>     if ply <= 0:
>         return evaluate(board) * colour
>     for new_board in all_moves(board[:], colour):
>         score = -next_move(new_board, -colour, -beta, -alpha, ply - 1)
>         if score >= beta:
>             return score
>         if score > alpha:
>             alpha = score
>     return alpha
> 
> And I notice I don't get the strange problem on a non-recursive
> function ! Or at least I don't seam to.

That's indeed the problem: 

>             if len(lru_list) >= maxsize:
>                 del(lru_cache[lru_list[0]])
>                 del(lru_list[0])
>             ret = func(*args, **kwargs)
>             lru_cache[key] = ret
>             lru_list.append(key)

You delete a cached item, then call the original function which causes calls 
of the decorated function. This causes a length check which sees the already 
reduced length and decides that the cache is not yet full.

If you remove the oldest item after calling the original function you should 
be OK.

[toc] | [prev] | [next] | [standalone]

#51058

On 2013-07-22 19:47:33 +0000, Peter Otten said:

> Chris Hinsley wrote:
> 
>> On 2013-07-22 18:36:41 +0000, Chris Hinsley said:
>> 
>>> Folks, I have this decorator:
>>> 
>>> def memoize(maxsize):
>>> def _memoize(func):
>>> lru_cache = {}
>>> lru_list = []
>> 
>> Other clues, I use it on a recursive function:
>> 
>> @memoize(64)
>> def next_move(board, colour, alpha, beta, ply):
>> if ply <= 0:
>> return evaluate(board) * colour
>> for new_board in all_moves(board[:], colour):
>> score = -next_move(new_board, -colour, -beta, -alpha, ply - 1)
>> if score >= beta:
>> return score
>> if score > alpha:
>> alpha = score
>> return alpha
>> 
>> And I notice I don't get the strange problem on a non-recursive
>> function ! Or at least I don't seam to.
> 
> That's indeed the problem:
> 
>> if len(lru_list) >= maxsize:
>> del(lru_cache[lru_list[0]])
>> del(lru_list[0])
>> ret = func(*args, **kwargs)
>> lru_cache[key] = ret
>> lru_list.append(key)
> 
> You delete a cached item, then call the original function which causes calls
> of the decorated function. This causes a length check which sees the already
> reduced length and decides that the cache is not yet full.
> 
> If you remove the oldest item after calling the original function you should
> be OK.

Ah ! Thank you kindly sir !

Chris

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web