Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Thu, 19 Jun 2014 12:25:23 +0100
From: MRAB <python@mrabarnett.plus.com>
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Not Responding When Dealing with Large Data
References: <00d330e3-fc8a-4b7e-b2bd-f1a48bc335c1@googlegroups.com> <1abc3e5d-8002-4ecc-b433-a862c4918e74@googlegroups.com> <mailman.11127.1403134519.18130.python-list@python.org> <69964062-b0a4-4153-a488-22202dea62bb@googlegroups.com>
In-Reply-To: <69964062-b0a4-4153-a488-22202dea62bb@googlegroups.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.11141.1403177316.18130.python-list@python.org>
Lines: 57
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:73413

On 2014-06-19 09:17, cutey Love wrote:
> update_idletasks didn't work.
>
> The code is this
>
>      file_path = filedialog.askopenfilename(filetypes=[('text files', '.txt')], multiple=True, defaultextension=".txt")
>
>      for path in file_path:
>
>          fo = open(path, "r")
>
>          for line in fo:
>              if myCase(line.lower()):
>                  myList.append(line.lower())
>          fo.close()
>
>
> def myCase(c):
>
>          if c in myList:
>              return False
>
>          if len(c) < 8 or len(c) > 80:
>              return False
>
> return True
>
>
>
> This processes a fair bit of data
>
It's quicker to look for something in a set than in a list, so if you
can use a set instead of a list, do so.

Also, checking the length of a string is quick, quicker than searching
a list.

Therefore, before processing the file, do:

     mySet = set(myList)

and then you can say:

     def myCase(c):
         if len(c) < 8 or len(c) > 80:
             return False

         if c in mySet:
             return False

         return True

which can be shortened to:

     def myCase(c):
         return 8 <= len(c) <= 80 and c in mySet