Path: csiph.com!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: MRAB <python@mrabarnett.plus.com>
Newsgroups: comp.lang.python
Subject: Re: A fun python CLI program for all to enjoy!
Date: Fri, 6 May 2016 21:30:17 +0100
Lines: 89
Message-ID: <mailman.436.1462566630.32212.python-list@python.org>
References: <ngiq0u$brn$1@dont-email.me> <7e52b918-2087-f93f-43cb-3411c1cdc881@mrabarnett.plus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.0
In-Reply-To: <ngiq0u$brn$1@dont-email.me>
Precedence: list
Xref: csiph.com comp.lang.python:108237

On 2016-05-06 20:10, DFS wrote:
> getAddresses.py
>
> Scrapes addresses from www.usdirectory.com and stores them in a SQLite
> database, or writes them to text files for mailing labels, etc
>
> Now, just by typing 'fast food Taco Bell <city> 10 db all' you can find
> out how many Taco Bells are within 10 miles of you, and store all the
> addresses in your own address database.
>
> No more convoluted Googling, or hitting the 'Next Page' button, or
> fumbling with the Yellow Pages...
>
> Note: the db structure is flat on purpose, and the .csv files aren't
> quote delimited.
>
> Put the program in its own directory.  It creates the SQLite database
> there, and writes files there, too.
>
> Reviews of code, bug reports, criticisms, suggestions for improvement,
> etc are all welcome.
>
OK, you asked for it... :-)

1. It's shorter and clearer not to compare with True or False:

        if verbose:

    and:

        if not dupeRow:

2. You can print a blank line with an empty print statement:

        print

3. When looking for unique items, a set is a better choice than a list:

        addrCheck = set()

        def addrUnique(addr):
            if addr not in addrCheck:
                x = True
                addrCheck.add(addr)
            else:
                x = False
            return x

4. Try string formatting instead multiple concatenation:

        print "%s arguments" % argCnt

5. Strings have a .join method, and when you combine it with string slicing:

        keyw = "+".join(sys.argv[1 : argCnt - 5])

6. Another example of string formatting:

        search = "%s %s %s %s %s" % (keyw, cityzip, state, miles, addrWant)

7. It's recommended to use the 'with' statement when handling files:

        with open(webfile, "w") as f:
            if store == "csv":
                f.write("Name,Address,CityStateZip\n")

    If you don't want to use the 'with' statement, note that closing the 
file is:

            f.close()

    It needs the "()"!

8. When using SQL, you shouldn't try to insert the values yourself; you 
should use parametrised queries:

        cSQL = "DELETE FROM addresses WHERE datasrc = ? AND search = ?;"
        if verbose:
            print cSQL
        db.execute(cSQL, (datasrc, search))
        conn.commit()

     It'll insert the values where the "?" are and will do any necessary 
quoting itself. (Actually, some drivers use "?", others use "%s", so if 
it doesn't work with one, try the other.)

     The way you wrote it, it would fail if a value contained a "'". 
It's that kind of thing that leads to SQL injection attacks.