How can I export data from a website and write the contents to a text file?

X-Received	by 10.140.156.71 with SMTP id c68mr1967776qhc.3.1447864667454; Wed, 18 Nov 2015 08:37:47 -0800 (PST)
X-Received	by 10.50.126.99 with SMTP id mx3mr116474igb.1.1447864667422; Wed, 18 Nov 2015 08:37:47 -0800 (PST)
Path	csiph.com!au2pb.net!usenet.blueworldhosting.com!feeder01.blueworldhosting.com!peer01.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!b51no616043qgf.0!news-out.google.com!f6ni6409igq.0!nntp.google.com!i2no4748140igv.0!postnews.google.com!glegroupsg2000goo.googlegroups.com!not-for-mail
Newsgroups	comp.lang.python
Date	Wed, 18 Nov 2015 08:37:47 -0800 (PST)
Complaints-To	groups-abuse@google.com
Injection-Info	glegroupsg2000goo.googlegroups.com; posting-host=155.201.35.66; posting-account=QHCkKAoAAAAtwxtoSlGaj-ksHegzSKUu
NNTP-Posting-Host	155.201.35.66
User-Agent	G2/1.0
MIME-Version	1.0
Message-ID	<9365cf2f-e9c7-4338-83b4-ce3d1d7ce1d6@googlegroups.com> (permalink)
Subject	How can I export data from a website and write the contents to a text file?
From	ryguy7272 <ryanshuell@gmail.com>
Injection-Date	Wed, 18 Nov 2015 16:37:47 +0000
Content-Type	text/plain; charset=ISO-8859-1
X-Received-Bytes	1738
X-Received-Body-CRC	941722891
Xref	csiph.com comp.lang.python:98978

Show key headers only | View raw

I'm trying the script below, and it simple writes the last line to a text file.  I want to add a '\n' after each line is written, so I don't overwrite all the lines.


from bs4 import BeautifulSoup
import urllib2

var_file = urllib2.urlopen("http://www.imdb.com/chart/top")

var_html  = var_file.read()

var_file.close()
soup = BeautifulSoup(var_html)
for item in soup.find_all(class_='lister-list'):
    for link in item.find_all('a'):
        print(link)
        text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb")
        z = str(link)
        text_file.write(z + "\n")
        text_file.write("\n")
        text_file.close()


Can someone please help me get this working?
Thanks!!

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

csiph-web