Re: How can I export data from a website and write the contents to a text file?

Newsgroups	comp.lang.python
Date	2015-11-18 09:43 -0800
References	<9365cf2f-e9c7-4338-83b4-ce3d1d7ce1d6@googlegroups.com> <n2ibuh$bm9$2@dont-email.me> <6e0f470b-f896-43ae-8f83-b20f22a9db8d@googlegroups.com>
Message-ID	<e0edf996-9ce8-404e-b4e0-1e9a7b9af706@googlegroups.com> (permalink)
Subject	Re: How can I export data from a website and write the contents to a text file?
From	ryguy7272 <ryanshuell@gmail.com>

Show all headers | View raw

On Wednesday, November 18, 2015 at 12:41:19 PM UTC-5, ryguy7272 wrote:
> On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote:
> > On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote:
> > 
> > > I'm trying the script below...
> > 
> > The problem isn't that you're over-writing the lines (although it may 
> > seem that way to you), the problem is that you're overwriting the whole 
> > file every time you write a link to it. This is because you open and 
> > close the file for every link you write, and you do so in file mode "wb" 
> > which restarts writing at the first byte of the file every time.
> > 
> > You only need to open and close the text file once, instead of for every 
> > link you output. Try moving the lines to open and close the file outside 
> > the outer for loop to change the loop from:
> > 
> > for item in soup.find_all(class_='lister-list'):
> >     for link in item.find_all('a'):
> >         # open file
> >         # write link to file
> >         # close file
> > 
> > to:
> > 
> > # open file
> > for item in soup.find_all(class_='lister-list'):
> >     for link in item.find_all('a'):
> >         # write link to file
> > # close file
> > 
> > Alternatively, use the with form:
> > 
> > with open("blah","wb") as text_file:
> >     for item in soup.find_all(class_='lister-list'):
> >         for link in item.find_all('a'):
> >             # write link to file
> > 
> > -- 
> > Denis McMahon, 
> 
> 
> Yes, I just figured it out.  Thanks.  
> 
> It doesn't seem like the '\n' is doing anything useful.  All the text is jumbled together.  When I open the file in Excel, or Notepad++, it is easy to read.  However, when I open it in as a regular text file, everything is jumbled together.  Is there an easy way to fix this?

I finally got it working.  It's like this:
"\r\n"

Thanks everyone!!

Thread

csiph-web