Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!border1.nntp.ams2.giganews.com!border3.nntp.ams.giganews.com!border1.nntp.ams.giganews.com!nntp.giganews.com!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Newsgroups: comp.lang.python
Date: Thu, 21 Feb 2013 21:09:50 -0800 (PST)
In-Reply-To: <mailman.2162.1361451589.2939.python-list@python.org>
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=99.121.53.60; posting-account=XH8cbwoAAADsKhxZKJ-sG4EEz1Bcfsty
References: <34998ea2-6b19-4a98-8ea0-389aca0192ca@googlegroups.com> <mailman.2162.1361451589.2939.python-list@python.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Subject: Re: Urllib's urlopen and urlretrieve
From: qoresucks@gmail.com
To: comp.lang.python@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org, qoresucks@gmail.com
Precedence: list
Message-ID: <mailman.2238.1361509800.2939.python-list@python.org>
Lines: 119
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:39521

Initially I was just trying the html, but later when I attempted more compl=
icated sites that weren't my own I noticed that large bulks of the site wer=
e lost in the process. The urllib code essentially looks like what I was tr=
ying but it didn't work as I had expected.

To be more specific, after I got it working for my own little page, I attem=
pted to take it further and get all the lessons from Learn Python The Hard =
Way. When I tried the same method on the first intro page to see if I was e=
ven getting it right, the html code was all there but upon opening it I not=
iced the format was all wrong, colors were off for the background, images, =
etc... were all missing. So clearly I ended up misunderstanding something a=
nd its something critical I need to understand.=20

As for the OS, I primarily use Mac OS, however well versed in linux and win=
dows if there is anything specific out there that might help.=20

As for which version if Python, I have been using Python 2 to learn on as I=
 heard that Python 3 was still largely unadopted due to a lack of library s=
upport etc... by comparison. Are people adopting it fast enough now that I =
should consider learning on 3 instead of 2?

Also, it isn't so much to do it for technical reasons but rather I thought =
it would be something interesting and fun to learn some form of internet/ne=
twork programming. Granted, its not the best approach, but I'm not really a=
ware of too many others, and I it does seem interesting to me.=20

Python programming probably isn't the best way to initially approach this I=
 agree, but I wasn't sure what to research on or to get a better grasp of n=
etwork/internet/web programming so I figured I would just dive head first a=
nd figure things out, and reinforce more programming while learning interne=
t/network programming was my initial goal.=20

Thank you all for your responses though. :)


On Thursday, February 21, 2013 7:59:26 AM UTC-5, Michael Herman wrote:
> Are you just trying to get the html? If so, you can use this code-
>=20
>=20
>=20
> import urllib
>=20
>=20
> # fetch the and download a webpage, nameing it test.html
> urllib.urlretrieve("http://www.web2py.com/", filename=3D"test.html")
>=20
>=20
>=20
>=20
>=20
>=20
> I recommend using the requests library, as it's easier to use and more po=
werful:
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> import requests
>=20
> # retrive the webpage
> r =3D requests.get("http://www.web2py.com/")
>=20
> # write the content to test_request.html
> with open("test_requests.html", "wb") as code: =A0=20
>=20
>=20
>=20
>=20
>=20
> code.write(r.content)
>=20
> If you want to get up to speed quickly on internet programming, I have a =
course I am developing. It's on kickstarter -=A0http://kck.st/VQj8hq. The f=
irst section of the book dives into web fundamentals and internet programmi=
ng.=A0
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> On Thu, Feb 21, 2013 at 4:12 AM,  <qore...@gmail.com> wrote:
>=20
>=20
> I only just started Python and given that I know nothing about network pr=
ogramming or internet programming of any kind really, I thought it would be=
 interesting to try write something that could create an archive of a websi=
te for myself. With this I started trying to use the urllib library, howeve=
r I am having a problem understanding why certain things wont work with the=
 urllib.urlretrieve and urllib.urlopen then reading.
>=20
>=20
>=20
>=20
>=20
> Why is it that when using urllib.urlopen then reading or urllib.urlretrie=
ve, does it only give me parts of the sites, loosing the formatting, images=
, etc...? How can I get around this?
>=20
>=20
>=20
> Lastly, while its a bit off topic, I lack a good understanding of network=
 programming as a whole. From making programs communicate or to simply extr=
act data from URL's, I don't know where to even begin, which has lead me to=
 learning python to better understand it hopefully then carry it over to ot=
her languages I know. Can anyone give me some advice on where to begin lear=
ning this information? Even if its in another language.
>=20
>=20
>=20
> --
>=20
> http://mail.python.org/mailman/listinfo/python-list