Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #103220 > unrolled thread

downloading a CSV

Back to article view | Back to comp.lang.python

Contents

  downloading a CSV noydb <jenn.duerr@gmail.com> - 2016-02-19 10:46 -0800
    Re: downloading a CSV MRAB <python@mrabarnett.plus.com> - 2016-02-19 19:24 +0000
    Re: downloading a CSV Grant Edwards <invalid@invalid.invalid> - 2016-02-19 19:43 +0000
    Re: downloading a CSV noydb <jenn.duerr@gmail.com> - 2016-02-19 12:05 -0800
      Re: downloading a CSV Joel Goldstick <joel.goldstick@gmail.com> - 2016-02-19 16:03 -0500
    Re: downloading a CSV Tim Chase <python.list@tim.thechases.com> - 2016-02-19 18:47 -0600

#103220 — downloading a CSV

Greetings All,

Python v 3.4, windows

I want to be able to download this CSV file and save to disk
>> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv 

This (interacting with the web using python) is very new to me, so can anyone provide direction on how to go about doing this?  Is it involved?  Is there a good example out there that can illustrate the needs/mechanics/how-to?

Many thanks in advance!

[toc] | [next] | [standalone]

#103221

On 2016-02-19 18:46, noydb wrote:
> Greetings All,
>
> Python v 3.4, windows
>
> I want to be able to download this CSV file and save to disk
>>> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv
>
> This (interacting with the web using python) is very new to me, so can anyone provide direction on how to go about doing this?  Is it involved?  Is there a good example out there that can illustrate the needs/mechanics/how-to?
>
> Many thanks in advance!
>
Lookup "urlretrieve" in the help file.

[toc] | [prev] | [next] | [standalone]

#103222

On 2016-02-19, noydb <jenn.duerr@gmail.com> wrote:
> Greetings All,
>
> Python v 3.4, windows
>
> I want to be able to download this CSV file and save to disk
>>> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv 

Unless you really want to write a Python program, "wget" is a good solution:

    http://gnuwin32.sourceforge.net/packages/wget.htm

Just do this on a command line:

 wget http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv


Curl is another good option:

    https://curl.haxx.se/

Just do this at the command line:

 curl http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv > all_month.csv

-- 
Grant Edwards               grant.b.edwards        Yow! I just had my entire
                                  at               INTESTINAL TRACT coated
                              gmail.com            with TEFLON!

[toc] | [prev] | [next] | [standalone]

#103224

Thanks!  That was pretty easy.

import urllib.request
url = 'http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv'
urllib.request.urlretrieve(url, csv_file)
csv_file = r"C:\Temp\earthquakeAll_last30days.csv"
urllib.request.urlretrieve(url, csv_file)

I do want to use python -- there's much more I need to do with the file after downloading.

[toc] | [prev] | [next] | [standalone]

#103225

On Fri, Feb 19, 2016 at 3:05 PM, noydb <jenn.duerr@gmail.com> wrote:

> Thanks!  That was pretty easy.
>
> import urllib.request
> url = '
> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv'
> urllib.request.urlretrieve(url, csv_file)
> csv_file = r"C:\Temp\earthquakeAll_last30days.csv"
> urllib.request.urlretrieve(url, csv_file)
>
> I do want to use python -- there's much more I need to do with the file
> after downloading.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

You might look at the Requests library.  Its third party, but easier to
understand and use than urllib

-- 
Joel Goldstick
http://joelgoldstick.com/ <http://joelgoldstick.com/stats/birthdays>
http://cc-baseballstats.info/

[toc] | [prev] | [next] | [standalone]

#103229

On 2016-02-19 10:46, noydb wrote:
> I want to be able to download this CSV file and save to disk
> >> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv 

  from urllib.request import urlopen
  data = urlopen("http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv")

> This (interacting with the web using python) is very new to me, so
> can anyone provide direction on how to go about doing this?  Is it
> involved?  Is there a good example out there that can illustrate
> the needs/mechanics/how-to?

You can then either iterate over the incoming data and write it to a
file:

  with open('output.csv', 'wb') as output:
    output.writelines(data)

or you can process it as it comes in:

  import csv
  r = csv.DictReader(line.decode('utf8') for line in data)
  for i, row in enumerate(r):
    do_something(row)
    if i == 0: print(repr(r))

{'rms': '', 'gap': '158.52', 'latitude': '38.3625', 'magType': 'ml',
'net': 'nn', 'horizontalError': '', 'status': 'automatic', 'depth':
'3', 'nst': '5', 'magError': '', 'magSource': 'nn', 'depthError': '',
'longitude': '-118.4693', 'updated': '2016-02-20T00:17:41.069Z',
'mag': '1.1', 'type': 'earthquake', 'time':
'2016-02-20T00:15:24.120Z', 'id': 'nn00532282', 'locationSource':
'nn', 'dmin': '0.128', 'magNst': '', 'place': '22km SE of Hawthorne,
Nevada'}



-tkc

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web