Re: downloading a CSV

Path	csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From	Tim Chase <python.list@tim.thechases.com>
Newsgroups	comp.lang.python
Subject	Re: downloading a CSV
Date	Fri, 19 Feb 2016 18:47:59 -0600
Lines	43
Message-ID	<mailman.8.1455929895.13884.python-list@python.org> (permalink)
References	<1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com>
Mime-Version	1.0
Content-Type	text/plain; charset=US-ASCII
Content-Transfer-Encoding	7bit
X-Trace	news.uni-berlin.de UU5mfNSg8VfJ5Gc7g6bGhQ5LQJVyI0oGQm95eQ0vEYpg==
Return-Path	<python.list@tim.thechases.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.012
X-Spam-Evidence	'H': 0.98; 'S': 0.00; 'python)': 0.05; "'',": 0.07; 'csv': 0.09; 'iterate': 0.09; 'subject:CSV': 0.09; '-tkc': 0.16; 'data)': 0.16; 'from:addr:python.list': 0.16; 'from:addr:tim.thechases.com': 0.16; 'from:name:tim chase': 0.16; 'involved?': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'row': 0.16; 'wrote:': 0.16; 'file:': 0.22; 'import': 0.24; 'header:In-Reply-To:1': 0.24; 'example': 0.26; 'disk': 0.27; 'url:gov': 0.27; "skip:' 10": 0.28; 'anyone': 0.32; 'received:10.43': 0.33; "skip:' 20": 0.34; 'file': 0.34; 'this?': 0.34; 'skip:c 30': 0.35; 'direction': 0.35; 'there': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'received:10': 0.37; 'charset:us-ascii': 0.37; 'doing': 0.38; 'skip:o 20': 0.38; 'data': 0.39; 'to:addr:python.org': 0.40; 'save': 0.60; 'skip:u 10': 0.61; 'provide': 0.61; 'url:0': 0.63; 'incoming': 0.70; "'3',": 0.84; 'received:23': 0.84; 'url:v1': 0.84
X-Sender-Id	wwwh\|x-authuser\|tim@thechases.com
X-Sender-Id	wwwh\|x-authuser\|tim@thechases.com
X-MC-Relay	Neutral
X-MailChannels-SenderId	wwwh\|x-authuser\|tim@thechases.com
X-MailChannels-Auth-Id	wwwh
X-MC-Loop-Signature	1455929446568:3018359909
X-MC-Ingress-Time	1455929446568
In-Reply-To	<1efa02db-a26a-44bd-a0ea-db81e612ccd6@googlegroups.com>
X-Mailer	Claws Mail 3.11.1 (GTK+ 2.24.25; x86_64-pc-linux-gnu)
X-AuthUser	tim@thechases.com
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.21rc2
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref	csiph.com comp.lang.python:103229

Show key headers only | View raw

On 2016-02-19 10:46, noydb wrote:
> I want to be able to download this CSV file and save to disk
> >> http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv 

  from urllib.request import urlopen
  data = urlopen("http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv")

> This (interacting with the web using python) is very new to me, so
> can anyone provide direction on how to go about doing this?  Is it
> involved?  Is there a good example out there that can illustrate
> the needs/mechanics/how-to?

You can then either iterate over the incoming data and write it to a
file:

  with open('output.csv', 'wb') as output:
    output.writelines(data)

or you can process it as it comes in:

  import csv
  r = csv.DictReader(line.decode('utf8') for line in data)
  for i, row in enumerate(r):
    do_something(row)
    if i == 0: print(repr(r))

{'rms': '', 'gap': '158.52', 'latitude': '38.3625', 'magType': 'ml',
'net': 'nn', 'horizontalError': '', 'status': 'automatic', 'depth':
'3', 'nst': '5', 'magError': '', 'magSource': 'nn', 'depthError': '',
'longitude': '-118.4693', 'updated': '2016-02-20T00:17:41.069Z',
'mag': '1.1', 'type': 'earthquake', 'time':
'2016-02-20T00:15:24.120Z', 'id': 'nn00532282', 'locationSource':
'nn', 'dmin': '0.128', 'magNst': '', 'place': '22km SE of Hawthorne,
Nevada'}



-tkc

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

downloading a CSV noydb <jenn.duerr@gmail.com> - 2016-02-19 10:46 -0800
  Re: downloading a CSV MRAB <python@mrabarnett.plus.com> - 2016-02-19 19:24 +0000
  Re: downloading a CSV Grant Edwards <invalid@invalid.invalid> - 2016-02-19 19:43 +0000
  Re: downloading a CSV noydb <jenn.duerr@gmail.com> - 2016-02-19 12:05 -0800
    Re: downloading a CSV Joel Goldstick <joel.goldstick@gmail.com> - 2016-02-19 16:03 -0500
  Re: downloading a CSV Tim Chase <python.list@tim.thechases.com> - 2016-02-19 18:47 -0600

csiph-web