Re: problem with netCDF4 OpenDAP

Path	csiph.com!news.mixmin.net!newsfeed.fsmpi.rwth-aachen.de!newsfeed.straub-nv.de!news-1.dfn.de!news.dfn.de!news.informatik.hu-berlin.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From	Tom P <werotizy@freent.dd>
Newsgroups	comp.lang.python
Subject	Re: problem with netCDF4 OpenDAP
Date	Fri, 14 Aug 2015 17:26:27 +0200
Lines	50
Message-ID	<d36fl4F1udiU1@mid.individual.net> (permalink)
References	<d33a27F7p5oU1@mid.individual.net> <mailman.169.1439481351.3627.python-list@python.org> <d35j2uFp5hdU1@mid.individual.net> <mailman.9.1439558127.4764.python-list@python.org>
Mime-Version	1.0
Content-Type	text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding	8bit
X-Trace	individual.net uktTWYp2/zKJKn31KncpQQN4YVF6pgyR+qAKuOo4TlO5d9RNo=
Cancel-Lock	sha1:7lPorMD/JkiVoHesYCpFMtnY2Oo=
User-Agent	Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0
In-Reply-To	<mailman.9.1439558127.4764.python-list@python.org>
Xref	csiph.com comp.lang.python:95376

Show key headers only | View raw

On 08/14/2015 03:15 PM, Jason Swails wrote:
>
>> On Aug 14, 2015, at 3:18 AM, Tom P <werotizy@freent.dd> wrote:
>>
>> Thanks for the reply but that is not what the documentation says.
>>
>> http://unidata.github.io/netcdf4-python/#section8
>> "Remote OPeNDAP-hosted datasets can be accessed for reading over http if a URL is provided to the netCDF4.Dataset constructor instead of a filename. However, this requires that the netCDF library be built with OPenDAP support, via the --enable-dap configure option (added in version 4.0.1).”
>
> Huh, so it does.  Your error message says "file not found", though, which suggested to me that it's trying to interpret the NetCDF file as a local file instead of a URL.  Indeed, when I run that example, the traceback is more complete (the traceback you printed had omitted some information):
>
>>>> netCDF4.Dataset('http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc')
> syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
> context: <!DOCTYPE^ HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"><html><head><title>404 Not Found</title></head><body><h1>Not Found</h1><p>The requested URL /pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc.dds was not found on this server.</p></body></html>
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "netCDF4/_netCDF4.pyx", line 1547, in netCDF4._netCDF4.Dataset.__init__ (netCDF4/_netCDF4.c:9551)
> RuntimeError: NetCDF: file not found
>
> So it’s clear that netCDF4 is at least *trying* to go online to look for the file, but it simply can’t find it.  Since the docs say it’s linking to libcurl, I tried using curl to download the file (curl -# http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc > test.nc) and it worked fine.  What’s more, it *seems* like the file (/pub/.../ersst.201507.nc.dds) was decorated with the ‘.dds’ suffix for some reason (not sure if the server redirected the request there or not).  But this looks like a netCDF4 issue.  Perhaps you can go to their project page on Github and file an issue there -- they will be more likely to have your answer than people here.
>
> HTH,
> Jason

Hi,
   yes the file does appear to be there, I can download it and I can 
open and read the URL using urllib. Since there are a whole bunch of 
files in the directory, I really need MFDataset, but according to the 
documentation that doesn't work with URLs. Maybe the solution really is 
to D/L them all into a temporary folder and use MFDataset.

>
>>
>> and for the Anaconda package -
>> http://docs.continuum.io/anaconda/changelog
>> "2013-05-08: 1.5.0:
>> Highlights:
>>   updates to all important packages: python, numpy, scipy, ipython, matplotlib, pandas, cython
>>   added netCDF4 (with OpenDAP support) on Linux and MacOSX"
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
> --
> Jason M. Swails
> BioMaPS,
> Rutgers University
> Postdoctoral Researcher
>

Thread

problem with netCDF4 OpenDAP Tom P <werotizy@freent.dd> - 2015-08-13 12:32 +0200
  Re: problem with netCDF4 OpenDAP Tom P <werotizy@freent.dd> - 2015-08-14 09:18 +0200
    Re: problem with netCDF4 OpenDAP Jason Swails <jason.swails@gmail.com> - 2015-08-14 09:15 -0400
      Re: problem with netCDF4 OpenDAP Tom P <werotizy@freent.dd> - 2015-08-14 17:26 +0200
        Re: problem with netCDF4 OpenDAP Michael Ströder <michael@stroeder.com> - 2015-08-15 02:02 +0200

csiph-web