Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #43160

newbie question about confusing exception handling in urllib

Show all headers | View raw

Hi,

I have been using Java/Perl professionally for many years and have been trying to learn python3 recently. As my first program, I tried writing a class for a small project, and I am having really hard time understanding exception handling in urllib and in python in general...
Basically, what I want to do is very simple, try to fetch something "tryurllib.request.urlopen(request)", and:
  - If request times out or connection is reset, re-try n times
  - If it fails, return an error
  - If it works return the content.

But, this simple requirement became a nightmare for me. I am really confused about how I should be checking this because:
  - When connection times out, I sometimes get URLException with "reason" field set to socket.timeout, and checking (isinstance(exception.reason, socket.timeout)) works fine
  - But sometimes I get socket.timeout exception directly, and it has no "reason" field, so above statement fails, since there is no reason field there.
  - Connection reset is a totally different exception
  - Not to mention, some exceptions have msg / reason / errno fields but some don't, so there is no way of knowing exception details unless you check them one by one. The only common thing I could was to find call __str__()?
  - Since, there are too many possible exceptions, you need to catch BaseException (I received URLError, socket.timeout, ConnectionRefusedError, ConnectionResetError, BadStatusLine, and none share a common parent). And, catching the top level exception is not a good thing.

So, I ended up writing the following, but from everything I know, this looks really ugly and wrong???

        try: 
            response = urllib.request.urlopen(request)
            content = response.read()
        except BaseException as ue:
            if (isinstance(ue, socket.timeout) or (hasattr(ue, "reason") and isinstance(ue.reason, socket.timeout)) or isinstance(ue, ConnectionResetError)):
                print("REQUEST TIMED OUT")

or, something like:

        except:
            (a1,a2,a3) = sys.exc_info()
            errorString = a2.__str__()
            if ((errorString.find("Connection reset by peer") >= 0) or (errorString.find("error timed out") >= 0)):

Am I missing something here? I mean, is this really how I should be doing it?

Thanks.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

newbie question about confusing exception handling in urllib cabbar@gmail.com - 2013-04-09 04:41 -0700
  Re: newbie question about confusing exception handling in urllib Peter Otten <__peter__@web.de> - 2013-04-09 14:19 +0200
  Re: newbie question about confusing exception handling in urllib cabbar@gmail.com - 2013-04-09 06:19 -0700
    Re: newbie question about confusing exception handling in urllib Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-04-09 15:05 +0000
      Re: newbie question about confusing exception handling in urllib Chris Angelico <rosuav@gmail.com> - 2013-04-10 02:23 +1000
      RE: newbie question about confusing exception handling in urllib "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2013-04-12 21:29 +0000
  Re: newbie question about confusing exception handling in urllib Terry Jan Reedy <tjreedy@udel.edu> - 2013-04-09 10:19 -0400
  Re: newbie question about confusing exception handling in urllib Ian Kelly <ian.g.kelly@gmail.com> - 2013-04-09 13:11 -0600

csiph-web