Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #75600 > unrolled thread

try/exception - error block

Back to article view | Back to comp.lang.python

Contents

  try/exception - error block bruce <badouglas@gmail.com> - 2014-08-03 10:29 -0400
    Re: try/exception - error block Roy Smith <roy@panix.com> - 2014-08-03 10:39 -0400
      Re: try/exception - error block Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-08-03 15:55 +0100
        Re: try/exception - error block Roy Smith <roy@panix.com> - 2014-08-03 12:27 -0400
          Re: try/exception - error block Chris Angelico <rosuav@gmail.com> - 2014-08-04 02:42 +1000
          Re: try/exception - error block Grant Edwards <invalid@invalid.invalid> - 2014-08-04 15:12 +0000
      Re: try/exception - error block CHIN Dihedral <dihedral88888@gmail.com> - 2014-08-03 14:13 -0700
      Re: try/exception - error block Cameron Simpson <cs@zip.com.au> - 2014-08-05 09:04 +1000
    Re: try/exception - error block Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-08-04 11:41 +1000

#75600 — try/exception - error block

Hi.

I have a long running process, it generates calls to a separate py
app. The py app appears to generate errors, as indicated in the
/var/log/messages file for the abrtd daemon.. The errors are
intermittent.

So, to quickly capture all possible exceptions/errors, I decided to
wrap the entire "main" block of the test py func in a try/exception
block.

This didn't work, as I'm not getting any output in the err file
generated in the exception block.

I'm posting the test code I'm using. Pointers/comments would be helpful/useful.

////////////////////
 the if that gets run is the fac1 logic which operates on the input
packet/data..
    elif (level=='collegeFaculty1'):
        #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
      ret=getParseCollegeFacultyList1(url,content)
////////////////////

Thanks.

if __name__ == "__main__":
# main app

  try:
    #college="asu"
    #url="https://webapp4.asu.edu/catalog"
    #termurl="https://webapp4.asu.edu/catalog/TooltipTerms.ext"


    #termVal=2141
    #
    # get the input struct, parse it, determine the level
    #

    #cmd='cat /apps/parseapp2/asuclass1.dat'
    #print "cmd= "+cmd
    #proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
    #content=proc.communicate()[0].strip()
    #print content
    #sys.exit()

    #s=getClasses(content)

    #print "arg1 =",sys.argv[0]
    if(len(sys.argv)<2):
      print "error\n"
      sys.exit()

    a=sys.argv[1]
    aaa=a

    #
    # data is coming from the parentApp.php
    #        data has been rawurlencode(json_encode(t))
    #        -reverse/split the data..
    #        -do the fetch,
    #        -save the fetched page/content if any
    #        -create the returned struct
    #        -echo/print/return the struct to the
    #         calling parent/call
    #

    ##print urllib.unquote_plus(a).decode('utf8')
    #print "\n"
    #print simplejson.loads(urllib.unquote_plus(a))
    z=simplejson.loads(urllib.unquote_plus(a))
    ##z=simplejson.loads(urllib.unquote(a).decode('utf8'))
    #z=simplejson.loads(urllib2.unquote(a).decode('utf8'))

    #print "aa \n"
    print z
    #print "\n bb \n"

    #
    #-passed in
    #
    url=str(z['currentURL'])
    level=str(z['level'])
    cname=str(z['parseContentFileName'])


    #
    # need to check the contentFname
    # -should have been checked in the parentApp
    # -check it anyway, return err if required
    # -if valid, get/import the content into
    #     the "content" var for the function/parsing
    #

    ##cmd='echo ${yolo_clientFetchOutputDir}/'
    cmd='echo ${yolo_clientParseInputDir}/'
    #print "cmd= "+cmd
    proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
    cpath=proc.communicate()[0].strip()

    cname=cpath+cname
    #print "cn = "+cname+"\n"
    #sys.exit()


    cmd='test -e '+cname+' && echo 1'
    #print "cmd= "+cmd
    proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
    c1=proc.communicate()[0].strip()

    if(not c1):
      #got an error - process it, return
      print "error in parse"

    #
    # we're here, no err.. got content
    #

    #fff= "sdsu2.dat"
    with open(cname,"r") as myfile:
      content=myfile.read()
      myfile.close()


    #-passed in
    #college="louisville"
    #url="http://htmlaccess.louisville.edu/classSchedule/"
    #termVal="4138"


    #print "term = "+str(termVal)+"\n"
    #print "url = "+url+"\n"

    #jtest()
    #sys.exit()

    #getTerm(url,college,termVal)


    ret={} # null it out to start
    if (level=='rState'):
      #ret=getTerm(content,termVal)
      ret=getParseStates(content)

    elif (level=='stateCollegeList'):
        #getDepts(url,college, termValue,termName)
      ret=getParseStateCollegeList(url,content)

    elif (level=='collegeFaculty1'):
        #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
      ret=getParseCollegeFacultyList1(url,content)

    elif (level=='collegeFaculty2'):
        #getClasses(url, college, termVal,termName,deptName,deptAbbrv)
      ret=getParseCollegeFacultyList2(content)



    #
    # the idea of this section.. we have the resulting
    #     fetched content/page...
    #

    a={}
    status=False
    if(ret['status']==True):

      s=ascii_strip(ret['data'])
      if(((s.find("</html")>-1) or (s.find("</HTML")>-1)) and
          ((s.find("<html")>-1) or (s.find("<HTML")>-1)) and
           level=='classSectionDay'):

        status=True
      #print "herh"
      #sys.exit()

      #
      # build the returned struct
      #
      #

      a['Status']=True
      a['recCount']=ret['count']
      a['data']=ret['data']
      a['nextLevel']=''
      a['timestamp']=''
      a['macAddress']=''
    elif(ret['status']==False):
      a['Status']=False
      a['recCount']=0
      a['data']=''
      a['nextLevel']=''
      a['timestamp']=''
      a['macAddress']=''

    res=urllib.quote(simplejson.dumps(a))
    ##print res

    name=subprocess.Popen('uuidgen -t', shell=True,stdout=subprocess.PIPE)
    name=name.communicate()[0].strip()
    name=name.replace("-","_")


  ##  if status==True:
    name2=tmpParseDir+"/rr_"+name+".dat"
    ofile1=open(name2,"w+")
    ofile1.write(res)
    ofile1.close()
    print name2

    if status==False:
      sname=tmpParseDir+"/serr_"+name+".dat"
      ofile1=open(sname,"w+")
      ofile1.write(aaa)
      ofile1.close()


    sys.exit()




    print "term = "+str(termVal)+"\n"
    print "url = "+url+"\n"

    getTerm(url,college,termVal)


    print "exit"
    sys.exit()

  except Exception, e:
    print e
    print "pycolFac1 - error!! \n";
    name=subprocess.Popen('uuidgen -t', shell=True,stdout=subprocess.PIPE)
    name=name.communicate()[0].strip()
    name=name.replace("-","_")
    name2="/home/ihubuser/parseErrTest/pp_"+name+".dat"
    ofile1=open(name2,"w+")
    ofile1.write(e)
    ofile1.write(aaa)
    ofile1.close()

    sys.exit()

[toc] | [next] | [standalone]

#75602

In article <mailman.12586.1407076172.18130.python-list@python.org>,
 bruce <badouglas@gmail.com> wrote:

> I'm posting the test code I'm using. Pointers/comments would be 
> helpful/useful.

It would be really helpful if you could post a minimal code example 
which demonstrates the problem you're having.  Leave out everything 
(including the commented-out code) which isn't necessary to demonstrate 
the behavior.

[toc] | [prev] | [next] | [standalone]

#75604

On 03/08/2014 15:39, Roy Smith wrote:
> In article <mailman.12586.1407076172.18130.python-list@python.org>,
>   bruce <badouglas@gmail.com> wrote:
>
>> I'm posting the test code I'm using. Pointers/comments would be
>> helpful/useful.
>
> It would be really helpful if you could post a minimal code example
> which demonstrates the problem you're having.  Leave out everything
> (including the commented-out code) which isn't necessary to demonstrate
> the behavior.
>

How to go about this is at "Short, Self Contained, Correct (Compilable), 
Example" at http://sscce.org/

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#75610

In article <mailman.12587.1407077774.18130.python-list@python.org>,
 Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:

> How to go about this is at "Short, Self Contained, Correct (Compilable), 
> Example" at http://sscce.org/

It says there, "most readers will stop reading by 100 lines of code".  I 
guess I have a short attention span relative to "most readers", because 
my tl;dnr threshold is a lot shorter than that.

The other advantage to coming up with a minimal example is that often 
the exercise of cutting your problem down to a minimal example is enough 
to allow you to figure it out for yourself :-)

[toc] | [prev] | [next] | [standalone]

#75611

On Mon, Aug 4, 2014 at 2:27 AM, Roy Smith <roy@panix.com> wrote:
> It says there, "most readers will stop reading by 100 lines of code".  I
> guess I have a short attention span relative to "most readers", because
> my tl;dnr threshold is a lot shorter than that.

"by" 100 lines includes everyone who stops at 10 lines, 40 lines, or
80 lines, too. What it means is that if you post >100 lines of code,
you've lost most of your audience.

ChrisA

[toc] | [prev] | [next] | [standalone]

#75696

On 2014-08-03, Roy Smith <roy@panix.com> wrote:
> In article <mailman.12587.1407077774.18130.python-list@python.org>,
>  Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
>
>> How to go about this is at "Short, Self Contained, Correct (Compilable), 
>> Example" at http://sscce.org/
>
> It says there, "most readers will stop reading by 100 lines of code".  I 
> guess I have a short attention span relative to "most readers", because 
> my tl;dnr threshold is a lot shorter than that.

Mine too.  My limit is about one screen full -- how many lines that is
varies, but is rarely more than 40 lines.

> The other advantage to coming up with a minimal example is that often 
> the exercise of cutting your problem down to a minimal example is enough 
> to allow you to figure it out for yourself :-)

That is very true.

-- 
Grant Edwards               grant.b.edwards        Yow! I am NOT a nut....
                                  at               
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#75619

On Sunday, August 3, 2014 10:39:19 PM UTC+8, Roy Smith wrote:
> In article <mailman.12586.1407076172.18130.python-list@python.org>,
> 
>  bruce <badouglas@gmail.com> wrote:
> 
> 
> 
> > I'm posting the test code I'm using. Pointers/comments would be 
> 
> > helpful/useful.
> 
> 
> 
> It would be really helpful if you could post a minimal code example 
> 
> which demonstrates the problem you're having.  Leave out everything 
> 
> (including the commented-out code) which isn't necessary to demonstrate 
> 
> the behavior.

Oh, I remember I was working with 
a professional programer who agreed
with me that a subroutine or function
longer than 80 lines in C/C++/PASCAL/Fortan, etc. must explain the 
reason in the commented preamble claearly for other programer to check
the necessity of the unusual length.

 

[toc] | [prev] | [next] | [standalone]

#75715

On 03Aug2014 10:39, Roy Smith <roy@panix.com> wrote:
>In article <mailman.12586.1407076172.18130.python-list@python.org>,
> bruce <badouglas@gmail.com> wrote:
>> I'm posting the test code I'm using. Pointers/comments would be
>> helpful/useful.
>
>It would be really helpful if you could post a minimal code example
>which demonstrates the problem you're having.  Leave out everything
>(including the commented-out code) which isn't necessary to demonstrate
>the behavior.

Just FYI, Bruce posted an earlier version of this question to 
users@lists.fedoraproject.org in late July, wondering why he didn't get a good 
error message in the abrt logs.

I asked then where the stderr of his python subprocess went, but never heard 
back.

Cheers,
Cameron Simpson <cs@zip.com.au>

Reality is that which refuses to go away when I stop believing in it.
- Phillip K. Dick

[toc] | [prev] | [next] | [standalone]

#75644

You also posted this question to the tutor@python.org mailing list, which is
where I gave an answer. This is a much better place, so I'll re-post the
most important parts of my answer here. If you read nothing else, scroll
down to the end and read the last part of my comment.


bruce wrote:

> I have a long running process, it generates calls to a separate py
> app. The py app appears to generate errors, as indicated in the
> /var/log/messages file for the abrtd daemon.. The errors are
> intermittent.

Well, what do the errors say?


[...]
> if __name__ == "__main__":
> # main app
>
>   try:
[deleting lots of commented out code]
>     if(len(sys.argv)<2):
>       print "error\n"
>       sys.exit()

You really should raise an exception on errors, but if you insist on
doing things this way, you should print to stderr, not stdout, and you
should exit with a non-zero status:

    print >>sys.stdout, "descriptive error messages are better\n"
    sys.exit(101)

>     a=sys.argv[1]
>     aaa=a

A minor stylistic thing: you can write this as:

    a = aaa = sys.argv[1]

but of course you really ought to use descriptive variable names rather
than cryptic "a" and "aaa" and "z" and other meaningless names.


[deleting more fossil code]
>     z=simplejson.loads(urllib.unquote_plus(a))
>     print z
>     url=str(z['currentURL'])
>     level=str(z['level'])
>     cname=str(z['parseContentFileName'])

>     cmd='echo ${yolo_clientParseInputDir}/'
>     proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
>     cpath=proc.communicate()[0].strip()

Hmmm. Are you trying to read the value of an environment variable using
subprocess? If so, then try this instead:

    cpath = os.getenv('yolo_clientParseInputDir')

If not, then sorry for the noise. Perhaps you could explain what your
call to echo in the shell is meant to do?

>     cname=cpath+cname

>     cmd='test -e '+cname+' && echo 1'
>     proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
>     c1=proc.communicate()[0].strip()

And here I think you're trying to test whether a file exists?

    os.path.exists(cname)

>     if(not c1):
>       #got an error - process it, return
>       print "error in parse"

Um, surely not? Surely the error is that the file doesn't exist, not
that it is a parsing error?

>     with open(cname,"r") as myfile:
>       content=myfile.read()
>       myfile.close()

If you use the "with open" form, there is no need to manually close the
file, it will be automatically closed for you.

    with open(cname,"r") as myfile:
        content = myfile.read()

is all you need (assuming you have permission to open the file, and that
it still exists).

[lots more fossils deleted]
>     ret={} # null it out to start
>     if (level=='rState'):
>       ret=getParseStates(content)
>     elif (level=='stateCollegeList'):
>       ret=getParseStateCollegeList(url,content)
>     elif (level=='collegeFaculty1'):
>       ret=getParseCollegeFacultyList1(url,content)
>     elif (level=='collegeFaculty2'):
>       ret=getParseCollegeFacultyList2(content)

I'm not really sure if any of that code is relevant to the problem
you're having.

>     a={}
>     status=False
>     if(ret['status']==True):
>       s=ascii_strip(ret['data'])
>       if(((s.find("</html")>-1) or (s.find("</HTML")>-1)) and
>           ((s.find("<html")>-1) or (s.find("<HTML")>-1)) and
>            level=='classSectionDay'):
>         status=True
>       a['Status']=True
>       a['recCount']=ret['count']
>       a['data']=ret['data']
>       a['nextLevel']=''
>       a['timestamp']=''
>       a['macAddress']=''
>     elif(ret['status']==False):
>       a['Status']=False
>       a['recCount']=0
>       a['data']=''
>       a['nextLevel']=''
>       a['timestamp']=''
>       a['macAddress']=''
>     res=urllib.quote(simplejson.dumps(a))

Your code will be much, much, much more readable if you use a reasonable
indent between levels. Four spaces rather than two, or a tab. I'm
finding it quite difficult to keep track of the levels when they are so
close together.

>     name=subprocess.Popen('uuidgen -t', shell=True,stdout=subprocess.PIPE)
>     name=name.communicate()[0].strip()
>     name=name.replace("-","_")
>     name2=tmpParseDir+"/rr_"+name+".dat"
>     ofile1=open(name2,"w+")
>     ofile1.write(res)
>     ofile1.close()
>     print name2

So does this file get written to?

Does name2 get printed?


>     if status==False:
>       sname=tmpParseDir+"/serr_"+name+".dat"
>       ofile1=open(sname,"w+")
>       ofile1.write(aaa)
>       ofile1.close()

How about this one? Does it get written to?

>     sys.exit()

Since you exit here, the rest of the code in the block is never
executed:

>     print "term = "+str(termVal)+"\n"
>     print "url = "+url+"\n"
>     getTerm(url,college,termVal)
>     print "exit"
>     sys.exit()

That's all dead code. I hope it isn't important.

>   except Exception, e:
>     print e
>     print "pycolFac1 - error!! \n";

Does that get printed?

>     name=subprocess.Popen('uuidgen -t', shell=True,stdout=subprocess.PIPE)
>     name=name.communicate()[0].strip()
>     name=name.replace("-","_")
>     name2="/home/ihubuser/parseErrTest/pp_"+name+".dat"
>     ofile1=open(name2,"w+")
>     ofile1.write(e)
>     ofile1.write(aaa)
>     ofile1.close()
>     sys.exit()

That's awfully ambitious code for an except clause that you're not even
sure is working. Simplify, simplify, simplify.

    except Exception, e:
        with open("/tmp/myerror.txt", "w") as f:
            f.write("%r" % e)
        sys.exit(102)


Now you should be able to see the error written to the file, which you
should have write privileges to unless you're doing something very
unusual.




-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web