Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #55742 > unrolled thread

python crash problem

Back to article view | Back to comp.lang.python

Contents

  python crash problem Yelena <yelena.mqw@gmail.com> - 2011-02-02 11:19 -0800
    Re: python crash problem Yelena <yelena.mqw@gmail.com> - 2011-02-02 11:21 -0800
    Re: python crash problem Emile van Sebille <emile@fenx.com> - 2011-02-02 13:40 -0800

#55742 — python crash problem

I've been using python to run some large database recently.
I used dbfpy module to access some information stored in dbf format.
However, after 20hrs of running it gave me an error of :
Runtime Error!
This application has requested the Runtime to terminate it in an
unusual way.Please contact the application's support team for more
information.

The error popped out every time I ran them after 20 hrs. My prediction
was that
it ran over its memory usage, since it has been accumulating the
memory usage
when I was running the program. So I tried deleting/closing I/O
immediately
after they were processed. However, such changes did not make any
difference
in memory usage accumulation.

Could you please give me some advice in fixing problems like this?

Thank you

[toc] | [next] | [standalone]

#55759

Here is the code, I edited out some private info:

import sys
import urllib2
import string
import re
import datetime
from mx import DateTime
from dbfpy import dbf

addylist = ['add.dbf']
okset = ['P','L','I','B']
for addy in addylist:
    db = dbf.Dbf(addy)
    print (addy)
    count = len(db)
    db.close()
    del db
    foo = ((0, 1000),(1000,count/3), (count/3, count/3*2), (count/3*2,
count))
    for (start,finish) in foo:
        ab = dbf.Dbf(addy)
        for x in range(start, finish):
            print x
            rec = ab[x]
            if rec["flag"] in [1,3]:
                INP_info = rec["address"] + " " + rec["city"] + " "
+rec["state"] + " " + rec["zip"]
                INP_info = INP_info.replace(' ',',')
                HOST_URL = '***'
                KEY = '***'
                STD_URL = '***'
                STD_URL = STD_URL.replace('location=','location=' +
INP_info)
                STD_URL = STD_URL.replace('key=','key=' + KEY)
                Addy_INPUT = HOST_URL + STD_URL
                try:
                    for line in urllib2.urlopen(Addy_INPUT):
                        safe = line
                        geoQ = safe.find('\"geocodeQualityCode\"',1)
                        geoCode = safe[(geoQ+22):(geoQ+26)]
                        if geoCode[0] in okset:
                            lat = safe.find('\"lat\":', 1)
                            lng = safe.find('\"lng\":', 1)
                            lngbound = safe.find('\"mapUrl":', 1)
                            zipCode = safe.find('\"postalCode\"', 1)
                            zipbound = safe.find('\"adminArea1\"', 1)
                            rec["lat"] = float(safe[(lat + 6):
(lng-1)])
                            rec["long"] = float(safe[(lng+6):
(lngbound-2)])
                            rec["zip_final"] = safe[(zipCode+14):
(zipbound-2)]
                            rec["flag"] = 2
                            print ('ok ')
                        else:
                            rec["flag"] = 5
                except:
                    rec["flag"] = 5
                rec.store()
                del rec
        ab.close()
        del ab

[toc] | [prev] | [next] | [standalone]

#55847

On 2/2/2011 11:19 AM Yelena said...
> This application has requested the Runtime to terminate it in an
> unusual way

This is a MS message  -- did you look in the application event viewer?

Otherwise, you could try upping/reducing memory and confirm it dies 
later/earlier.  Perhaps there's an at or scheduled job killing off 
python processes?  How big is count?  Does this job run for 20 hours 
straight? Or is it launched periodically?  Why the foo fudging?  Just to 
break the process into pieces? You could have the job log it's record 
number after N hours (where N<20) and simply restart it...

Emile

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web