Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #63935

fork seems to make urlopen into a black hole?

Show all headers | View raw

A program took much too long to check some texts collected from web pages.
As this could be made parallel easily, I put in fork.
And the result seems to be that the program simply stops in the line with urlopen. Any suggestions?

Relevant part:

try:
            print 'urlopen by', kind_nummer, '- before'
            extern_adres = urlopen(adres).geturl()
            print 'urlopen by', kind_nummer, '- after'
        except IOError:
            tekst_string = 'URL "' + adres +'" geeft IO Error'
            print tekst_string, 'door', kind_nummer
            bloklijst_verslag.append(tekst_string + '\n')
            tekst_string = '\t**** Deze pagina wordt overgeslagen\n'
            bloklijst_verslag.append(tekst_string + '\n')
            adres = ''
        except:
            fout = sys.exc_info()[:2]
            tekst_string = 'URL "' + adres + '" geeft fout ' + fout
            print tekst_string, 'door', kind_nummer
            bloklijst_verslag.append(tekst_string + '\n')
            tekst_string = '\t**** Deze pagina wordt overgeslagen\n'
            bloklijst_verslag.append(tekst_string + '\n')
            adres = ''
        else:
            print 'urlopen by', kind_nummer, '- else'
            if extern_adres != adres:
                tekst_string = '\t**** redirect naar ' + extern_adres
                bloklijst_verslag.append(tekst_string + '\n')

From this block, the only response seen is the first print statement ("url open by .... - before") and then nothing else seems to happen in that child.
This happens for all children: the expected number of children is reached and they all reach (only) this point.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

fork seems to make urlopen into a black hole? BobAalsma <overhaalsgang_24_bob@me.com> - 2014-01-14 12:04 -0800
  Re: fork seems to make urlopen into a black hole? Chris Angelico <rosuav@gmail.com> - 2014-01-15 07:07 +1100

csiph-web