Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #102691

Help using Thread (or other method)

Show key headers only | View raw

Hi.  Need some urgent help.

I have a python app that uses `select` to wait for data from an arm
embedded linux kernel interrupt (every second).  The mainloop of the app
then grabs data from some memory mapped area, processes it and then does
a http post to a server.

The problem is the http post can take multiple seconds to respond and
blocks (using the requests module).  So I've put the posting in a
separate thread (using Thread module) and send data to it via a queue
(using the Queue module).  The http post can send multiple records by
pulling multiple items off the queue.  Sounds good so far, right?

This seems to work ok, but I also need to grab data from some other
serial devices (gps, weather station) to put in the post.  Various
sample apps on the web use Threads to do this, so I am doing the same. 
One of the threads uses `select` (see below), following by readlines()
on the device.

       def __init__(self):
        select_timeout = 1
        serial_timeout = 0.1

    def wait_and_process(self):
        '''wait for data and process it.'''
        r = select.select([self.serial_dev], [], [], self.select_timeout)
        if not r[0]:
            #print("DEBUG: TIMEOUT: wait_and_process")
            return

        #print("DEBUG: Weather Data Captured")
        for s in self.serial_dev.readlines():
            #print("INFO: Data: {}".format(s))

            temperature = extract_temperature(s)

Is using `select` here redundant?  It's used to block the thread until
the first character is ready to be read from the serial device?
Then I use serial_dev.readlines() to read the stream of chars from the
weather station.
Is that ok?

Does readline block until it sees an end-of-line char?  i.e. does it
only wake up the thread when it has a _complete string_ or a timeout of
0.1 seconds?
or will the process block other threads from running while it is
gathering the chars until the newline?
I'm assuming the gathering of chars will happen in the background and
allow my main thread to run, correct?

My application mainloop (not the threads) needs to be high priority and
always process as soon as it gets an interrupt.  Is using select a good
way of doing this?  How can I ensure that no other threads are utilizing
the CPU, etc?  I'm worried about the GIL (I'm using CPython 2.7).

Is there a better way of structuring this to ensure I process the
interrupt and not get interrupt overrun?

Is using select only in the mainloop, with multiple file descriptors, a
better way of doing things, so that I can process the file descriptor of
interest first, before any others if set?

Is using the multiprocessing module a better option?

Thanks for any advice,
Brendan.

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Help using Thread (or other method) "Brendan Simon (eTRIX)" <brendan.simon@etrix.com.au> - 2016-02-09 07:59 +1100
  Re: Help using Thread (or other method) Marko Rauhamaa <marko@pacujo.net> - 2016-02-08 23:27 +0200

csiph-web