Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #96754 > unrolled thread

pyserial and threads

Back to article view | Back to comp.lang.python

Contents

  pyserial and threads pozz <pozzugno@gmail.com> - 2015-09-17 11:28 +0200
    Re: pyserial and threads Chris Angelico <rosuav@gmail.com> - 2015-09-17 19:42 +1000
      Re: pyserial and threads pozz <pozzugno@gmail.com> - 2015-09-17 15:26 +0200
        Re: pyserial and threads Chris Angelico <rosuav@gmail.com> - 2015-09-17 23:45 +1000
    Re: pyserial and threads alister <alister.nospam.ware@ntlworld.com> - 2015-09-17 12:00 +0000
      Re: pyserial and threads Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-09-17 09:04 -0400
        Re: pyserial and threads pozz <pozzugno@gmail.com> - 2015-09-17 15:23 +0200
      Re: pyserial and threads pozz <pozzugno@gmail.com> - 2015-09-17 15:22 +0200

#96754 — pyserial and threads

I'm trying to create a simple program in Python that opens N serial 
ports (through pyserial) and forward every byte received on one of those 
ports to the other ports.

At startup I open the ports and create and start a thread to manage the 
receiving. When a byte is received, I call the .write() method for all 
the other ports.

It works, but sometimes it seems to block. I think I haven't used 
correctly the threads.

Below is my code, I hope someone can help me.

Consider that I'm a newbie in python and I never used threads before.


import serial
import threading
import sys, os
import signal
import time

class Miniterm(object):
   def __init__(self, port, baudrate):
     self.serial = serial.Serial(port, baudrate, timeout=1)

   def start(self, com_list):
     self.alive = True
     self.com_list = com_list
     self._reader_alive = True
     self.receiver_thread = threading.Thread(target=self.reader)
     self.receiver_thread.setDaemon(True)
     self.receiver_thread.start()

   def stop(self):
     self.alive = False

   def reader(self):
     try:
       while self.alive and self._reader_alive:
         data = self.serial.read(1)
           if len(data) > 0:
           for p in self.com_list:
             if p[1] != self:
               p[1].write(data)
     except serial.SerialException:
       self.alive = False
       raise

   def write(self, data):
     self.serial.write(data)
	
if __name__ == "__main__":
   ports = []
   for com in sys.argv[1:]:
     try:
       miniterm = Miniterm(com, 38400)
     except serial.SerialException:
       sys.stderr.write("could not open port " + com)
       sys.exit(1)
     ports.append((com, miniterm))
     for p in ports:
       p[1].start(ports)
       print("Port " + p[0] + " has started", flush=True)
     while True:
       time.sleep(1)

[toc] | [next] | [standalone]

#96755

On Thu, Sep 17, 2015 at 7:28 PM, pozz <pozzugno@gmail.com> wrote:
> At startup I open the ports and create and start a thread to manage the
> receiving. When a byte is received, I call the .write() method for all the
> other ports.
>
> It works, but sometimes it seems to block. I think I haven't used correctly
> the threads.
>

Seems a fairly reasonable model. From what I'm seeing here, you start
a thread to read from each serial port, but then those threads will
make blocking writes to all the other serial ports. Is it possible
that one of them is getting full?

When I do this kind of thing with TCP/IP sockets, I usually end up
having to go non-blocking in both directions, and maintaining a buffer
of unsent text for each socket. You may find that you need to do the
same thing here.

Where's the code getting blocked?

ChrisA

[toc] | [prev] | [next] | [standalone]

#96769

Il 17/09/2015 11:42, Chris Angelico ha scritto:
> On Thu, Sep 17, 2015 at 7:28 PM, pozz <pozzugno@gmail.com> wrote:
>> At startup I open the ports and create and start a thread to manage the
>> receiving. When a byte is received, I call the .write() method for all the
>> other ports.
>>
>> It works, but sometimes it seems to block. I think I haven't used correctly
>> the threads.
>>
>
> Seems a fairly reasonable model. From what I'm seeing here, you start
> a thread to read from each serial port, but then those threads will
> make blocking writes to all the other serial ports. Is it possible
> that one of them is getting full?
>
> When I do this kind of thing with TCP/IP sockets, I usually end up
> having to go non-blocking in both directions, and maintaining a buffer
> of unsent text for each socket. You may find that you need to do the
> same thing here.

How to have a non-blocking write?

Maybe the problem happens when port 1 thread is in .read() (it blocks 
for 1 second) and port 2 thread tries to write one byte (that was just 
received) to port 1.


> Where's the code getting blocked?

I don't knwo exactly. I can only see no more bytes are received on COM 
ports.

[toc] | [prev] | [next] | [standalone]

#96770

On Thu, Sep 17, 2015 at 11:26 PM, pozz <pozzugno@gmail.com> wrote:
> How to have a non-blocking write?
>
> Maybe the problem happens when port 1 thread is in .read() (it blocks for 1
> second) and port 2 thread tries to write one byte (that was just received)
> to port 1.

I'm not sure, as I've never worked with serial ports in this way. What
you'd want is some form of call that says "write these bytes if you
can, but don't if you can't, and just tell me how many you wrote". A
quick look at the pyserial docs suggests that you may be able to
accomplish this by opening the port with writeTimeout=0, or possibly
some other value (it'll wait that many seconds - float allowed -
before giving up).

If it returns 0, stating that the byte wasn't written, you'd need to
hang onto that byte until it can write successfully. I've no idea how
you'd go about knowing that. With TCP sockets, select.select() is your
friend; if you're really lucky, pyserial will work with the same kind
of structure.

>> Where's the code getting blocked?
>
> I don't knwo exactly. I can only see no more bytes are received on COM
> ports.

Here's a way to test: Bracket each potentially-blocking call with a
status update, and then have your main loop collect the statuses and
print them out. Something like this:

  def reader(self):
    try:
      while self.alive and self._reader_alive:
        self.status = 'R' # Reading
        data = self.serial.read(1)
        self.status = 'P' # Processing
        if len(data) > 0:
          for n,p in enumerate(self.com_list):
            if p[1] != self:
              self.status = n # Writing to port n
              p[1].write(data)
              self.status = 'P'
    except serial.SerialException:
      # This looks like a job for try/finally, actually
      self.status = 'Z' # Dead
      self.alive = False
      raise

Then your main thread, instead of just sleeping forever, does this:

    while True:
      time.sleep(1)
      print(" ".join(port.status for port in ports), end="\r", flush=True)

You should normally see most of the threads blocked on reading,
assuming that the traffic levels are lower than the ports' capacities.
If you start seeing them blocked on writing, chances are they'll all
be blocking on the same port, and that's the port that's holding you
up.

Caution: Code utterly untested. You may have to debug some stuff.

ChrisA

[toc] | [prev] | [next] | [standalone]

#96760

On Thu, 17 Sep 2015 11:28:04 +0200, pozz wrote:

> I'm trying to create a simple program in Python that opens N serial
> ports (through pyserial) and forward every byte received on one of those
> ports to the other ports.
> 
> At startup I open the ports and create and start a thread to manage the
> receiving. When a byte is received, I call the .write() method for all
> the other ports.
> 
> It works, but sometimes it seems to block. I think I haven't used
> correctly the threads.
> 
> Below is my code, I hope someone can help me.
> 
> Consider that I'm a newbie in python and I never used threads before.
> 
> 
> import serial import threading import sys, os import signal import time
> 
> class Miniterm(object):
>    def __init__(self, port, baudrate):
>      self.serial = serial.Serial(port, baudrate, timeout=1)
> 
>    def start(self, com_list):
>      self.alive = True self.com_list = com_list self._reader_alive =
>      True self.receiver_thread = threading.Thread(target=self.reader)
>      self.receiver_thread.setDaemon(True) self.receiver_thread.start()
> 
>    def stop(self):
>      self.alive = False
> 
>    def reader(self):
>      try:
>        while self.alive and self._reader_alive:
>          data = self.serial.read(1)
>            if len(data) > 0:
>            for p in self.com_list:
>              if p[1] != self:
>                p[1].write(data)
>      except serial.SerialException:
>        self.alive = False raise
> 
>    def write(self, data):
>      self.serial.write(data)
> 	
> if __name__ == "__main__":
>    ports = []
>    for com in sys.argv[1:]:
>      try:
>        miniterm = Miniterm(com, 38400)
>      except serial.SerialException:
>        sys.stderr.write("could not open port " + com) sys.exit(1)
>      ports.append((com, miniterm))
>      for p in ports:
>        p[1].start(ports)
>        print("Port " + p[0] + " has started", flush=True)
>      while True:
>        time.sleep(1)

I would like to know more about how many serial ports are connected & 
what the equipment they are connected to does and expects.

I can see the data being transmitted snowballing & running away in a +ve 
feedback loop very easily.




-- 
The only "ism" Hollywood believes in is plagiarism.
		-- Dorothy Parker

[toc] | [prev] | [next] | [standalone]

#96766

On Thu, 17 Sep 2015 12:00:08 +0000 (UTC), alister
<alister.nospam.ware@ntlworld.com> declaimed the following:


>I can see the data being transmitted snowballing & running away in a +ve 
>feedback loop very easily.

	Especially if a few of the remote devices are configured to ECHO
data... <G>

	Main thing I'd probably change is... Since the COM port identification
is already being provided during initialization of the handler object, why
maintain a list of (com, handler) pairs, and the subsequent subscripting --
just save the com port as an attribute of the object.

	One could also make a copy of the object list in the start method, and
at that point, scan the list and remove that one's own identity. That would
remove the need for always testing "is the object I'm about to send to
really me?"
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#96768

Il 17/09/2015 15:04, Dennis Lee Bieber ha scritto:
> On Thu, 17 Sep 2015 12:00:08 +0000 (UTC), alister
> <alister.nospam.ware@ntlworld.com> declaimed the following:
>
>
>> I can see the data being transmitted snowballing & running away in a +ve
>> feedback loop very easily.
>
> 	Especially if a few of the remote devices are configured to ECHO
> data... <G>

No ECHO.


> 	Main thing I'd probably change is... Since the COM port identification
> is already being provided during initialization of the handler object, why
> maintain a list of (com, handler) pairs, and the subsequent subscripting --
> just save the com port as an attribute of the object.
>
> 	One could also make a copy of the object list in the start method, and
> at that point, scan the list and remove that one's own identity. That would
> remove the need for always testing "is the object I'm about to send to
> really me?"

Ok, they are optimizations, but I don't think they solve my issue.

[toc] | [prev] | [next] | [standalone]

#96767

Il 17/09/2015 14:00, alister ha scritto:

> I would like to know more about how many serial ports are connected

One real serial port and two virtual serial ports, created by com0com 
(it's a free virtual serial port for Windows).


> what the equipment they are connected to does and expects.

Raw bytes arranged in a well defined protocol. I'm the author of the 
protocol and equipments :-)


> I can see the data being transmitted snowballing & running away in a +ve
> feedback loop very easily.

No, because the byte received by first COM port is forwarded/transmitted 
to all the OTHERS COM ports in the list.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web