Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #16007 > unrolled thread

Non-POSIX parity (mark/space) with Python-Serial on Linux.

Back to article view | Back to comp.lang.python

Contents

  Non-POSIX parity (mark/space) with Python-Serial on Linux. mlenz@nocturnal.org - 2011-11-21 06:00 -0800
    Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 06:16 -0800
    Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Nizamov Shawkat <nizamov.shawkat@gmail.com> - 2011-11-21 15:26 +0100
      Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 08:28 -0800
        Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Chris Angelico <rosuav@gmail.com> - 2011-11-22 03:41 +1100
        Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. David Riley <fraveydank@gmail.com> - 2011-11-21 11:47 -0500
          Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 08:52 -0800
          Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 08:52 -0800
            Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. David Riley <fraveydank@gmail.com> - 2011-11-21 12:22 -0500
              Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 09:59 -0800
              Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 09:59 -0800
                Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. David Riley <fraveydank@gmail.com> - 2011-11-21 13:12 -0500
            Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. MRAB <python@mrabarnett.plus.com> - 2011-11-21 18:20 +0000
        Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. gene heskett <gheskett@wdtv.com> - 2011-11-21 12:25 -0500
        Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. David Riley <fraveydank@gmail.com> - 2011-11-21 12:50 -0500
      Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 08:28 -0800
    Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. gene heskett <gheskett@wdtv.com> - 2011-11-21 13:33 -0500
      Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 11:29 -0800
        Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Grant Edwards <invalid@invalid.invalid> - 2011-11-21 19:42 +0000
      Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Matthew Lenz <matthew@nocturnal.org> - 2011-11-21 11:29 -0800
        Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. David Riley <fraveydank@gmail.com> - 2011-11-21 15:42 -0500
          Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Grant Edwards <invalid@invalid.invalid> - 2011-11-21 23:08 +0000
            Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Grant Edwards <invalid@invalid.invalid> - 2011-11-21 23:09 +0000
              Re: Non-POSIX parity (mark/space) with Python-Serial on Linux. Chris Angelico <rosuav@gmail.com> - 2011-11-22 11:32 +1100

Page 1 of 2  [1] 2  Next page →

#16007 — Non-POSIX parity (mark/space) with Python-Serial on Linux.

I'm working on a project where I need to communicate with some devices via modem which have the possibility of using MARK and SPACE parity.  These are not defined by POSIX and therefore are not directly supported under Linux.

I've found the following discussion on the topic:

http://www.lothosoft.ch/thomas/libmip/markspaceparity.php

and I have been trying to use this information (since the TERMIOS module is available) to proceed with the communication.

I was able to use minicom to determine that the first device I started testing with uses 7M1 but cannot figure out how to implement the solution described by the author above.

Any pointers would be greatly appreciated.

[toc] | [next] | [standalone]

#16008

I should also note that I am aware of the following discussion on the newsgroup:

https://groups.google.com/d/msg/comp.lang.python/1HyCqPSOf50/eQINFrrFKwoJ

However, I believe this refers to implementing the solution for 8M1 and 8S1.

[toc] | [prev] | [next] | [standalone]

#16009

2011/11/21  <mlenz@nocturnal.org>:
> I'm working on a project where I need to communicate with some devices via modem which have the possibility of using MARK and SPACE parity.  These are not defined by POSIX and therefore are not directly supported under Linux.
>
> I've found the following discussion on the topic:
>
> http://www.lothosoft.ch/thomas/libmip/markspaceparity.php
>
> and I have been trying to use this information (since the TERMIOS module is available) to proceed with the communication.
>
> I was able to use minicom to determine that the first device I started testing with uses 7M1 but cannot figure out how to implement the solution described by the author above.
>


"The modes 7M1 (7 data bits, MARK parity, 1 stop bit) and 7S1 (7 data
bits, SPACE parity, 1 stop bit) can easily be emulated using 8N1 (0
data bits, NO parity, 1 stop bit) and setting the 8th data bit to 1
resp. 0. This is relatively simple to implement and cannot be
distinguished by the receiver."

It means that 7M1 === 8N1. Set 8N1 mode on your side and 7M1 on the
other side. I really do not understand what is the reason to have
dedicated 7M1 or 7S1 mode - it is no different from regular 8 bit mode
from the hardware point of view. From the software point of view it is
just the matter of the definition of the highest bit. In other words,
7M1/7S1 are two complementary subsets of a single 8N1 set.

HTH

[toc] | [prev] | [next] | [standalone]

#16019

Using 8N1 under minicom with this device resulted in garbled text when once connected.  Connection using 7M1 resulted in the correct text.  So there must be something else that needs to be done in my python program correct?

[toc] | [prev] | [next] | [standalone]

#16023

On Tue, Nov 22, 2011 at 3:28 AM, Matthew Lenz <matthew@nocturnal.org> wrote:
> Using 8N1 under minicom with this device resulted in garbled text when once connected.  Connection using 7M1 resulted in the correct text.  So there must be something else that needs to be done in my python program correct?

Using 8N1 when it's really 7M1 means you have the high bit set on
every byte. I don't know if there's an easy way to do this fast in
Python, but what you need to do is mask them all off... I'm not sure
if there's a better way, but this ought to work:

string = "".join(chr(ord(x)&0x7f) for x in string)

In Python 3, iterating over a 'bytes' string produces integers, so
omit the ord() call. Other than that, code is not tested.

ChrisA

[toc] | [prev] | [next] | [standalone]

#16024

On Nov 21, 2011, at 11:28 AM, Matthew Lenz wrote:

> Using 8N1 under minicom with this device resulted in garbled text when once connected.  Connection using 7M1 resulted in the correct text.  So there must be something else that needs to be done in my python program correct?

Under minicom in 8N1, it's going to look garbled because the high bit will always be set. Minicom will try to spit out those characters anyway, which will print out whatever extended ASCII garbage your terminal supports in the 0x80-0xFF range. Programmatically, though, you can strip off the high bit when you're receiving it in Python.

"Space" parity, on the other hand, should look normal under Minicom because the high bit will always be low, giving you standard 7-bit ASCII.

- Dave

[toc] | [prev] | [next] | [standalone]

#16025

Ahh. Ok.  So how would I go about doing that with python?  I think in perl (sorry for the naughty word) I could use the tr// (translate) but is there a quick way to do so with python?  Is it going to be necessary to convert commands I SEND to the device or only convert what I receive?

[toc] | [prev] | [next] | [standalone]

#16029

Ahh. Ok.  So how would I go about doing that with python?  I think in perl (sorry for the naughty word) I could use the tr// (translate) but is there a quick way to do so with python?  Is it going to be necessary to convert commands I SEND to the device or only convert what I receive?

[toc] | [prev] | [next] | [standalone]

#16031

On Nov 21, 2011, at 11:52 AM, Matthew Lenz wrote:

> Ahh. Ok.  So how would I go about doing that with python?  I think in perl (sorry for the naughty word) I could use the tr// (translate) but is there a quick way to do so with python?  Is it going to be necessary to convert commands I SEND to the device or only convert what I receive?

The high-level overview is that you'll want to OR in 0x80 on transmit, and AND 0x7F on receive (thus inserting the high bit when you send it out and removing it when you receive).  If you wanted to be extra-sure you're receiving things correctly, you should also check to see if your value ANDed with 0x80 is not zero.

In Python 2.x, as mentioned, when you iterate over a string, you get a bunch of tiny one-character strings, which you then need to convert into numbers with ord() and back into strings with chr() when you re-concatenate it.  ord() and chr() may be familiar to you from Perl.  For example, you could do this on reception:

----

# However you get your data out of serial
received_str = receive_my_string()

ord_str = [ord(x) for x in received_str]

# An exception may be extreme in this case, but hey.  Note that
# the filter() function actually returns a list of all the
# characters that don't have the high bit set, so you could
# actually use that if you wanted to.
if filter((lambda x: x < 0x80), ord_str):
   raise IOError("Received character without mark parity")

return "".join([chr(x & 0x7F) for x in received_str])

----

In Python 3.x, iterating over a bytes array (which is, hopefully, what your serial interface returns) will give you a bunch of ints, so you shouldn't need to do the conversions:


----

# However you get your data out of serial
received_bytes = receive_my_string()

# In Python 3.x, filter() returns an iterator, which is generally a
# better thing to return, but doesn't test as nicely for a bool.
for b in received_bytes:
   if b < 0x80:
       raise IOError("Received character without mark parity")

# None of this "".join() nonsense with byte arrays!
return bytes([(x & 0x7F) for x in received_bytes])


----


There are surely more efficient ways to do what I've typed up there, but those should work pretty well for whatever you're looking to do.  For sending, you pretty much want to swap (x & 0x7F) with (x | 0x80) to insert that high bit.


- Dave

[toc] | [prev] | [next] | [standalone]

#16035

Thanks, this will be a great help.

Just wanted to confirm that you meant to use [ .. for x in ord_str] in the example conversion?  Got a TypeError using the received_str.

[toc] | [prev] | [next] | [standalone]

#16036

Thanks, this will be a great help.

Just wanted to confirm that you meant to use [ .. for x in ord_str] in the example conversion?  Got a TypeError using the received_str.

[toc] | [prev] | [next] | [standalone]

#16038

On Nov 21, 2011, at 12:59 PM, Matthew Lenz wrote:

> Thanks, this will be a great help.
> 
> Just wanted to confirm that you meant to use [ .. for x in ord_str] in the example conversion?  Got a TypeError using the received_str.

Yes, I probably should have double-checked that.  ord_str is indeed what I meant. :-)


- Dave

[toc] | [prev] | [next] | [standalone]

#16039

On 21/11/2011 16:52, Matthew Lenz wrote:
> Ahh. Ok.  So how would I go about doing that with python?  I think in
> perl (sorry for the naughty word) I could use the tr// (translate)
> but is there a quick way to do so with python?  Is it going to be
> necessary to convert commands I SEND to the device or only convert
> what I receive?

Python strings have a .translate method:

# Example in Python 2
import string

# From top bit set...
from_chars = "".join(chr(c | 0x80) for c in range(0x7F))

# ...to top bit clear.
to_chars = "".join(chr(c) for c in range(0x7F))

# Build the translation table.
force_clear = string.maketrans(from_chars, to_chars)

s = "\x41\xC1"
print s
print s.translate(force_clear)

[toc] | [prev] | [next] | [standalone]

#16032

On Monday, November 21, 2011 11:58:53 AM David Riley did opine:

> On Nov 21, 2011, at 11:28 AM, Matthew Lenz wrote:
> > Using 8N1 under minicom with this device resulted in garbled text when
> > once connected.  Connection using 7M1 resulted in the correct text. 
> > So there must be something else that needs to be done in my python
> > program correct?
> 
> Under minicom in 8N1, it's going to look garbled because the high bit
> will always be set. Minicom will try to spit out those characters
> anyway, which will print out whatever extended ASCII garbage your
> terminal supports in the 0x80-0xFF range. Programmatically, though, you
> can strip off the high bit when you're receiving it in Python.

I have been using 8n1 in minicom for years, never ever had such a problem. 
In fact, I don't even know if I can set the path to mark parity as it is so 
rarely used.  E or O as error detectors are much more commonly used.

Example copy/paste from minicom, talking to a trs-80 Color Computer 3 
running a shell under nitros9, which is a bit like unix.  I am asking it 
for the settings of its own output path, .1=stdout:

{t2|07}/DD/NITROS9/dw3install/6309L2/SCRIPTS:tmode .1
/t2
 upc=00 bso=01 dlo=00 eko=01 alf=01 nul=00 pau=01 pag=18
 bsp=08 del=18 eor=0D eof=1B rpr=09 dup=01 psc=17 int=03
 qut=05 bse=08 ovf=07 par=01 bau=06 xon=00 xof=00
{t2|07}/DD/NITROS9/dw3install/6309L2/SCRIPTS:

And that is 9600 baud 8n1 on both ends.  Ascii is normally 7 bit and will 
have a low 8th bit if fed normal ascii data, so how is the 8th bit getting 
set other than purposely setting 7M1 on the other end of the cable?

> "Space" parity, on the other hand, should look normal under Minicom
> because the high bit will always be low, giving you standard 7-bit
> ASCII.
> 
Yes.

> - Dave


Cheers, Gene
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
My web page: <http://coyoteden.dyndns-free.com:85/gene>
Everything is controlled by a small evil group to which, unfortunately,
no one we know belongs.

[toc] | [prev] | [next] | [standalone]

#16033

On Nov 21, 2011, at 12:25 PM, gene heskett wrote:

> And that is 9600 baud 8n1 on both ends.  Ascii is normally 7 bit and will 
> have a low 8th bit if fed normal ascii data, so how is the 8th bit getting 
> set other than purposely setting 7M1 on the other end of the cable?

That's what I thought the OP was doing; it sounds like he's trying to receive 7M1 in Minicom using 8N1 on the terminal and getting garbled data because the high bit is set (because the other end is sending 7M1).  I never meant to imply that 8N1 would give garbled data if both ends were set to it; indeed, that's pretty much standard communications settings for short cables in low to moderate noise environments.  If anyone else read it that way, that's not what I meant. :-)

- Dave

[toc] | [prev] | [next] | [standalone]

#16021

Using 8N1 under minicom with this device resulted in garbled text when once connected.  Connection using 7M1 resulted in the correct text.  So there must be something else that needs to be done in my python program correct?

[toc] | [prev] | [next] | [standalone]

#16040

On Monday, November 21, 2011 01:28:16 PM David Riley did opine:

> On Nov 21, 2011, at 12:25 PM, gene heskett wrote:
> > And that is 9600 baud 8n1 on both ends.  Ascii is normally 7 bit and
> > will have a low 8th bit if fed normal ascii data, so how is the 8th
> > bit getting set other than purposely setting 7M1 on the other end of
> > the cable?
> 
> That's what I thought the OP was doing; it sounds like he's trying to
> receive 7M1 in Minicom using 8N1 on the terminal and getting garbled
> data because the high bit is set (because the other end is sending
> 7M1).  I never meant to imply that 8N1 would give garbled data if both
> ends were set to it; indeed, that's pretty much standard communications
> settings for short cables in low to moderate noise environments.  If
> anyone else read it that way, that's not what I meant. :-)
> 
> - Dave

I think that getting the other end off 7M1 was what I was saying.  Trying 
to attack the bad data after capture by writing code always seems extremely 
masochistic to me.

The amount of miss-understanding that seems to pervade rs-232 
communications is mind boggling at times.  The tech itself is so old it is 
being forgotten!

Cheers, Gene
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
My web page: <http://coyoteden.dyndns-free.com:85/gene>
Whatever occurs from love is always beyond good and evil.
		-- Friedrich Nietzsche

[toc] | [prev] | [next] | [standalone]

#16045

Another thing I noticed is that the & and | appear to give the same result as adding or subtracting 128 from the ordinal value.  I'm assuming that isn't coincidence. :)

[toc] | [prev] | [next] | [standalone]

#16047

On 2011-11-21, Matthew Lenz <matthew@nocturnal.org> wrote:

> Another thing I noticed is that the & and | appear to give the same
> result as adding or subtracting 128 from the ordinal value.

Nope, that's only true for some values.

If we're limiting ourselves to byte values, then we're talking
modulo-256 arithmetic, so:

128 + 128 = 0 
128 | 128 = 128

0 - 128  = 128
0 & 0x7f = 0


What's is true is that adding 128 is actullay the same as subtracting
128, and both are the same as exclusive-or 128 (v ^ 128):

>>> x = 128
>>> (x + 128) & 0xff
0
>>> (x - 128) & 0xff
0
>>> (x ^ 128) & 0xff
0

>>> x = 0
>>> (x + 128) & 0xff
128
>>> (x - 128) & 0xff
128
>>> (x ^ 128) & 0xff
128

> I'm assuming that isn't coincidence. :)

Well, the weighting of the high-order bit in an 8-bit wide binary
number is 128, if that's what you're getting at...

-- 
Grant Edwards               grant.b.edwards        Yow! How's it going in
                                  at               those MODULAR LOVE UNITS??
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#16046

Another thing I noticed is that the & and | appear to give the same result as adding or subtracting 128 from the ordinal value.  I'm assuming that isn't coincidence. :)

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web