Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #78171 > unrolled thread
| Started by | Irmen de Jong <irmen.NOSPAM@xs4all.nl> |
|---|---|
| First post | 2014-09-22 19:47 +0200 |
| Last post | 2014-09-24 00:16 +0200 |
| Articles | 5 — 2 participants |
Back to article view | Back to comp.lang.python
Pyrolite, lightweight pickle and pyro client library, seeking a bit of testing help Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2014-09-22 19:47 +0200
Re: Pyrolite, lightweight pickle and pyro client library, seeking a bit of testing help Chris Angelico <rosuav@gmail.com> - 2014-09-23 03:53 +1000
Re: Pyrolite, lightweight pickle and pyro client library, seeking a bit of testing help Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2014-09-22 20:23 +0200
Re: Pyrolite, lightweight pickle and pyro client library, seeking a bit of testing help Chris Angelico <rosuav@gmail.com> - 2014-09-23 04:28 +1000
Re: Pyrolite, lightweight pickle and pyro client library, seeking a bit of testing help Irmen de Jong <irmen.NOSPAM@xs4all.nl> - 2014-09-24 00:16 +0200
| From | Irmen de Jong <irmen.NOSPAM@xs4all.nl> |
|---|---|
| Date | 2014-09-22 19:47 +0200 |
| Subject | Pyrolite, lightweight pickle and pyro client library, seeking a bit of testing help |
| Message-ID | <542060bf$0$2929$e4fe514c@news.xs4all.nl> |
Hi, I've developed Pyrolite (https://github.com/irmen/Pyrolite), a lightweight client library for Java and .NET to gain access to Python servers running Pyro. As such it also contains a complete pickle and unpickle implementation in these languages. Quite recently I got a pull request to fix a big/little-endianness decoding issue regarding machine floats and doubles in pickled arrays. It was a mistake on my part but it got me thinking: the unit tests of my library should ideally be run on both little- and big-endian machines, because some of the code depends on this. I've only got access to little endian systems myself. So basically I'm looking for some tips here, for instance can Travis-ci.org perhaps help out? Or can someone here with access to a big endian system help me out with running the Java and .NET (Microsoft or Mono) unit test suite of my project on that system? (just a one time request, to verify the current version). Irmen de Jong PS the reason why I'm posting a Java or .NET related issue here is because the core protocol my library handles is a Python invented one ;)
[toc] | [next] | [standalone]
| From | Chris Angelico <rosuav@gmail.com> |
|---|---|
| Date | 2014-09-23 03:53 +1000 |
| Message-ID | <mailman.14225.1411408441.18130.python-list@python.org> |
| In reply to | #78171 |
On Tue, Sep 23, 2014 at 3:47 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote: > I've developed Pyrolite (https://github.com/irmen/Pyrolite), a lightweight client > library for Java and .NET to gain access to Python servers running Pyro. As such it also > contains a complete pickle and unpickle implementation in these languages. Does this imply that you send pickle data across the internet and unpickle it? Isn't that rather dangerous? ChrisA
[toc] | [prev] | [next] | [standalone]
| From | Irmen de Jong <irmen.NOSPAM@xs4all.nl> |
|---|---|
| Date | 2014-09-22 20:23 +0200 |
| Message-ID | <5420690b$0$2833$e4fe514c@news.xs4all.nl> |
| In reply to | #78172 |
On 22-9-2014 19:53, Chris Angelico wrote: > On Tue, Sep 23, 2014 at 3:47 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote: >> I've developed Pyrolite (https://github.com/irmen/Pyrolite), a lightweight client >> library for Java and .NET to gain access to Python servers running Pyro. As such it also >> contains a complete pickle and unpickle implementation in these languages. > > Does this imply that you send pickle data across the internet and > unpickle it? Isn't that rather dangerous? > > ChrisA Yes it is, good to point this out. This is why Pyro has been using a different (and safe) serializer by default for a while now. You have to plow through the usual security warnings in the docs and make a conscious effort in your code to enable the pickle serializer if you really want/need it. Pyrolite also contains a Java and .NET version of that safe serializer so you should not be using pickle at all when dealing with Pyro, but its implementation is there. And the pickle code can be used independently. Irmen
[toc] | [prev] | [next] | [standalone]
| From | Chris Angelico <rosuav@gmail.com> |
|---|---|
| Date | 2014-09-23 04:28 +1000 |
| Message-ID | <mailman.14229.1411410527.18130.python-list@python.org> |
| In reply to | #78174 |
On Tue, Sep 23, 2014 at 4:23 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote: > This is why Pyro has been using a different (and safe) serializer by default for a while > now. You have to plow through the usual security warnings in the docs and make a > conscious effort in your code to enable the pickle serializer if you really want/need it. Is the safe serializer affected by byte order? If not, you could just mark this off as a known bug, and say "if anyone has a big-endian system to test this on, please help out". It would be another incentive to use the safe serializer rather than pickle. And it'd save you a lot of trouble. :) I would have offered one of my systems for the test, except ... they're all little-endian. I'm all PC-based hardware here (mainly Linux running on Intel CPUs, though there are some variations). ChrisA
[toc] | [prev] | [next] | [standalone]
| From | Irmen de Jong <irmen.NOSPAM@xs4all.nl> |
|---|---|
| Date | 2014-09-24 00:16 +0200 |
| Message-ID | <5421f152$0$2932$e4fe514c@news2.news.xs4all.nl> |
| In reply to | #78177 |
On 22-9-2014 20:28, Chris Angelico wrote: > On Tue, Sep 23, 2014 at 4:23 AM, Irmen de Jong <irmen.NOSPAM@xs4all.nl> wrote: >> This is why Pyro has been using a different (and safe) serializer by default for a while >> now. You have to plow through the usual security warnings in the docs and make a >> conscious effort in your code to enable the pickle serializer if you really want/need it. > > Is the safe serializer affected by byte order? If not, you could just > mark this off as a known bug, and say "if anyone has a big-endian > system to test this on, please help out". It would be another > incentive to use the safe serializer rather than pickle. And it'd save > you a lot of trouble. :) The safe serializer is this one: https://github.com/irmen/serpent and it is a text based protocol, so no, it is not affected by byte order. It uses Python literal expressions as understood by ast.literal_eval(). Note that the endianness bug in my pickle implementation was fixed by the pull request so it is no longer a known bug. But I still would like to be able to run the test suite on both endianness platforms (or let someone else try it for me :-) ) > > I would have offered one of my systems for the test, except ... > they're all little-endian. I'm all PC-based hardware here (mainly > Linux running on Intel CPUs, though there are some variations). Heh, yes; I used to have a PowerPC G4 mac mini (a big endian platform), but that machine got replaced by an Intel based one (little endian, as all the other computers I have access to). Ah well. Irmen
[toc] | [prev] | [standalone]
Back to top | Article view | comp.lang.python
csiph-web