Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #97868 > unrolled thread

A high-level cross-platform API for terminal/console access

Back to article view | Back to comp.lang.python

Contents

  A high-level cross-platform API for terminal/console access Peter Brittain <peter.brittain.os@gmail.com> - 2015-10-21 11:30 -0700
    Re: A high-level cross-platform API for terminal/console access Laura Creighton <lac@openend.se> - 2015-10-21 21:41 +0200
      Re: A high-level cross-platform API for terminal/console access Peter Brittain <peter.brittain.os@gmail.com> - 2015-10-21 15:18 -0700
        Re: A high-level cross-platform API for terminal/console access eryksun <eryksun@gmail.com> - 2015-10-21 17:26 -0500
          Re: A high-level cross-platform API for terminal/console access Peter Brittain <peter.brittain.os@gmail.com> - 2015-10-22 02:02 -0700
            Re: A high-level cross-platform API for terminal/console access Laura Creighton <lac@openend.se> - 2015-10-22 11:23 +0200
              Re: A high-level cross-platform API for terminal/console access Peter Brittain <peter.brittain.os@gmail.com> - 2015-10-22 03:56 -0700
                Re: A high-level cross-platform API for terminal/console access Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-10-22 12:10 +0100
    Re: A high-level cross-platform API for terminal/console access "Gisle Vanem" <gvanem@yahoo.no> - 2015-10-22 13:22 +0200
      Re: A high-level cross-platform API for terminal/console access Peter Brittain <peter.brittain.os@gmail.com> - 2015-10-22 05:04 -0700
    Re: A high-level cross-platform API for terminal/console access Chris Angelico <rosuav@gmail.com> - 2015-10-22 23:09 +1100
    Re: A high-level cross-platform API for terminal/console access "Gisle Vanem" <gvanem@yahoo.no> - 2015-10-22 14:42 +0200
      Re: A high-level cross-platform API for terminal/console access Peter Brittain <peter.brittain.os@gmail.com> - 2015-10-24 02:40 -0700
    Re: A high-level cross-platform API for terminal/console access Chris Angelico <rosuav@gmail.com> - 2015-10-22 23:48 +1100
    Re: A high-level cross-platform API for terminal/console access Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-10-22 13:56 +0100

#97868 — A high-level cross-platform API for terminal/console access

I have recently been working on a terminal/console animation package (https://github.com/peterbrittain/asciimatics). Beyond the high-level animation methods/objects it provides, it also needed to be cross-platform and and simple to install with pip (including any dependencies).

This cross-platform requirement meant I needed a curses equivalent for Windows.  This appears to have been a regular issue over the years and I've looked around at previous answers on the web and this newsgroup.  Now while there's some pretty neat stuff out there, nothing actually gave me everything I needed.

1) colorama only gives you colours and cursor positioning, but no input, resizing or screen-scraping options.

2) blessings/blessed provide better (but incomplete) APIs than the curses package, but no Windows support unless you use colorama (which is still limited as above).

3) console (from effbot.org) is a native installer and so fails the pip test.  It does however show direct use of the win32 API is a tenable approach.

4) The other packages I found were either dead projects or relied on a native installation of an open source implementation of curses - with no pip install option available.

I therefore started writing a consistent wrapper API (the Screen object) that uses the curses package on Unix variants and pywin32 to access the Windows console API.  

Over the last few months I've rounded out the features and proved that asciimatics works identically on Linux (various distros), OSX and Windows 7-10.   I know that there are still some rough edges due to the history of the project and so I've been deprecating old APIs in preparation for creating a completely clean API when I create the next major (v2.x) release.

I suspect that it could be even better though, so am looking for feedback and ideas from the community - both on the high-level animation features and the low-level cross-platform terminal API.  

Is there something you can see that could be usefully improved?  All feedback welcome.

[toc] | [next] | [standalone]

#97870

In a message of Wed, 21 Oct 2015 11:30:25 -0700, Peter Brittain writes:
>I have recently been working on a terminal/console animation package (https://github.com/peterbrittain/asciimatics). Beyond the high-level animation methods/objects it provides, it also needed to be cross-platform and and simple to install with pip (including any dependencies).
>
>This cross-platform requirement meant I needed a curses equivalent for Windows.  This appears to have been a regular issue over the years and I've looked around at previous answers on the web and this newsgroup.  Now while there's some pretty neat stuff out there, nothing actually gave me everything I needed.
>
>1) colorama only gives you colours and cursor positioning, but no input, resizing or screen-scraping options.
>
>2) blessings/blessed provide better (but incomplete) APIs than the curses package, but no Windows support unless you use colorama (which is still limited as above).
>
>3) console (from effbot.org) is a native installer and so fails the pip test.  It does however show direct use of the win32 API is a tenable approach.
>
>4) The other packages I found were either dead projects or relied on a native installation of an open source implementation of curses - with no pip install option available.
>
>I therefore started writing a consistent wrapper API (the Screen object) that uses the curses package on Unix variants and pywin32 to access the Windows console API.  
>
>Over the last few months I've rounded out the features and proved that asciimatics works identically on Linux (various distros), OSX and Windows 7-10.   I know that there are still some rough edges due to the history of the project and so I've been deprecating old APIs in preparation for creating a completely clean API when I create the next major (v2.x) release.
>
>I suspect that it could be even better though, so am looking for feedback and ideas from the community - both on the high-level animation features and the low-level cross-platform terminal API.  
>
>Is there something you can see that could be usefully improved?  All feedback welcome.
>-- 

Did you try https://pypi.python.org/pypi/UniCurses ?

Laura

[toc] | [prev] | [next] | [standalone]

#97873

> 
> Did you try https://pypi.python.org/pypi/UniCurses ?
> 

Yes - it failed to install with pip and also looked like a dead project when I followed the project home page URL.

[toc] | [prev] | [next] | [standalone]

#97874

On 10/21/15, Peter Brittain <peter.brittain.os@gmail.com> wrote:
>>
>> Did you try https://pypi.python.org/pypi/UniCurses ?
>>
>
> Yes - it failed to install with pip and also looked like a dead project when
> I followed the project home page URL.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Also check out the curses module that's available on Christoph Gohlke's site:

    http://www.lfd.uci.edu/~gohlke/pythonlibs/#curses

[toc] | [prev] | [next] | [standalone]

#97885

On Wednesday, October 21, 2015 at 11:26:40 PM UTC+1, eryksun wrote:
> 
> Also check out the curses module that's available on Christoph Gohlke's site:
> 
>     http://www.lfd.uci.edu/~gohlke/pythonlibs/#curses

Neat.  I wasn't aware of this library of wheel installations.  I'll have a look at how that works out and see if I can rationalize my mapping code.  

First impression is that this might be tricky, though, as I've had issues with older Linux distributions using ncurses 5 and so handling 256 colour modes has been difficult due to limits on colour pairs.  This meant I had to fall back to looking up and using codes in the terminfo database using tigetstr.  According to the PDcurses docs, these APIs are all just stubs that return an error and so I'll probably need a curses and PDcurses mapping layer from the looks of things - which is not much better than a curses and win32 mapping layer.

I'm also still not convinced that curses package is the right API to expose for Python.  While ncurses does a great job of abstracting away the issues of terminal inter-operation, the Python curses package is just a thin wrapper of that C library and, as can be seen above, not truly cross-platform due to the restrictions of PDcurses.

Shouldn't we have a higher level simplification?  Something that hides away all the complexity of handling all these different platforms and so exposes a simple API?  One that humans can use without worrying about these issues?

[toc] | [prev] | [next] | [standalone]

#97887

In a message of Thu, 22 Oct 2015 02:02:28 -0700, Peter Brittain writes:
>On Wednesday, October 21, 2015 at 11:26:40 PM UTC+1, eryksun wrote:
>> 
>> Also check out the curses module that's available on Christoph Gohlke's site:
>> 
>>     http://www.lfd.uci.edu/~gohlke/pythonlibs/#curses
>
>Neat.  I wasn't aware of this library of wheel installations.  I'll have a look at how that works out and see if I can rationalize my mapping code.  
>
>First impression is that this might be tricky, though, as I've had issues with older Linux distributions using ncurses 5 and so handling 256 colour modes has been difficult due to limits on colour pairs.  This meant I had to fall back to looking up and using codes in the terminfo database using tigetstr.  According to the PDcurses docs, these APIs are all just stubs that return an error and so I'll probably need a curses and PDcurses mapping layer from the looks of things - which is not much better than a curses and win32 mapping layer.
>
>I'm also still not convinced that curses package is the right API to expose for Python.  While ncurses does a great job of abstracting away the issues of terminal inter-operation, the Python curses package is just a thin wrapper of that C library and, as can be seen above, not truly cross-platform due to the restrictions of PDcurses.
>
>Shouldn't we have a higher level simplification?  Something that hides away all the complexity of handling all these different platforms and so exposes a simple API?  One that humans can use without worrying about these issues?
>-- 
>https://mail.python.org/mailman/listinfo/python-list

Fredrik Lundh's console implementation
http://effbot.org/zone/console-handbook.htm
might be of interest in that case, but I think it is 'old versions
of windows only'.  But it's a different take on the abstraction 
problem.  I haven't used it for something like 15 years now, though,
so can barely remember it ...

Laura

[toc] | [prev] | [next] | [standalone]

#97891

On Thursday, October 22, 2015 at 10:24:11 AM UTC+1, Laura Creighton wrote:
> Fredrik Lundh's console implementation
> http://effbot.org/zone/console-handbook.htm
> might be of interest in that case, but I think it is 'old versions
> of windows only'.  But it's a different take on the abstraction 
> problem.  I haven't used it for something like 15 years now, though,
> so can barely remember it ...

Thanks, but I think this is already covered in my original post: I couldn't use it directly due to the pip installation restriction.  I've now created a working Windows implementation, so I don't need another mapping for win32.

More generally, as these posts begin to show, there was no _simple_ way for me to get a cross-platform console API that "just worked" out of the box.  I've had to jump through various hoops to get to where I am and don't think that other people should have to go through the same pain as I have.  

This is one of the reasons why I've been tidying up my package to make this as simple as possible for the next person.  I've now got to the stage where I have something that works for me, but it is almost certainly missing something.  Maybe it's not good enough documentation, maybe there's some clunkiness left in the API due to the history of the project, maybe there's an even better way to represent terminals than what I've come up with so far?

I was hoping for feedback on this front rather than other ways I could recreate the curses package - unless of course, the general consensus is that this really is the way that Python should expose access to the terminal/console.

[toc] | [prev] | [next] | [standalone]

#97892

On 22 October 2015 at 11:56, Peter Brittain <peter.brittain.os@gmail.com> wrote:
> On Thursday, October 22, 2015 at 10:24:11 AM UTC+1, Laura Creighton wrote:
>> Fredrik Lundh's console implementation
>> http://effbot.org/zone/console-handbook.htm
>> might be of interest in that case, but I think it is 'old versions
>> of windows only'.  But it's a different take on the abstraction
>> problem.  I haven't used it for something like 15 years now, though,
>> so can barely remember it ...
>
> Thanks, but I think this is already covered in my original post: I couldn't use it directly due to the pip installation restriction.  I've now created a working Windows implementation, so I don't need another mapping for win32.
>
> More generally, as these posts begin to show, there was no _simple_ way for me to get a cross-platform console API that "just worked" out of the box.  I've had to jump through various hoops to get to where I am and don't think that other people should have to go through the same pain as I have.

I've looked for this in the past and I also found that there was no
general solution that I could just pick and use for both Windows and
everything else. Even just writing a cross-platform getch function is
needlessly complicated.

> This is one of the reasons why I've been tidying up my package to make this as simple as possible for the next person.  I've now got to the stage where I have something that works for me, but it is almost certainly missing something.  Maybe it's not good enough documentation, maybe there's some clunkiness left in the API due to the history of the project, maybe there's an even better way to represent terminals than what I've come up with so far?
>
> I was hoping for feedback on this front rather than other ways I could recreate the curses package - unless of course, the general consensus is that this really is the way that Python should expose access to the terminal/console.

Thanks for creating this. I will try it out when I next want to create
something like this and give some feedback then. I don't really know
when that will be though...

--
Oscar

[toc] | [prev] | [next] | [standalone]

#97895

"Peter Brittain" <peter.brittain.os@gmail.com> wrote:

>I have recently been working on a terminal/console animation package
> (https://github.com/peterbrittain/asciimatics). 

I tried installing your package with "pip.exe -v install asciimatics".
Some problem with pypiwin32 it seems:

Installing collected packages: pypiwin32, future, Pillow, pyfiglet, asciimatics

  Cleaning up...
  Exception:
  Traceback (most recent call last):
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\basecommand.py", line 232, in main
      status = self.run(options, args)
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\commands\install.py", line 347, in run
      root=options.root_path,
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\req\req_set.py", line 549, in install
      **kwargs
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\req\req_install.py", line 751, in install
      self.move_wheel_files(self.source_dir, root=root)
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\req\req_install.py", line 960, in move_wheel_files
      isolated=self.isolated,
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\wheel.py", line 234, in move_wheel_files
      clobber(source, lib_dir, True)
    File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\wheel.py", line 212, in clobber
      shutil.copyfile(srcfile, destfile)
    File "g:\Programfiler\Python27\lib\shutil.py", line 83, in copyfile
      with open(dst, 'wb') as fdst:
  IOError: [Errno 13] Permission denied: 'g:\\Programfiler\\Python27\\Lib\\site-packages\\win32\\win32api.pyd'

-----

BTW, this is on Python 2.7.9 on Win-XP SP3.

--gv

[toc] | [prev] | [next] | [standalone]

#97897

On Thursday, October 22, 2015 at 12:25:09 PM UTC+1, Gisle Vanem wrote:
> I tried installing your package with "pip.exe -v install asciimatics".
> Some problem with pypiwin32 it seems:
> 
> Installing collected packages: pypiwin32, future, Pillow, pyfiglet, asciimatics
> 
>   Cleaning up...
>   Exception:
>   Traceback (most recent call last):
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\basecommand.py", line 232, in main
>       status = self.run(options, args)
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\commands\install.py", line 347, in run
>       root=options.root_path,
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\req\req_set.py", line 549, in install
>       **kwargs
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\req\req_install.py", line 751, in install
>       self.move_wheel_files(self.source_dir, root=root)
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\req\req_install.py", line 960, in move_wheel_files
>       isolated=self.isolated,
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\wheel.py", line 234, in move_wheel_files
>       clobber(source, lib_dir, True)
>     File "g:\Programfiler\Python27\lib\site-packages\pip-6.0.6-py2.7.egg\pip\wheel.py", line 212, in clobber
>       shutil.copyfile(srcfile, destfile)
>     File "g:\Programfiler\Python27\lib\shutil.py", line 83, in copyfile
>       with open(dst, 'wb') as fdst:
>   IOError: [Errno 13] Permission denied: 'g:\\Programfiler\\Python27\\Lib\\site-packages\\win32\\win32api.pyd'
> 
> -----
> 
> BTW, this is on Python 2.7.9 on Win-XP SP3.
> 
> --gv

That's a new one on me.  The pypiwin32 package has installed fine for me on Python 3 in  Windows 7, 8 and 10.  Sadly I don't have ready access to an XP machine, so can't easily try a repro myself...  Does the file already exist, or are you lacking permissions to update the folder?

[toc] | [prev] | [next] | [standalone]

#97899

On Thu, Oct 22, 2015 at 10:22 PM, Gisle Vanem <gvanem@yahoo.no> wrote:
>  IOError: [Errno 13] Permission denied:
> 'g:\\Programfiler\\Python27\\Lib\\site-packages\\win32\\win32api.pyd'
>
> -----
>
> BTW, this is on Python 2.7.9 on Win-XP SP3.

Does that file exist? A .pyd file is a DLL, so if it already exists
and is in use, it's entirely possible it can't be overwritten (eg to
upgrade it).

ChrisA

[toc] | [prev] | [next] | [standalone]

#97900

"Chris Angelico" <rosuav@gmail.com> wrote:

> On Thu, Oct 22, 2015 at 10:22 PM, Gisle Vanem <gvanem@yahoo.no> wrote:
>>  IOError: [Errno 13] Permission denied:
>> 'g:\\Programfiler\\Python27\\Lib\\site-packages\\win32\\win32api.pyd'
>>
>> -----
>>
>> BTW, this is on Python 2.7.9 on Win-XP SP3.
> 
> Does that file exist? A .pyd file is a DLL, so if it already exists
> and is in use, it's entirely possible it can't be overwritten (eg to
> upgrade it).

It exists all right and I do have full access to that folder.

I think the Errno 13 (=EACCESS) is due to another module (WConio.pyd) 
that seems gets loaded via my site-customize.py is using this
win32api.pyd. Hence it's is in use and shutil fails in updating it. 

--gv

[toc] | [prev] | [next] | [standalone]

#97929

For some reason your last post didn't come through to my newsgroup feed.  I've tried to reply to this one instead...

> But the samples have problems if launched from the sample directory!
> It seems to be related to my Windows associations for .png files; 
Not sure quite sure how your file associations could have caused this.  Maybe the reference to python has picked up the PNG file instead of the executable and then invoked the association for that for the rest of the arguments?

> 'python2 samples\win_demo.py' works fine (see attached screen-shot).
> Allthough it messes up the screen's scrollback buffer. Similar to how
> most PDcurses do on Windows :-(
I'm not surprised PDcurses does this too.  To recover, I need to save off the entire state of the screen and reset afterwards.  I'll add an issue to cover this and see what can be done.  Main issue will probably be if you resize while running asciimatics.

BTW, you can run all the other samples in Windows too, apart from not_curses.py.  The win_demo dates back to when there was no unified wrapper to create the Screen.

[toc] | [prev] | [next] | [standalone]

#97901

On Thu, Oct 22, 2015 at 11:42 PM, Gisle Vanem <gvanem@yahoo.no> wrote:
> I think the Errno 13 (=EACCESS) is due to another module (WConio.pyd) that
> seems gets loaded via my site-customize.py is using this
> win32api.pyd. Hence it's is in use and shutil fails in updating it.

Ah, that might well be it. Does it work if you run:

python -S -m pip install --upgrade win32api

? That might stop win32api.pyd from getting loaded.

ChrisA

[toc] | [prev] | [next] | [standalone]

#97902

On 22 October 2015 at 13:42, Gisle Vanem <gvanem@yahoo.no> wrote:
> "Chris Angelico" <rosuav@gmail.com> wrote:
>
>> On Thu, Oct 22, 2015 at 10:22 PM, Gisle Vanem <gvanem@yahoo.no> wrote:
>>>
>>>  IOError: [Errno 13] Permission denied:
>>> 'g:\\Programfiler\\Python27\\Lib\\site-packages\\win32\\win32api.pyd'
>>>
>>> -----
>>>
>>> BTW, this is on Python 2.7.9 on Win-XP SP3.
>>
>>
>> Does that file exist? A .pyd file is a DLL, so if it already exists
>> and is in use, it's entirely possible it can't be overwritten (eg to
>> upgrade it).
>
>
> It exists all right and I do have full access to that folder.
>
> I think the Errno 13 (=EACCESS) is due to another module (WConio.pyd) that
> seems gets loaded via my site-customize.py is using this
> win32api.pyd. Hence it's is in use and shutil fails in updating it.

Does it work if you ask pip to install each dependency separately?
i.e. you could just run pip install pypiwin32.

--
Oscar

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web