Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #78628 > unrolled thread

Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Ned Deily <nad@acm.org> - 2014-10-02 15:08 -0700
    Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Christian Gollwitzer <auriocus@gmx.de> - 2014-10-03 08:57 +0200
      Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Ned Deily <nad@acm.org> - 2014-10-03 12:55 -0700
        Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Kevin Walzer <kw@codebykevin.com> - 2014-10-03 17:16 -0400
          Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Ned Deily <nad@acm.org> - 2014-10-03 22:53 -0700
            Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Christian Gollwitzer <auriocus@gmx.de> - 2014-10-05 21:26 +0200
              Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk Ned Deily <nad@acm.org> - 2014-10-05 13:33 -0700

#78628 — Re: Python 3.4.1 installer on Mac links Python to old Tcl/Tk

In article <m0jciu$hoi$1@ger.gmane.org>,
 "Peter Tomcsanyi" <tomcsanyi@slovanet.sk> wrote:
> "Ned Deily" <nad@acm.org> wrote in message 
> news:nad-40CB03.10344701102014@news.gmane.org...
> > The python.org 3.4.x series of installers will likely never change from
> > linking with Tk 8.5 by default.  That would break a number of important
> > third-party Python packages that supply binary distributions built for
> > the python.org OS X pythons, like matplotlib.
> I respect your decision.
> But it surprises me because the Windows version of Python 3.4.1 already 
> comes with Tcl/Tk 8.6. Does it mean that there are no such problems with 
> matplotlib etc. in Windows?

The main issue is that, unlike MS and Windows, Apple has long shipped 
versions of Tcl and Tk with OS X.  As I understand it, Apple even helped 
sponsor the writing of the newer Cocoa-based Tk that would also work on 
64-bit systems after they decided to not extend the older Carbon APIs to 
64-bit, making the original OS X native Carbon-based Tk problematic on 
newer Macs and releases of OS X.  When the Cocoa Tk was first shipped 
with OS X 10.6, Tk 8.6 hadn't been released yet so they went with a 
backport to 8.5 and, for whatever reasons, Apple has been very slow to 
upgrade their 8.5.x Tk since then and have not shipped a 8.6 version 
yet.  That leaves us in a bit of a quandary for the python.org 
installers.  Ideally, we'd just like to continue to depend on an 
Apple-supplied version but even the most recent releases of OS X ship 
with serious bugs.  Fortunately, it's been possible and long been our 
practice to ship the python.org installers Pythons built in such a way 
that, at run time, they will automatically use a compatible third-party 
Tcl and Tk, if present, otherwise fall back to the system version.  By 
far, the leading third-party distributor of Tcl/Tk is ActiveState and 
they are actively involved in the development and maintenance of Tcl and 
Tk.  But, their releases are not open-source and have a license that, 
while generous, do not permit totally unrestricted usage.  Thus it is 
problematic to depend totally on their releases. So, to really support 
Tk 8.6, the only viable option at the moment would be for us to ship our 
own versions of Tk, like the Windows installer does.  But it wouldn't be 
acceptable, IMO, to force other projects and users to migrate to 8.6 in 
the middle of a release cycle, e.g. 3.4.x or 2.7.x.  Providing it as an 
option, though, would be OK.

> I think that one of the point when advertising Python is that it works 
> accross the platforms, so keeping one platform (mac) "behind" for months 
> seems to me going against the multiplatform promise of Python...

I don't think we mean to imply that all of the third-party libraries 
that Python is able to use are the same across all platforms.  It 
depends on the platform and the practices there.  The fact is that Tk 
8.5 is still the de facto standard for OS X 10.6+ and Tk 8.4 is the de 
facto standard for OS X 10.5 and earlier systems, because that's what 
Apple has shipped.  And that's why matplotlib et al and we still ship 
binary installers for OS X with 8.5 support.  Third-party open source 
packages distributors, like MacPorts or Anaconda, can more easily move 
to newer versions of things, like Tk 8.6, because they provide complete 
solutions for many applications and libraries: they have ports for 
matplotlib, numpy, pythonx.x, Tk, and all other dependencies.  The 
python.org installers have a much more limited scope.

-- 
 Ned Deily,
 nad@acm.org

[toc] | [next] | [standalone]

#78640

Am 03.10.14 00:08, schrieb Ned Deily:
> So, to really support
> Tk 8.6, the only viable option at the moment would be for us to ship our
> own versions of Tk, like the Windows installer does.  But it wouldn't be
> acceptable, IMO, to force other projects and users to migrate to 8.6 in
> the middle of a release cycle, e.g. 3.4.x or 2.7.x.  Providing it as an
> option, though, would be OK.

Hmm, I'm not sure that the other projects would need an update. In Tcl 
world, there is the concept of stub libraries, an extra level of 
indirect linking provided by both the Tcl and Tk core. If the extensions 
link to the stub library (the standard way), they can be loaded into any 
later version of Tcl and do not directly depend on libtcl or libtk. If 
the extensions link directly to libtcl, they are either broken or very 
special (providing one's own interpreter, doing nasty stuff with 
internals etc.)

	Christian

[toc] | [prev] | [next] | [standalone]

#78664

In article <m0lhbq$np4$1@dont-email.me>,
 Christian Gollwitzer <auriocus@gmx.de> wrote:
> Hmm, I'm not sure that the other projects would need an update. In Tcl 
> world, there is the concept of stub libraries, an extra level of 
> indirect linking provided by both the Tcl and Tk core. If the extensions 
> link to the stub library (the standard way), they can be loaded into any 
> later version of Tcl and do not directly depend on libtcl or libtk. If 
> the extensions link directly to libtcl, they are either broken or very 
> special (providing one's own interpreter, doing nasty stuff with 
> internals etc.)

Even if there were no incompatibilities, on OS X with Tcl and Tk (and 
other) frameworks, the version number is embedded in the path to the 
shared library and the linker normally creates an absolute path at that.

$ otool -L _tkagg.so
_tkagg.so:
   /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current 
version 60.0.0)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 1197.1.1)
   /Library/Frameworks/Tcl.framework/Versions/8.5/Tcl (compatibility 
version 8.5.0, current version 8.5.15)
   /Library/Frameworks/Tk.framework/Versions/8.5/Tk (compatibility 
version 8.5.0, current version 8.5.15)

-- 
 Ned Deily,
 nad@acm.org

[toc] | [prev] | [next] | [standalone]

#78666

On 10/3/14, 3:55 PM, Ned Deily wrote:
> Even if there were no incompatibilities, on OS X with Tcl and Tk (and
> other) frameworks, the version number is embedded in the path to the
> shared library and the linker normally creates an absolute path at that.

Is this because Python lacks the concept of stubs?

A Tcl library compiled for 8.5 can be loaded into 8.6 with no 
re-compiling required because of stubs.
-- 
Kevin Walzer
Code by Kevin/Mobile Code by Kevin
http://www.codebykevin.com
http://www.wtmobilesoftware.com

[toc] | [prev] | [next] | [standalone]

#78680

In article <m0n3n3$48j$1@dont-email.me>,
 Kevin Walzer <kw@codebykevin.com> wrote:
> On 10/3/14, 3:55 PM, Ned Deily wrote:
> > Even if there were no incompatibilities, on OS X with Tcl and Tk (and
> > other) frameworks, the version number is embedded in the path to the
> > shared library and the linker normally creates an absolute path at that.
> 
> Is this because Python lacks the concept of stubs?
> 
> A Tcl library compiled for 8.5 can be loaded into 8.6 with no 
> re-compiling required because of stubs.

It has nothing to do with Python per se; that's just the way linking 
with OS frameworks work.  Both Apple and ActiveState install their 
versions of Tcl and Tk as frameworks.  What gets installed and what the 
OS X compilers and linkers expect is something like this:

$ cd /Library/Frameworks/Tk.framework/Versions/8.5
$ ls -l
total 5720
drwxr-xr-x+ 3 root  wheel      272 Nov 24  2013 Headers
drwxr-xr-x+ 2 root  wheel      340 Nov 24  2013 PrivateHeaders
drwxr-xr-x+ 4 root  wheel      272 Nov 24  2013 Resources
-rw-r--r--+ 1 root  wheel  2905000 Oct 27  2013 Tk
-rw-r--r--+ 1 root  wheel    12240 Oct 27  2013 libtkstub8.5.a
-rw-r--r--+ 1 root  wheel     4622 Oct 27  2013 tkConfig.sh
$ file Tk
Tk: Mach-O universal binary with 2 architectures
Tk (for architecture x86_64): Mach-O 64-bit dynamically linked shared 
library x86_64
Tk (for architecture i386):   Mach-O dynamically linked shared library 
i386
$ file libtkstub8.5.a
libtkstub8.5.a: Mach-O universal binary with 2 architectures
libtkstub8.5.a (for architecture x86_64): current ar archive random 
library
libtkstub8.5.a (for architecture i386):   current ar archive random 
library
$ otool -D Tk
Tk:
/Library/Frameworks/Tk.framework/Versions/8.5/Tk

When Python or other programs link with a framework, they use the cc or 
ld -framework option, rather than -l:
    -framework Tcl -framework Tk

That causes the linker to look in the default search paths for 
frameworks, /Library/Frameworks followed by /System/Library/Frameworks.  
The install names of the framework shared libraries used are embedded in 
the Mach-O file (executable, bundle, or shared library) produced by ld.  
So the stub library archive is there but is not used, AFAIK.

There may be other ways to do it but that's how Python has always linked 
to Tcl and Tk.  FWIW, that's how both Apple's and ActiveState's wish 
executables are linked as well:

$ more /usr/local/bin/wish8.5
#!/bin/sh
"$(dirname 
$0)/../../../Library/Frameworks/Tk.framework/Versions/8.5/Resources/Wish.
app/Contents/MacOS/Wish" "$@"
$ otool -L 
/Library/Frameworks/Tk.framework/Versions/8.5/Resources/Wish.app/Contents
/MacOS/Wish
/Library/Frameworks/Tk.framework/Versions/8.5/Resources/Wish.app/Contents
/MacOS/Wish:
   /Library/Frameworks/Tk.framework/Versions/8.5/Tk (compatibility 
version 8.5.0, current version 8.5.15)
   /Library/Frameworks/Tcl.framework/Versions/8.5/Tcl (compatibility 
version 8.5.0, current version 8.5.15)
   /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 125.2.0)
   /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa 
(compatibility version 1.0.0, current version 15.0.0)
   /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon 
(compatibility version 2.0.0, current version 152.0.0)
   /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit 
(compatibility version 1.0.0, current version 275.0.0)
   /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current 
version 625.0.0)

There's more info about frameworks in various Apple developer docs and 
in the man pages for ld, otool, and install_name_tool.

-- 
 Ned Deily,
 nad@acm.org

[toc] | [prev] | [next] | [standalone]

#78721

Am 04.10.14 07:53, schrieb Ned Deily:
> In article <m0n3n3$48j$1@dont-email.me>,
>   Kevin Walzer <kw@codebykevin.com> wrote:
>> A Tcl library compiled for 8.5 can be loaded into 8.6 with no
>> re-compiling required because of stubs.
>
> It has nothing to do with Python per se; that's just the way linking
> with OS frameworks work.

 > [...]
>
> When Python or other programs link with a framework, they use the cc or
> ld -framework option, rather than -l:
>      -framework Tcl -framework Tk
>
> That causes the linker to look in the default search paths for
> frameworks, /Library/Frameworks followed by /System/Library/Frameworks.
> The install names of the framework shared libraries used are embedded in
> the Mach-O file (executable, bundle, or shared library) produced by ld.
> So the stub library archive is there but is not used, AFAIK.

Then it is done in a non-optimal way, though admittedly in the case of 
Tkinter, which is a hybrid Python/Tcl extension, the optimal way is not 
obvious. In the Tcl world, there are two different ways to link to Tcl 
(and analogously to Tk):

  A) you can link directly to libtcl.dylib (.so, .dll)

  B) you can define USE_TCL_STUBS during compilation and link to 
libtclstub.a

If you embed an interpreter, you must do A. So for Tkinter, linking 
directly to the dynamic lib (== -framework in OSX) is correct.

If you extend an interpreter by defining new commands, new Tcl_Obj, new 
Tk image types etc., you can do either A or B, but the recommended way 
is always B because this makes it possible to load an extension into any 
later interpreter than the one it was compiled against. The magic behind 
stubs are macros which replace function calls by invocations of an index 
into a function pointer table; that table is initialized at runtime, 
when the extension calls Tcl_InitStubs().

Now, tkagg is not loaded by the Tcl load() command. This means that 
after loading via Pythons import, the stub table should be initialized. 
Obviously this is not the case; if I'm not mistaken, just before this line

https://github.com/matplotlib/matplotlib/blob/master/src/_tkagg.cpp#L252

if you insert
	Tcl_InitStubs(interp, TCL_VERSION, 0);
	Tk_InitStubs(interp, TCL_VERSION, 0);

then it should be possible to compile tkagg with -DUSE_TCL_STUBS 
-DUSE_TK_STUBS, link to tclstub and tkstub and load the resulting tkagg 
into any Tkinter that links to an interpreter at least as new as 
TCL_VERSION. (If the result of either call is NULL, the interpreter is 
too old). Admittedly I'm too lazy to setup a build environment for 
matplotlib to try if this actually works.

>
> There may be other ways to do it but that's how Python has always linked
> to Tcl and Tk.  FWIW, that's how both Apple's and ActiveState's wish
> executables are linked as well:

wish is a type A program, it creates an interpreter and therefore must 
link to the actual library. So is Tkinter. But Tkagg is not, it extends 
a preexisting interpreter.

	Christian

[toc] | [prev] | [next] | [standalone]

#78723

In article <m0s610$qu9$1@dont-email.me>,
 Christian Gollwitzer <auriocus@gmx.de> wrote:
[... much good stuff deleted ... ]
> wish is a type A program, it creates an interpreter and therefore must 
> link to the actual library. So is Tkinter. But Tkagg is not, it extends 
> a preexisting interpreter.

Thanks, Christian.  That's a good summary, I think.  After some more 
research, I agree with your conclusion that the stubs library approach 
does not apply to the case of embedding Tcl ("type A") which is what 
Python tkinter does.  As far as matplotlib or PIL/Pillow or other 
third-party packages that may extend Tck/Tk, it might be helpful to open 
an issue with suggested code on the various projects' issue trackers if 
someone cares to pursue this.

-- 
 Ned Deily,
 nad@acm.org

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web