Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #17188 > unrolled thread

Bug#1117718: Introducing Free-Threaded Python to Debian

Back to article view | Back to linux.debian.maint.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#1117718: Introducing Free-Threaded Python to Debian Matthias Klose <doko@debian.org> - 2025-11-22 13:00 +0100
    Re: Bug#1117718: Introducing Free-Threaded Python to Debian Jerome Kieffer <Jerome.Kieffer@esrf.fr> - 2025-11-24 13:30 +0100
      Re: Bug#1117718: Introducing Free-Threaded Python to Debian Matthias Klose <doko@debian.org> - 2025-11-24 14:30 +0100

#17188 — Bug#1117718: Introducing Free-Threaded Python to Debian

I am not keen introducing this before 3.14 is the default.  This way, we 
can concentrate on the 3.14 transition first.  Then we can look at the 
free-threaded build, and also have a chance to look for regressions.

Matthias


On 11/16/25 16:16, ben0i0d wrote:
> Hello everyone,
> In my previous email (https://lists.debian.org/debian-python/2025/11/ 
> msg00002.html), I introduced my work on bringing free-threaded Python to 
> Debian. I'm now pleased to announce that the integration of nogil with 
> the GIL version has been successfully completed and has passed necessary 
> testing.
> Benefits of Introducing Free-Threaded Python:
> 1.Significant improvement in multi-threaded parallel performance
> 2.No need to migrate to multiprocessing or alternative languages - 
> currently multiprocessing is required to utilize multi-core performance, 
> but it comes with substantial overhead
> 3.Reduced maintenance burden for C/C++ extensions - currently complex 
> GIL management or custom thread pools are needed to work around the GIL
> 4.Alignment with future direction - PEP 703 has been accepted, and 
> upstream has confirmed that No-GIL will be the default in the future
> Current Progress:
> A merge request has been opened on salsa: https://salsa.debian.org/ 
> cpython-team/python3/-/merge_requests/41
> An immediately testable package repository is available at: https:// 
> salsa.debian.org/ben0i0d/python3t-repo
> The related Debian Bug report (#1117718) is also relevant to this work.
> Design Decisions:
> Based on discussions with Stefano, the current approach:
> -Maintains isolated standard libraries
> -Does NOT isolate dist-packages
> -Is not a separate new package, but rather a variant of python3
> My guiding principle remains: introducing free-threaded Python should 
> not create new problems, break existing functionality, or hinder 
> anyone's work.
> Technical Implementation Details:
> Key changes in python3.14:
> 1.Introduced ABI variant control: 'ABI_VARIANTS := gil nogil', with '-- 
> disable-gil --with-suffix=t' enabled for nogil
> (Note: '--enable-experimental-jit' cannot be used with '--disable-gil')
> 2.Build system optimization: Implemented duplication of nogil tasks via 
> '$(foreach abi,$(ABI_VARIANTS),$(eval $(call BUILD_STATIC,$(abi))))' to 
> avoid code redundancy
> 3.Test adjustments: Added 'TEST_EXCLUDES += test_tools' since 'Tools/ 
> freeze/test/Makefile' uses hardcoded 'python'
> 4.venv fix: Added 'add-abiflags-sitepackages.diff' to fix venv 
> sitepackages recognition after nogil build (this issue has been fixed 
> upstream and the patch will be removed after 3.14.1 release)
> 5.Minimal installation: The nogil version excludes documentation, idle, 
> tk, desktop, binfmt, etc., ensuring it's an extension rather than a 
> rewrite of the original python3
> 6.Code readability: Made harmless text adjustments to rules file, such 
> as consolidating scattered 'TEST_EXCLUDES +=' statements
> Build Process:
> git clone git@salsa.debian.org:ben0i0d/python3t.git
> cd python3t
> git checkout python3
> uscan --download-current-version --verbose
> dpkg-source -b .
> sudo env DEB_BUILD_OPTIONS="nocheck nobench" pbuilder build ../ 
> python3.14_3.14.0-5.dsc 2>&1 | tee ../log.txt
> Test Environment:
> -OS: Debian GNU/Linux forky/sid (forky) x86_64
> -CPU: AMD Ryzen 5 9600X (12) @ 5.68 GHz
> -Memory: 30.47 GiB
> -Kernel: 6.16.12+deb14+1-amd64
> Test Results:
> 1.Basic Performance Test
> # GIL version, single-threaded
> python3.14 benchmark.py --n 512 --threads 1
> Elapsed: 1.836 s
> # GIL version, 8 threads
> python3.14 benchmark.py --n 512 --threads 8
> Elapsed: 2.026 s
> # nogil version, single-threaded
> python3.14t benchmark.py --n 512 --threads 1
> Elapsed: 2.408 s
> # nogil version, 8 threads
> python3.14t benchmark.py --n 512 --threads 8
> Elapsed: 0.674 s
> 2.NumPy Compatibility Test
> Both versions successfully create virtual environments and install numpy:
> GIL environment:
> python3.14 -m venv gil
> source gil/bin/activate
> pip install numpy
> # Runs normally, Elapsed: 0.003 s
> nogil environment:
> python3.14t -m venv nogil
> source nogil/bin/activate
> pip install numpy
> # Runs normally, Elapsed: 0.009 s
> Important Notes:
> -The GIL version build results are identical to the current master branch
> -Building only the GIL version is supported, but building only nogil is 
> not (ensuring nogil doesn't become default)
> -No backport to python3.13 is planned since nogil support there is still 
> experimental
> Regarding dist-packages:
> I understand some colleagues have concerns about not isolating dist- 
> packages. Currently, I recommend users employ the nogil version within 
> venv environments. I will fully support subsequent migration efforts to 
> help more people transition smoothly.
> The current implementation requires more testing and refinement. I'm not 
> a CPython expert, so I genuinely welcome valuable suggestions from the 
> community and commit to actively participating in improvement efforts.
> Please refer to the attachments for detailed build logs and test scripts.
> I look forward to your feedback!
> Best regards,
> *Xu Chen (ben0i0d)*

[toc] | [next] | [standalone]

#17194

Well, today 3.13+3.13t, both exist, the same with 3.14+3.14t ... but
apparently for 3.15, i.e. in a year from now, the free-threaded version
will become the default, i.e 3.15t, which is a big change !

We still have time but postponing because it is not yet urgent looks
like procrastination to me.

Jerome

On Sat, 22 Nov 2025 12:52:23 +0100
Matthias Klose <doko@debian.org> wrote:

> I am not keen introducing this before 3.14 is the default.  This way, we 
> can concentrate on the 3.14 transition first.  Then we can look at the 
> free-threaded build, and also have a chance to look for regressions.
> 
> Matthias

[toc] | [prev] | [next] | [standalone]

#17195

On 11/24/25 13:24, Jerome Kieffer wrote:
> Well, today 3.13+3.13t, both exist, the same with 3.14+3.14t ... but
> apparently for 3.15, i.e. in a year from now, the free-threaded version
> will become the default, i.e 3.15t, which is a big change !
> 
> We still have time but postponing because it is not yet urgent looks
> like procrastination to me.

please re-read my email. I didn't say to postpone the free-threaded 
version until 3.15.

Matthias

> 
> Jerome
> 
> On Sat, 22 Nov 2025 12:52:23 +0100
> Matthias Klose <doko@debian.org> wrote:
> 
>> I am not keen introducing this before 3.14 is the default.  This way, we
>> can concentrate on the 3.14 transition first.  Then we can look at the
>> free-threaded build, and also have a chance to look for regressions.
>>
>> Matthias
> 

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web