Path: csiph.com!fu-berlin.de!bofh.it!news.nic.it!robomod From: Salvo Tomaselli Newsgroups: linux.debian.maint.python Subject: Re: Introducing Free-Threaded Python to Debian Date: Sat, 22 Nov 2025 11:10:02 +0100 Message-ID: References: X-Mailbox-Line: From debian-python-request@lists.debian.org Sat Nov 22 10:02:37 2025 Old-Return-Path: X-Amavis-Spam-Status: No, score=-6.994 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, LDO_WHITELIST=-5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001] autolearn=ham autolearn_force=no X-Policyd-Weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 CL_IP_EQ_HELO_IP=-2 (check from: .gmail. - helo: .mail-wr1-f51.google. - helo-domain: .google.) FROM/MX_MATCHES_HELO(DOMAIN)=-2; rate: -5.5 X-Gm-Message-State: AOJu0YzpUWFLya3xoL5p5MLe9cUxdEuFSTdQibuyN3Us5/lhg6JDnqBV dZD8IJz60oxxVuDfeirFRLDXmZAylgQ+QY6fAL8PruaEevxvQVlXkkiwtLU5L3ygzKoEBt9s2C+ DCPTbaXxDI4821PudV30ALhkaw54h/9c= X-Gm-Gg: ASbGncuN2yMC4ptFJvADSqEh3RHNtEbrzga1KxfaFFnz7Rd9UhNDG0q8w+fVoEzaCUJ sOqaGz/8q9WO+p1euQamw2lvA8Dg7P4ubix+9wucW+qbATFd4/WZv0Qq3yXZPzdjVd+EulYpwx6 Wn4p75YMLmIwXadlNebjBafiRq8yjmcmJSVqE2/oUZKyq/rcq6uBx86IvGTlGG5BrgxFlgrG7jV 3zSPdDZ53esI5JS2X8o9Rwl/KVygp/cqHUVkIBeJIsAvdA6CN6x9Na9mL+u3kJOJw4QoBLX9pyq 2gCpGGk3ZcdfvSzPgvKOSIpa1axt/EFZNWRqMQ== X-Google-SMTP-Source: AGHT+IFu3xDHK2kNx0gABLC2ySKbNTSy2sasdm8TJYWwYxefHnQvYBHBH8wu9owUE5ITtNErJD0rKhQKwLPQDP5RKKs= X-Received: by 2002:a05:6000:1889:b0:429:b21e:49c3 with SMTP id ffacd0b85a97d-42cc1390e79mr6070050f8f.26.1763805737092; Sat, 22 Nov 2025 02:02:17 -0800 (PST) MIME-Version: 1.0 X-Gm-Features: AWmQ_bl3TOE9VC71I-qd4n7nXkYfaP2pmv6B6f1JFBOVU_mcDWvMJ2m7TgI-GEM Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailing-List: archive/latest/23407 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/CAOOnQAdWwn4710RFtVPUofBFDjzV_dmEk1zSHcfu-9tLP7RaBg@mail.gmail.com Approved: robomod@news.nic.it Lines: 134 Organization: linux.* mail to news gateway Sender: robomod@news.nic.it X-Original-Cc: debian-python@lists.debian.org X-Original-Date: Sat, 22 Nov 2025 11:02:05 +0100 X-Original-Message-ID: X-Original-References: Xref: csiph.com linux.debian.maint.python:17186 Hey, Thanks for all your work and for the nice email! > 4. Alignment with future direction - PEP 703 has been accepted, and upstr= eam has confirmed that No-GIL will be the default in the future They have backtracked in the past about such decisions. For example the whole mess with PEP 563. So we will see what happens. The single thread slowdown is quite considerable. Best Il giorno dom 16 nov 2025 alle ore 16:16 ben0i0d ha scritto: > > Hello everyone, > In my previous email (https://lists.debian.org/debian-python/2025/11/msg0= 0002.html), I introduced my work on bringing free-threaded Python to Debian= . I'm now pleased to announce that the integration of nogil with the GIL ve= rsion has been successfully completed and has passed necessary testing. > Benefits of Introducing Free-Threaded Python: > 1. Significant improvement in multi-threaded parallel performance > 2. No need to migrate to multiprocessing or alternative languages - curre= ntly multiprocessing is required to utilize multi-core performance, but it = comes with substantial overhead > 3. Reduced maintenance burden for C/C++ extensions - currently complex GI= L management or custom thread pools are needed to work around the GIL > 4. Alignment with future direction - PEP 703 has been accepted, and upstr= eam has confirmed that No-GIL will be the default in the future > Current Progress: > A merge request has been opened on salsa: https://salsa.debian.org/cpytho= n-team/python3/-/merge_requests/41 > An immediately testable package repository is available at: https://salsa= .debian.org/ben0i0d/python3t-repo > The related Debian Bug report (#1117718) is also relevant to this work. > Design Decisions: > Based on discussions with Stefano, the current approach: > - Maintains isolated standard libraries > - Does NOT isolate dist-packages > - Is not a separate new package, but rather a variant of python3 > My guiding principle remains: introducing free-threaded Python should not= create new problems, break existing functionality, or hinder anyone's work= . > Technical Implementation Details: > Key changes in python3.14: > 1. Introduced ABI variant control: 'ABI_VARIANTS :=3D gil nogil', with '-= -disable-gil --with-suffix=3Dt' enabled for nogil > (Note: '--enable-experimental-jit' cannot be used with '--disable-gil') > 2. Build system optimization: Implemented duplication of nogil tasks via = '$(foreach abi,$(ABI_VARIANTS),$(eval $(call BUILD_STATIC,$(abi))))' to avo= id code redundancy > 3. Test adjustments: Added 'TEST_EXCLUDES +=3D test_tools' since 'Tools/f= reeze/test/Makefile' uses hardcoded 'python' > 4. venv fix: Added 'add-abiflags-sitepackages.diff' to fix venv sitepacka= ges recognition after nogil build (this issue has been fixed upstream and t= he patch will be removed after 3.14.1 release) > 5. Minimal installation: The nogil version excludes documentation, idle, = tk, desktop, binfmt, etc., ensuring it's an extension rather than a rewrite= of the original python3 > 6. Code readability: Made harmless text adjustments to rules file, such a= s consolidating scattered 'TEST_EXCLUDES +=3D' statements > Build Process: > git clone git@salsa.debian.org:ben0i0d/python3t.git > cd python3t > git checkout python3 > uscan --download-current-version --verbose > dpkg-source -b . > sudo env DEB_BUILD_OPTIONS=3D"nocheck nobench" pbuilder build ../python3.= 14_3.14.0-5.dsc 2>&1 | tee ../log.txt > Test Environment: > - OS: Debian GNU/Linux forky/sid (forky) x86_64 > - CPU: AMD Ryzen 5 9600X (12) @ 5.68 GHz > - Memory: 30.47 GiB > - Kernel: 6.16.12+deb14+1-amd64 > Test Results: > 1. Basic Performance Test > # GIL version, single-threaded > python3.14 benchmark.py --n 512 --threads 1 > Elapsed: 1.836 s > # GIL version, 8 threads > python3.14 benchmark.py --n 512 --threads 8 > Elapsed: 2.026 s > # nogil version, single-threaded > python3.14t benchmark.py --n 512 --threads 1 > Elapsed: 2.408 s > # nogil version, 8 threads > python3.14t benchmark.py --n 512 --threads 8 > Elapsed: 0.674 s > 2. NumPy Compatibility Test > Both versions successfully create virtual environments and install numpy: > GIL environment: > python3.14 -m venv gil > source gil/bin/activate > pip install numpy > # Runs normally, Elapsed: 0.003 s > nogil environment: > python3.14t -m venv nogil > source nogil/bin/activate > pip install numpy > # Runs normally, Elapsed: 0.009 s > Important Notes: > - The GIL version build results are identical to the current master branc= h > - Building only the GIL version is supported, but building only nogil is = not (ensuring nogil doesn't become default) > - No backport to python3.13 is planned since nogil support there is still= experimental > Regarding dist-packages: > I understand some colleagues have concerns about not isolating dist-packa= ges. Currently, I recommend users employ the nogil version within venv envi= ronments. I will fully support subsequent migration efforts to help more pe= ople transition smoothly. > The current implementation requires more testing and refinement. I'm not = a CPython expert, so I genuinely welcome valuable suggestions from the comm= unity and commit to actively participating in improvement efforts. > Please refer to the attachments for detailed build logs and test scripts. > I look forward to your feedback! > Best regards, > Xu Chen (ben0i0d) --=20 Salvo Tomaselli I difensori della morale tradizionale sono raramente persone di cuore. Si = =C3=A8 tentati di pensare che essi si servano della morale come di legittimo sfogo al loro desiderio di fare del male agli altri. -- Bertrand Russell, Perch=C3=A9 non sono cristiano. 1957 http://ltworf.github.io/