Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.rc > #405101 > unrolled thread

Bug#1121177: openblas breaks multiple autopkgtest on ppc64el

Started byPaul Gevers <elbrus@debian.org>
First post2025-11-22 12:10 +0100
Last post2026-01-18 09:10 +0100
Articles 6 — 3 participants

Back to article view | Back to linux.debian.bugs.rc

Contents

  Bug#1121177: openblas breaks multiple autopkgtest on ppc64el Paul Gevers <elbrus@debian.org> - 2025-11-22 12:10 +0100
    Bug#1121177: openblas breaks multiple autopkgtest on ppc64el Sébastien Villemot <sebastien@debian.org> - 2025-11-25 15:50 +0100
    Processed: Re: Bug#1121177: openblas breaks multiple autopkgtest  on ppc64el "Debian Bug Tracking System" <owner@bugs.debian.org> - 2025-11-25 15:50 +0100
    Bug#1121177: openblas breaks multiple autopkgtest on ppc64el Sébastien Villemot <sebastien@debian.org> - 2026-01-14 20:10 +0100
    Bug#1121177: openblas breaks multiple autopkgtest on ppc64el Sébastien Villemot <sebastien@debian.org> - 2026-01-16 10:10 +0100
    Processed: Re: Bug#1121177: openblas breaks multiple autopkgtest  on ppc64el "Debian Bug Tracking System" <owner@bugs.debian.org> - 2026-01-18 09:10 +0100

#405101 — Bug#1121177: openblas breaks multiple autopkgtest on ppc64el

FromPaul Gevers <elbrus@debian.org>
Date2025-11-22 12:10 +0100
SubjectBug#1121177: openblas breaks multiple autopkgtest on ppc64el
Message-ID<LTTF7-eSNL-1@gated-at.bofh.it>

[Multipart message — attachments visible in raw view] — view raw

Source: openblas
Control: found -1 openblas/0.3.30+ds-3
Severity: serious
Tags: sid forky
User: debian-ci@lists.debian.org
Usertags: breaks
Control: affects -1 src:dolfin src:gemma src:openmolcas src:xtensor-blas

Dear maintainer(s),

With a recent upload of openblas the autopkgtests of dolfin, gemma, 
openmolcas, and xtensor-blas fail on ppc64el in testing when their 
autopkgtest is run with the binary packages of openblas from unstable. 
It passes when run with only packages from testing. In tabular form (for 
dolfin):

                        pass            fail
openblas               from testing    0.3.30+ds-3
dolfin                 from testing    2019.2.0~legacy20240219.1c52e83-24
all others             from testing    from testing

I copied some of the output at the bottom of this report.

Currently this regression is blocking the migration of openblas to 
testing [1]. Can you please investigate the situation?

Someone pointed me at https://github.com/OpenMathLib/OpenBLAS/pull/5463 
which may completely unrelated, but at least include changes for ppc64el.

Paul

[1] https://qa.debian.org/excuses.php?package=openblas

https://ci.debian.net/data/autopkgtest/testing/ppc64el/d/dolfin/66420095/log.gz

284s       Start 72: demo_singular-poisson_serial
285s 29/49 Test #72: demo_singular-poisson_serial 
..............Subprocess aborted***Exception:   1.17 sec
285s terminate called after throwing an instance of 'std::runtime_error'
285s   what():  285s 285s *** 
-------------------------------------------------------------------------
285s *** DOLFIN encountered an error. If you are not able to resolve 
this issue
285s *** using the information listed below, you can ask for help at
285s ***
285s ***     https://fenicsproject.discourse.group/
285s ***
285s *** Remember to include the error message listed below and, if 
possible,
285s *** include a *minimal* running example to reproduce the error.
285s ***
285s *** 
-------------------------------------------------------------------------
285s *** Error:   Unable to successfully call PETSc function 'KSPSolve'.
285s *** Reason:  PETSc error code is: 76 (Error in external library).
285s *** Where:   This error was encountered inside 
./dolfin/la/PETScKrylovSolver.cpp.
285s *** Process: 0
285s *** 285s *** DOLFIN version: 2019.2.0.64.dev0
285s *** Git changeset:  debian_2019.2.0~legacy20240219.1c52e83-24
285s *** 
-------------------------------------------------------------------------
285s 285s [ci-325-a328d227:14164] *** Process received signal ***
285s [ci-325-a328d227:14164] Signal: Aborted (6)
285s [ci-325-a328d227:14164] Signal code:  (-6)
285s [ci-325-a328d227:14164] [ 0] 
linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0) [0x3fffbb002494]
285s [ci-325-a328d227:14164] [ 1] 
/lib/powerpc64le-linux-gnu/libc.so.6(+0xafd3c) [0x3fffb807fd3c]
285s [ci-325-a328d227:14164] [ 2] 
/lib/powerpc64le-linux-gnu/libc.so.6(gsignal+0x2c) [0x3fffb801663c]
285s [ci-325-a328d227:14164] [ 3] 
/lib/powerpc64le-linux-gnu/libc.so.6(abort+0x28) [0x3fffb7ff65f0]
285s [ci-325-a328d227:14164] [ 4] 
/lib/powerpc64le-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x158) 
[0x3fffb83ae858]
285s [ci-325-a328d227:14164] [ 5] 
/lib/powerpc64le-linux-gnu/libstdc++.so.6(+0x119ad4) [0x3fffb83a9ad4]
285s [ci-325-a328d227:14164] [ 6] 
/lib/powerpc64le-linux-gnu/libstdc++.so.6(_ZSt9terminatev+0x20) 
[0x3fffb835527c]
285s [ci-325-a328d227:14164] [ 7] 
/lib/powerpc64le-linux-gnu/libstdc++.so.6(__cxa_throw+0x7c) [0x3fffb83a9fec]
285s [ci-325-a328d227:14164] [ 8] 
/lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZNK6dolfin6Logger12dolfin_errorENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_S6_i+0xc1c) 
[0x3fffbada1bbc]
285s [ci-325-a328d227:14164] [ 9] 
/lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin12dolfin_errorENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES5_S5_z+0x184) 
[0x3fffbad9e354]
285s [ci-325-a328d227:14164] [10] 
/lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin11PETScObject11petsc_errorEiNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_+0x370) 
[0x3fffbad72cc0]
285s [ci-325-a328d227:14164] [11] 
/lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin17PETScKrylovSolver5solveERNS_11PETScVectorERKS1_b+0x1c44) 
[0x3fffbad57ad4]
285s [ci-325-a328d227:14164] [12] 
/lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin17PETScKrylovSolver5solveERNS_13GenericVectorERKS1_+0x58) 
[0x3fffbad58068]
285s [ci-325-a328d227:14164] [13] 
/tmp/autopkgtest-lxc.qr0i5fy5/downtmp/build.bm6/src/dolfin-demo/documented/singular-poisson/cpp/demo_singular-poisson(+0xb5c8) 
[0x13e65b5c8]
285s [ci-325-a328d227:14164] [14] 
/lib/powerpc64le-linux-gnu/libc.so.6(+0x26f0c) [0x3fffb7ff6f0c]
285s [ci-325-a328d227:14164] [15] 
/lib/powerpc64le-linux-gnu/libc.so.6(__libc_start_main+0x1ac) 
[0x3fffb7ff714c]
285s [ci-325-a328d227:14164] *** End of error message ***

[toc] | [next] | [standalone]

#405304

FromSébastien Villemot <sebastien@debian.org>
Date2025-11-25 15:50 +0100
Message-ID<LV2wF-fEEZ-3@gated-at.bofh.it>
In reply to#405101

[Multipart message — attachments visible in raw view] — view raw

Control: tags -1 + help

Le samedi 22 novembre 2025 à 12:03 +0100, Paul Gevers a écrit :
> Source: openblas
> Control: found -1 openblas/0.3.30+ds-3
> Severity: serious
> Tags: sid forky
> User: debian-ci@lists.debian.org
> Usertags: breaks
> Control: affects -1 src:dolfin src:gemma src:openmolcas src:xtensor-blas
> 
> Dear maintainer(s),
> 
> With a recent upload of openblas the autopkgtests of dolfin, gemma, 
> openmolcas, and xtensor-blas fail on ppc64el in testing when their 
> autopkgtest is run with the binary packages of openblas from unstable. 
> It passes when run with only packages from testing. In tabular form (for 
> dolfin):
> 
>                         pass            fail
> openblas               from testing    0.3.30+ds-3
> dolfin                 from testing    2019.2.0~legacy20240219.1c52e83-24
> all others             from testing    from testing
> 
> I copied some of the output at the bottom of this report.
> 
> Currently this regression is blocking the migration of openblas to 
> testing [1]. Can you please investigate the situation?
> 
> Someone pointed me at https://github.com/OpenMathLib/OpenBLAS/pull/5463 
> which may completely unrelated, but at least include changes for ppc64el.

I talked to upstream about the problem (in an issue that was initially
about a FTBFS, due to a failure in OpenBLAS own testsuite, which has
since been fixed):
https://github.com/OpenMathLib/OpenBLAS/issues/5372#issuecomment-3353517450

Unfortunately upstream does not really know where the test failures in
third-party software come from. In particular, they can’t replicate the
issue (note that they tried with more recent git snapshot than version
0.3.30), and I couldn’t either with Debian version 0.3.30+ds-3 (tried
on the ppc64el Debian porterbox).

At this point, fixing this issue is beyond my time budget and skills (I
know next to zero about PowerPC, and the issue is probably due to some
changes to PowerPC assembly code). CC’ing the Debian PowerPC porters,
with the hope that they can help.

> [1] https://qa.debian.org/excuses.php?package=openblas
> 
> https://ci.debian.net/data/autopkgtest/testing/ppc64el/d/dolfin/66420095/log.gz
> 
> 284s       Start 72: demo_singular-poisson_serial
> 285s 29/49 Test #72: demo_singular-poisson_serial 
> ..............Subprocess aborted***Exception:   1.17 sec
> 285s terminate called after throwing an instance of 'std::runtime_error'
> 285s   what():  285s 285s *** 
> -------------------------------------------------------------------------
> 285s *** DOLFIN encountered an error. If you are not able to resolve 
> this issue
> 285s *** using the information listed below, you can ask for help at
> 285s ***
> 285s ***     https://fenicsproject.discourse.group/
> 285s ***
> 285s *** Remember to include the error message listed below and, if 
> possible,
> 285s *** include a *minimal* running example to reproduce the error.
> 285s ***
> 285s *** 
> -------------------------------------------------------------------------
> 285s *** Error:   Unable to successfully call PETSc function 'KSPSolve'.
> 285s *** Reason:  PETSc error code is: 76 (Error in external library).
> 285s *** Where:   This error was encountered inside 
> ./dolfin/la/PETScKrylovSolver.cpp.
> 285s *** Process: 0
> 285s *** 285s *** DOLFIN version: 2019.2.0.64.dev0
> 285s *** Git changeset:  debian_2019.2.0~legacy20240219.1c52e83-24
> 285s *** 
> -------------------------------------------------------------------------
> 285s 285s [ci-325-a328d227:14164] *** Process received signal ***
> 285s [ci-325-a328d227:14164] Signal: Aborted (6)
> 285s [ci-325-a328d227:14164] Signal code:  (-6)
> 285s [ci-325-a328d227:14164] [ 0] 
> linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0) [0x3fffbb002494]
> 285s [ci-325-a328d227:14164] [ 1] 
> /lib/powerpc64le-linux-gnu/libc.so.6(+0xafd3c) [0x3fffb807fd3c]
> 285s [ci-325-a328d227:14164] [ 2] 
> /lib/powerpc64le-linux-gnu/libc.so.6(gsignal+0x2c) [0x3fffb801663c]
> 285s [ci-325-a328d227:14164] [ 3] 
> /lib/powerpc64le-linux-gnu/libc.so.6(abort+0x28) [0x3fffb7ff65f0]
> 285s [ci-325-a328d227:14164] [ 4] 
> /lib/powerpc64le-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x158) 
> [0x3fffb83ae858]
> 285s [ci-325-a328d227:14164] [ 5] 
> /lib/powerpc64le-linux-gnu/libstdc++.so.6(+0x119ad4) [0x3fffb83a9ad4]
> 285s [ci-325-a328d227:14164] [ 6] 
> /lib/powerpc64le-linux-gnu/libstdc++.so.6(_ZSt9terminatev+0x20) 
> [0x3fffb835527c]
> 285s [ci-325-a328d227:14164] [ 7] 
> /lib/powerpc64le-linux-gnu/libstdc++.so.6(__cxa_throw+0x7c) [0x3fffb83a9fec]
> 285s [ci-325-a328d227:14164] [ 8] 
> /lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZNK6dolfin6Logger12dolfin_errorENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_S6_i+0xc1c) 
> [0x3fffbada1bbc]
> 285s [ci-325-a328d227:14164] [ 9] 
> /lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin12dolfin_errorENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES5_S5_z+0x184) 
> [0x3fffbad9e354]
> 285s [ci-325-a328d227:14164] [10] 
> /lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin11PETScObject11petsc_errorEiNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_+0x370) 
> [0x3fffbad72cc0]
> 285s [ci-325-a328d227:14164] [11] 
> /lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin17PETScKrylovSolver5solveERNS_11PETScVectorERKS1_b+0x1c44) 
> [0x3fffbad57ad4]
> 285s [ci-325-a328d227:14164] [12] 
> /lib/powerpc64le-linux-gnu/libdolfin.so.2019.2t64(_ZN6dolfin17PETScKrylovSolver5solveERNS_13GenericVectorERKS1_+0x58) 
> [0x3fffbad58068]
> 285s [ci-325-a328d227:14164] [13] 
> /tmp/autopkgtest-lxc.qr0i5fy5/downtmp/build.bm6/src/dolfin-demo/documented/singular-poisson/cpp/demo_singular-poisson(+0xb5c8) 
> [0x13e65b5c8]
> 285s [ci-325-a328d227:14164] [14] 
> /lib/powerpc64le-linux-gnu/libc.so.6(+0x26f0c) [0x3fffb7ff6f0c]
> 285s [ci-325-a328d227:14164] [15] 
> /lib/powerpc64le-linux-gnu/libc.so.6(__libc_start_main+0x1ac) 
> [0x3fffb7ff714c]
> 285s [ci-325-a328d227:14164] *** End of error message ***

-- 
⢀⣴⠾⠻⢶⣦⠀  Sébastien Villemot
⣾⠁⢠⠒⠀⣿⡁  Debian Developer
⢿⡄⠘⠷⠚⠋⠀  https://sebastien.villemot.name
⠈⠳⣄⠀⠀⠀⠀  https://www.debian.org

[toc] | [prev] | [next] | [standalone]

#405305 — Processed: Re: Bug#1121177: openblas breaks multiple autopkgtest on ppc64el

From"Debian Bug Tracking System" <owner@bugs.debian.org>
Date2025-11-25 15:50 +0100
SubjectProcessed: Re: Bug#1121177: openblas breaks multiple autopkgtest on ppc64el
Message-ID<LV2wF-fEEZ-13@gated-at.bofh.it>
In reply to#405101
Processing control commands:

> tags -1 + help
Bug #1121177 [src:openblas] openblas breaks multiple autopkgtest on ppc64el
Added tag(s) help.

-- 
1121177: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1121177
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [next] | [standalone]

#408951

FromSébastien Villemot <sebastien@debian.org>
Date2026-01-14 20:10 +0100
Message-ID<MdepH-aydd-1@gated-at.bofh.it>
In reply to#405101

[Multipart message — attachments visible in raw view] — view raw

Le mercredi 07 janvier 2026 à 23:35 +0530, Trupti a écrit :
> On 2026-01-07 11:29, Simon Richter wrote:
> > On 1/7/26 5:57 AM, Sébastien Villemot wrote:
> > 
> > > If you have the time, could you possibly also check that the two other
> > > autopkgtest regressions (in src:gemma and src:openmolcas) are also
> > > tolerance-related? (see https://tracker.debian.org/pkg/openblas for 
> > > the
> > > list of autopkgtest regressions)
> 
> For src:gemma, the autopkgtest failure on ppc64el occurs during the 
> eigen-decomposition step.
> The run reports a warning about many eigenvalues close to zero, followed 
> by an LU decomposition failure in GSL/LAPACK.
> The failure is triggered in the following code path:
> 
> // LU decomposition.
> void LUDecomp(gsl_matrix *LU, gsl_permutation *p, int *signum) {
>    // debug_msg("entering");
>    enforce_gsl(gsl_linalg_LU_decomp(LU, p, signum));
>    return;
> }
> 
> 
> 
> For src:openmolcas, the autopkgtest failures on ppc64el are limited to 
> CASPT2 tests (standard tests 009, 010 and hdf5 test 601).
> The logs show floating-point exceptions (IEEE invalid, divide-by-zero, 
> underflow) followed by CASPT2 convergence failures (_NOT_CONVERGED_ / 
> _INTERNAL_ERROR_).
> All non-CASPT2 tests complete successfully.  The CASPT2 output itself 
> indicates numerical instability and suggests increasing 
> linear-dependence thresholds.

Thanks Trupti.

Do you consider the following as a good summary of your analysis: there
is no structural problem in the new OpenBLAS on ppc64el (just slightly
numerically different results, within the usual tolerance of numerical
software), and as a consequence the adjustment needs to be done in the
testsuite of the affected reverse dependencies (xtensor-blas, gemma and
openmolcas)? (that seems clear from what you said of src:xtensor-blas
and src:openmolcas, less so for the case of src:gemma, hence my
question)

Paul: if my statement above is correct, what would be the right course
of action?

-- 
⢀⣴⠾⠻⢶⣦⠀  Sébastien Villemot
⣾⠁⢠⠒⠀⣿⡁  Debian Developer
⢿⡄⠘⠷⠚⠋⠀  https://sebastien.villemot.name
⠈⠳⣄⠀⠀⠀⠀  https://www.debian.org

[toc] | [prev] | [next] | [standalone]

#409192

FromSébastien Villemot <sebastien@debian.org>
Date2026-01-16 10:10 +0100
Message-ID<MdO09-aWJv-1@gated-at.bofh.it>
In reply to#405101

[Multipart message — attachments visible in raw view] — view raw

Le mercredi 14 janvier 2026 à 20:29 +0100, Paul Gevers a écrit :
> On 1/14/26 20:05, Sébastien Villemot wrote:
> > Paul: if my statement above is correct, what would be the right course
> > of action?
> 
> The right action would be to get those pacakges fixed, e.g. via filing 
> bugs (cloning and reassigning might be appropriate). When the bugs are 
> in place, I can hint src:openblas into testing.

Thanks, this has been done by Trupti.

-- 
⢀⣴⠾⠻⢶⣦⠀  Sébastien Villemot
⣾⠁⢠⠒⠀⣿⡁  Debian Developer
⢿⡄⠘⠷⠚⠋⠀  https://sebastien.villemot.name
⠈⠳⣄⠀⠀⠀⠀  https://www.debian.org

[toc] | [prev] | [next] | [standalone]

#409318 — Processed: Re: Bug#1121177: openblas breaks multiple autopkgtest on ppc64el

From"Debian Bug Tracking System" <owner@bugs.debian.org>
Date2026-01-18 09:10 +0100
SubjectProcessed: Re: Bug#1121177: openblas breaks multiple autopkgtest on ppc64el
Message-ID<Mew1b-bqkD-5@gated-at.bofh.it>
In reply to#405101
Processing control commands:

> close -1
Bug #1121177 [src:openblas] openblas breaks multiple autopkgtest on ppc64el
Marked Bug as done

-- 
1121177: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1121177
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.rc

csiph-web