Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1271151 > unrolled thread

Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py Santiago Vila <sanvila@debian.org> - 2025-11-22 12:10 +0100
    Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py Santiago Vila <sanvila@debian.org> - 2025-11-22 12:40 +0100
      Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py Blair Noctis <ncts@debian.org> - 2025-11-22 14:20 +0100
    Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py Santiago Vila <sanvila@debian.org> - 2025-11-25 17:00 +0100
      Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py Johannes Altmanninger <aclopte@gmail.com> - 2025-11-26 15:10 +0100
        Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py Blair Noctis <ncts@debian.org> - 2025-11-30 07:40 +0100

#1271151 — Bug#1121178: fish: FTBFS: Failed tests: tests/pexpects/fkr.py

Package: src:fish
Version: 4.2.1-1
Severity: serious
Tags: ftbfs forky sid

Dear maintainer:

During a rebuild of all packages in unstable, this package failed to build.

Below you will find the last part of the build log (probably the most
relevant part, but not necessarily). If required, the full build log
is available here:

https://people.debian.org/~sanvila/build-logs/202511/

Note: The build does not always fail, but it fails 75% of the time here,
which is certainly not suitable for release.

Other failures, not the same, also happen here:

https://buildd.debian.org/status/package.php?p=fish


About the archive rebuild: The build was made on virtual machines from AWS,
using sbuild and a reduced chroot with only build-essential packages.

If you cannot reproduce the bug please contact me privately, as I
am willing to provide ssh access to a virtual machine where the bug is
fully reproducible (in this case, not fully but "just" 75% of the time).

If this is really a bug in one of the build-depends, please use
reassign and add an affects on src:fish, so that this is still
visible in the BTS web page for this package.

Thanks.

--------------------------------------------------------------------------------
[...]
tests/pexpects/bind_mode_events.py                               PASSED    5605 ms
tests/pexpects/terminal.py                                       PASSED    5234 ms
tests/checks/tmux-transient-prompt.fish                          PASSED    6115 ms
tests/pexpects/fkr.py                                            FAILED    6078 ms
Failed to match pattern: prompt 2
fkr.py:25: timeout from expect_prompt()

Escaped buffer:


When written to the tty, this looks like:
<-------

(no trailing newline)------->

Last 10 messages:
OUTPUT       0.00 ms (Line 21): prompt 1>
 INPUT      19.63 ms (Line 23): $fish_key_reader --version\n
OUTPUT      +4.66 ms (Line 24): fish_key_reader, version 4.2.1\r\n⏎                                                                               \r⏎ \r\rprompt 2>

Tmpdir is /tmp/fishtest-root-7wly9jge/fishtest-pvekotnp
tests/checks/tmux-autosuggestion.fish                            PASSED    6590 ms
tests/pexpects/fg.py                                             PASSED    6239 ms
tests/pexpects/signals.py                                        PASSED    5873 ms
tests/checks/tmux-empty-prompt.fish                              PASSED    6652 ms
tests/checks/fish_config.fish                                    PASSED    7489 ms
tests/checks/po-files-well-formed.fish                           PASSED    7707 ms
tests/pexpects/bind.py                                           PASSED    7216 ms
tests/checks/tmux-commandline.fish                               PASSED    7683 ms
tests/checks/tmux-history-search.fish                            PASSED    8088 ms
tests/checks/tmux-autosuggestion-multiline.fish                  PASSED    8609 ms
tests/checks/tmux-history-pager.fish                             PASSED    9698 ms
tests/checks/tmux-complete.fish                                  PASSED   10220 ms
tests/checks/sphinx-man.fish                                     PASSED   12210 ms
tests/checks/check-all-fish-files.fish                           PASSED   13093 ms
tests/checks/sphinx-html.fish                                    PASSED   14946 ms
219 / 220 passed (4 skipped)
Failed tests:
    tests/pexpects/fkr.py
FAILED: [code=1] CMakeFiles/fish_run_tests /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/CMakeFiles/fish_run_tests 
cd /<<PKGBUILDDIR>> && /<<PKGBUILDDIR>>/tests/test_driver.py /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu && env FISH_CMAKE_BINARY_DIR=/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu PREFIX=/usr DOCDIR=/usr/share/doc/fish DATADIR=/usr/share SYSCONFDIR=/etc BINDIR=/usr/bin CARGO_TARGET_DIR=/<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/cargo/build CARGO_BUILD_RUSTC=/usr/bin/rustc RUSTFLAGS=\ -g FISH_SPHINX=/usr/bin/sphinx-build FISH_USE_PREBUILT_DOCS=FALSE /usr/bin/cargo test --no-default-features --workspace --target-dir /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu/cargo/build/x86_64-unknown-linux-gnu --target x86_64-unknown-linux-gnu --lib
ninja: build stopped: subcommand failed.
make[1]: *** [debian/rules:30: override_dh_auto_test] Error 1
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:14: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
--------------------------------------------------------------------------------

[toc] | [next] | [standalone]

#1271154

Hi. Shortly after reporting this I see there is a 4.2.1-2 version uploaded today.

If the change "Reinstate fix-libc-timespec.patch for armhf" is only for armhf
then it will have no effect on the issue I reported, but I can't tell without
looking at the source. Could you please push the pending changes to salsa?

Thanks.

[toc] | [prev] | [next] | [standalone]

#1271162

[Multipart message — attachments visible in raw view] — view raw

-> Santiago Vila <sanvila@debian.org>, 2025-11-22T12:29:51+0100 ->
> Hi. Shortly after reporting this I see there is a 4.2.1-2 version uploaded today.
> 
> If the change "Reinstate fix-libc-timespec.patch for armhf" is only for armhf
> then it will have no effect on the issue I reported, but I can't tell without
> looking at the source. Could you please push the pending changes to salsa?
> 
> Thanks.

Hi Santiago, -2 was pushed to salsa, and you are right that it wasn't 
about the test failure you filed.

As for the "real" failures, tests/pexpects/fkr.py did fail in some runs, 
but it's not the only one, and not always a failure. Some other tests 
are also failing seemingly randomly, and in my latest few runs the main 
ones are tests/checks/{cd,path,fish_config}.fish. My initial thought was 
concurrency problems, but the exact reason has yet been confirmed, as 
well as if there is only one thing wrong. Unfortunately I'm not quite 
familiar with fish tests, so it's possible this has to be upstreamed.

-- 
​    ,Sdrager
Blair Noctis

🇵🇸

[toc] | [prev] | [next] | [standalone]

#1271555

On Tue, Nov 25, 2025 at 11:10:05PM +0800, Blair Noctis wrote:
> Control: severity -1 important
> 
> After a few givebacks, official architectures all pass (minus riscv64 the
> buildds of which are too slow to not have timeouts, so I skipped tests on it
> in -3). This further confirms my suspicion that the failures were indeed due
> to concurrency problems.

Well, sorry but I don't think downgrading this bug is ok. Flaky tests
are RC since trixie. See my build history, on AWS instances of type
c7a.large and m7a.large (both amd64), which incidentally have 2 CPUs,
nothing special:

Status: successful  fish_4.2.1-2_amd64-20251122T152823.838Z
Status: successful  fish_4.2.1-2_amd64-20251122T152828.212Z
Status: failed      fish_4.2.1-2_amd64-20251122T152835.065Z
Status: successful  fish_4.2.1-2_amd64-20251122T153152.998Z
Status: failed      fish_4.2.1-2_amd64-20251122T153156.907Z
Status: failed      fish_4.2.1-2_amd64-20251122T153200.267Z
Status: failed      fish_4.2.1-2_amd64-20251122T153250.171Z
Status: failed      fish_4.2.1-2_amd64-20251122T153256.782Z
Status: failed      fish_4.2.1-2_amd64-20251122T153258.062Z
Status: failed      fish_4.2.1-2_amd64-20251122T153259.916Z
Status: failed      fish_4.2.1-2_amd64-20251122T153300.655Z
Status: failed      fish_4.2.1-2_amd64-20251122T153300.063Z
Status: successful  fish_4.2.1-2_amd64-20251122T153301.720Z
Status: failed      fish_4.2.1-2_amd64-20251122T153302.716Z
Status: failed      fish_4.2.1-2_amd64-20251122T153308.287Z
Status: failed      fish_4.2.1-2_amd64-20251122T153410.511Z
Status: failed      fish_4.2.1-2_amd64-20251122T153447.462Z
Status: failed      fish_4.2.1-2_amd64-20251122T153532.887Z
Status: failed      fish_4.2.1-2_amd64-20251122T153541.244Z
Status: failed      fish_4.2.1-2_amd64-20251122T153607.111Z
Status: failed      fish_4.2.1-2_amd64-20251122T153721.453Z
Status: failed      fish_4.2.1-2_amd64-20251122T153726.299Z
Status: failed      fish_4.2.1-2_amd64-20251123T041940.252Z
Status: successful  fish_4.2.1-2_amd64-20251124T041935.168Z
Status: failed      fish_4.2.1-2_amd64-20251125T041741.873Z

In my opinion, a package which fails to build 80% of the time may
never be considered suitable for release, and if our current policy
says it is ok to downgrade this, maybe it's time to change policy.

Question for Paul: What needs to happen so that people stop
downgrading bugs in cases like this one?

Thanks.

[toc] | [prev] | [next] | [standalone]

#1271685

> > After a few givebacks, official architectures all pass (minus riscv64 the
> > buildds of which are too slow to not have timeouts, so I skipped tests on it
> > in -3). This further confirms my suspicion that the failures were indeed due
> > to concurrency problems.
> 
> Well, sorry but I don't think downgrading this bug is ok. Flaky tests
> are RC since trixie. See my build history, on AWS instances of type
> c7a.large and m7a.large (both amd64), which incidentally have 2 CPUs,
> nothing special:

Note that
- the (latent) flaky issues seem known and have existed for many years.
- upstream is working on resolving them (https://github.com/fish-shell/fish-shell/issues/11815);
  I recommend not spending too much time on this, since it's really an upstream problem.
- upstream does not run some of the flaky tests in CI,
  based on CI or FISH_CI_SAN environment variables.
  For example, the failing "fkr.py"

	# Disable under CI - keeps failing because the timing is too tight
	if "CI" in os.environ:
	    sys.exit(127)

  However, they all pass when given enough resources and/or with `export FISH_TEST_MAX_CONCURRENCY=1`,
  and having them enabled by default can help developers.
- So as a workaround, maybe either
  - set `export CI=1` when running tests
  - and/or `export FISH_TEST_MAX_CONCURRENCY=1` at cmake-configure time
  - or disable affected tests
  until upstream fixes these.

[toc] | [prev] | [next] | [standalone]

#1272167

[Multipart message — attachments visible in raw view] — view raw

On Wed, 26 Nov 2025 14:58:41 +0100 Johannes Altmanninger 
<aclopte@gmail.com> wrote:
> > > After a few givebacks, official architectures all pass (minus riscv64 the
> > > buildds of which are too slow to not have timeouts, so I skipped tests on it
> > > in -3). This further confirms my suspicion that the failures were indeed due
> > > to concurrency problems.
> > 
> > Well, sorry but I don't think downgrading this bug is ok. Flaky tests
> > are RC since trixie. See my build history, on AWS instances of type
> > c7a.large and m7a.large (both amd64), which incidentally have 2 CPUs,
> > nothing special:
> 
> Note that
> - the (latent) flaky issues seem known and have existed for many years.
> - upstream is working on resolving them (https://github.com/fish-shell/fish-shell/issues/11815);
>   I recommend not spending too much time on this, since it's really an upstream problem.
> - upstream does not run some of the flaky tests in CI,
>   based on CI or FISH_CI_SAN environment variables.
>   For example, the failing "fkr.py"
> 
> 	# Disable under CI - keeps failing because the timing is too tight
> 	if "CI" in os.environ:
> 	    sys.exit(127)
> 
>   However, they all pass when given enough resources and/or with `export FISH_TEST_MAX_CONCURRENCY=1`,
>   and having them enabled by default can help developers.
> - So as a workaround, maybe either
>   - set `export CI=1` when running tests
>   - and/or `export FISH_TEST_MAX_CONCURRENCY=1` at cmake-configure time
>   - or disable affected tests
>   until upstream fixes these.

Thanks Johannes for the info!

(Debian's BTS has its own quirks that replies are only forwarded to the 
Maintainer, so you have to include others you want receiving in To or Cc)

Santiago, Paul: Johannes is a core developer upstream 
(https://github.com/krobelus).

-- 
​    ,Sdrager
Blair Noctis

🇵🇸

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web