Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.debian.kernel > #83697 > unrolled thread

Bug#1079684: Network errors on VM

Started byDavid Prévot <taffit@debian.org>
First post2024-08-26 11:00 +0200
Last post2024-10-12 19:10 +0200
Articles 15 — 7 participants

Back to article view | Back to linux.debian.kernel


Contents

  Bug#1079684: Network errors on VM David Prévot <taffit@debian.org> - 2024-08-26 11:00 +0200
    Bug#1079684: Network errors on VM David Prévot <taffit@debian.org> - 2024-08-26 15:50 +0200
    Bug#1079684: still issues in linux 6.1.106-3 "Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch> - 2024-09-16 14:00 +0200
      Bug#1079684: still issues in linux 6.1.106-3 David Prévot <taffit@debian.org> - 2024-09-18 08:40 +0200
    Bug#1082081: Bug#1079684: still issues in linux 6.1.106-3 Salvatore Bonaccorso <carnil@debian.org> - 2024-09-18 10:30 +0200
      Bug#1082081: still issues in linux 6.1.106-3 "Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch> - 2024-09-18 14:20 +0200
        Bug#1082081: still issues in linux 6.1.106-3 Salvatore Bonaccorso <carnil@debian.org> - 2024-09-18 16:10 +0200
          Bug#1082081: still issues in linux 6.1.106-3 Florian La Roche <florian.laroche@googlemail.com> - 2024-09-21 09:00 +0200
    Bug#1082081: Problem actually unreproducible for us David Prevot <dprevot+debian@evolix.fr> - 2024-09-25 15:50 +0200
      Bug#1082081: Problem actually unreproducible for us "Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch> - 2024-09-25 23:20 +0200
        Bug#1082081: Problem actually unreproducible for us Salvatore Bonaccorso <carnil@debian.org> - 2024-10-04 22:30 +0200
          Bug#1082081: Problem actually unreproducible for us "Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch> - 2024-10-08 10:10 +0200
            Bug#1082081: Problem actually unreproducible for us Salvatore Bonaccorso <carnil@debian.org> - 2024-10-12 19:10 +0200
          Bug#1082081: Problem actually unreproducible for us Markus <markus@opsone.ch> - 2024-10-08 11:00 +0200
    Bug#1082081: marked as done (Performance issues on VM (virtio)) "Debian Bug Tracking System" <owner@bugs.debian.org> - 2024-10-12 19:10 +0200

#83697 — Bug#1079684: Network errors on VM

FromDavid Prévot <taffit@debian.org>
Date2024-08-26 11:00 +0200
SubjectBug#1079684: Network errors on VM
Message-ID<JfDJT-7IS1-1@gated-at.bofh.it>

[Multipart message — attachments visible in raw view] — view raw

Package: linux-image-6.1.0-24-cloud-amd64
Version: 6.1.106-2
Severity: serious
Tags: upstream
Forwarded: https://bugzilla.kernel.org/show_bug.cgi?id=219129

After the latest linux upgrade, the following message appear frequently
in /var/log/syslog:

	ens3: bad gso: type: 4, size: 1460

It seems real network issues have been noticed in some VMs we rebooted
on friday (they are now running the previous kernel).

The forwarded upstream issue seems to match.

Regards

taffit

[toc] | [next] | [standalone]


#83699

FromDavid Prévot <taffit@debian.org>
Date2024-08-26 15:50 +0200
Message-ID<JfIgy-7LYw-5@gated-at.bofh.it>
In reply to#83697

[Multipart message — attachments visible in raw view] — view raw

Hi,

Thanks Salvatore for the quick follow up on IRC and for pointing me at
the following doc in order to quickly rebuild the kernel with the
proposed patches.

https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4

Le Mon, Aug 26, 2024 at 10:54:41AM +0200, David Prévot a écrit :
[…]
> Forwarded: https://bugzilla.kernel.org/show_bug.cgi?id=219129
[…]
> It seems real network issues have been noticed in some VMs we rebooted
> on friday (they are now running the previous kernel).

There are indeed severe consequences in network performances: e.g., it
took me two minutes instead of a fraction of second to upload the
patched kernel to the server (~500kB/s instead of 62MB/s).

> The forwarded upstream issue seems to match.

I just rebuilt the kernel with the three following patches and can’t
reproduce the issue once booted on the patched version.

>   89add40066f9 ("net: drop bad gso csum_start and offset in virtio_net_hdr")
>   fc8b2a619469 ("net: more strict VIRTIO_NET_HDR_GSO_UDP_L4 validation")
>   9840036786d9 ("gso: fix dodgy bit handling for GSO_UDP_L4")

The first patch (in reverse order) didn’t apply cleanly, so I only used
the first diff (out of three):


Index: linux-6.1.106/net/ipv4/udp_offload.c
===================================================================
--- linux-6.1.106.orig/net/ipv4/udp_offload.c	2024-08-26 14:01:13.869618434 +0200
+++ linux-6.1.106/net/ipv4/udp_offload.c	2024-08-26 14:01:13.869618434 +0200
@@ -273,13 +273,20 @@
 	__sum16 check;
 	__be16 newlen;

-	if (skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST)
-		return __udp_gso_segment_list(gso_skb, features, is_ipv6);
-
 	mss = skb_shinfo(gso_skb)->gso_size;
 	if (gso_skb->len <= sizeof(*uh) + mss)
 		return ERR_PTR(-EINVAL);

+	if (skb_gso_ok(gso_skb, features | NETIF_F_GSO_ROBUST)) {
+		/* Packet is from an untrusted source, reset gso_segs. */
+		skb_shinfo(gso_skb)->gso_segs = DIV_ROUND_UP(gso_skb->len - sizeof(*uh),
+							     mss);
+		return NULL;
+	}
+
+	if (skb_shinfo(gso_skb)->gso_type & SKB_GSO_FRAGLIST)
+		return __udp_gso_segment_list(gso_skb, features, is_ipv6);
+
 	skb_pull(gso_skb, sizeof(*uh));

 	/* clear destructor to avoid skb_segment assigning it to tail */


Regards

taffit

[toc] | [prev] | [next] | [standalone]


#84012 — Bug#1079684: still issues in linux 6.1.106-3

From"Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch>
Date2024-09-16 14:00 +0200
SubjectBug#1079684: still issues in linux 6.1.106-3
Message-ID<JniyB-cGAw-7@gated-at.bofh.it>
In reply to#83697
Hi,

We still see issues with linux/6.1.106-3 in combination with virtio.

Network performance is down to ~4 Mb/s measured with iperf3 instead of 
~8 Gb/s (with linux/6.1.99-1 respectively 6.1.0-23-amd64). It is 
reproducible and booting the previous kernel resolves the issue.

Kind regards

-- 
Emanuel Kocher
Bedag Informatik AG
Senior System Engineer / SYX1
Mobile: 079 898 70 98
www.bedag.ch

[toc] | [prev] | [next] | [standalone]


#84014 — Bug#1079684: still issues in linux 6.1.106-3

FromDavid Prévot <taffit@debian.org>
Date2024-09-18 08:40 +0200
SubjectBug#1079684: still issues in linux 6.1.106-3
Message-ID<JnWw1-d696-1@gated-at.bofh.it>
In reply to#84012

[Multipart message — attachments visible in raw view] — view raw

Control: clone -1 -2
Control: reopen -2
Control: found -2 linux/6.1.106-3
Control: retitle -2 Performance issues on VM (virtio)

Hi,

Le Mon, Sep 16, 2024 at 11:37:07AM +0000, Kocher Emanuel, Bedag a écrit :
[…]
> We still see issues with linux/6.1.106-3 in combination with virtio.

Thanks for the feedback.

> Network performance is down to ~4 Mb/s measured with iperf3 instead of 
> ~8 Gb/s (with linux/6.1.99-1 respectively 6.1.0-23-amd64). It is 
> reproducible and booting the previous kernel resolves the issue.

Our network team also noticed performance issues, roughly 20 to 30%
increase speed on packages download once we reverted to 6.1.0-23-amd64
on a mirror we host. Not sure it’s exactly the same issue as the one
initially reported, hence the clone, but I agree there is still an issue
with the current stable kernel.

Regards,

[toc] | [prev] | [next] | [standalone]


#84015 — Bug#1082081: Bug#1079684: still issues in linux 6.1.106-3

FromSalvatore Bonaccorso <carnil@debian.org>
Date2024-09-18 10:30 +0200
SubjectBug#1082081: Bug#1079684: still issues in linux 6.1.106-3
Message-ID<JnYeu-d7mH-11@gated-at.bofh.it>
In reply to#83697
Resending to the correct bug (#1082081)

On Wed, Sep 18, 2024 at 10:19:28AM +0200, Salvatore Bonaccorso wrote:
> Hi David, hi Emanuel,
> 
> On Wed, Sep 18, 2024 at 08:29:54AM +0200, David Prévot wrote:
> > Control: clone -1 -2
> > Control: reopen -2
> > Control: found -2 linux/6.1.106-3
> > Control: retitle -2 Performance issues on VM (virtio)
> > 
> > Hi,
> > 
> > Le Mon, Sep 16, 2024 at 11:37:07AM +0000, Kocher Emanuel, Bedag a écrit :
> > […]
> > > We still see issues with linux/6.1.106-3 in combination with virtio.
> > 
> > Thanks for the feedback.
> > 
> > > Network performance is down to ~4 Mb/s measured with iperf3 instead of 
> > > ~8 Gb/s (with linux/6.1.99-1 respectively 6.1.0-23-amd64). It is 
> > > reproducible and booting the previous kernel resolves the issue.
> > 
> > Our network team also noticed performance issues, roughly 20 to 30%
> > increase speed on packages download once we reverted to 6.1.0-23-amd64
> > on a mirror we host. Not sure it’s exactly the same issue as the one
> > initially reported, hence the clone, but I agree there is still an issue
> > with the current stable kernel.
> 
> Thanks for reporting back. I will try to reproduce it so we can test
> against newer 6.1.y kernels as well. Do you have reproducers available
> to show it?
> 
> Regards,
> Salvatore

[toc] | [prev] | [next] | [standalone]


#84017 — Bug#1082081: still issues in linux 6.1.106-3

From"Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch>
Date2024-09-18 14:20 +0200
SubjectBug#1082081: still issues in linux 6.1.106-3
Message-ID<Jo1P3-d9Nj-1@gated-at.bofh.it>
In reply to#84015
Hi Salvatore

On 18/09/2024 10:21, Salvatore Bonaccorso wrote:
> Resending to the correct bug (#1082081)
>
> On Wed, Sep 18, 2024 at 10:19:28AM +0200, Salvatore Bonaccorso wrote:
>> Hi David, hi Emanuel,
>>
>> On Wed, Sep 18, 2024 at 08:29:54AM +0200, David Prévot wrote:
>>> Control: clone -1 -2
>>> Control: reopen -2
>>> Control: found -2 linux/6.1.106-3
>>> Control: retitle -2 Performance issues on VM (virtio)
>>>
>>> Hi,
>>>
>>> Le Mon, Sep 16, 2024 at 11:37:07AM +0000, Kocher Emanuel, Bedag a écrit :
>>> […]
>>>> We still see issues with linux/6.1.106-3 in combination with virtio.
>>> Thanks for the feedback.
>>>
>>>> Network performance is down to ~4 Mb/s measured with iperf3 instead of
>>>> ~8 Gb/s (with linux/6.1.99-1 respectively 6.1.0-23-amd64). It is
>>>> reproducible and booting the previous kernel resolves the issue.
>>> Our network team also noticed performance issues, roughly 20 to 30%
>>> increase speed on packages download once we reverted to 6.1.0-23-amd64
>>> on a mirror we host. Not sure it’s exactly the same issue as the one
>>> initially reported, hence the clone, but I agree there is still an issue
>>> with the current stable kernel.
>> Thanks for reporting back. I will try to reproduce it so we can test
>> against newer 6.1.y kernels as well. Do you have reproducers available
>> to show it?

Yes, we are able to reproduce it on Nutanix hypervisors (virtio 
modules). There are different effects, e.g. copying a 1G file from an 
NFS-Share to such a server results in input/output errors (after approx 
50s) and the NFS share going stale for a while, whereas on a healthy 
server the process takes approx. 6s.

We see a lot of kernel messages like the one below (size varying between 
1368, 1380, 1398 and 1448):

     eth0: bad gso: type: 1, size: 1448

Tell me if you want me to test something.

Regards,
Emanuel

[toc] | [prev] | [next] | [standalone]


#84021 — Bug#1082081: still issues in linux 6.1.106-3

FromSalvatore Bonaccorso <carnil@debian.org>
Date2024-09-18 16:10 +0200
SubjectBug#1082081: still issues in linux 6.1.106-3
Message-ID<Jo3xv-db0t-1@gated-at.bofh.it>
In reply to#84017
Hi,

On Wed, Sep 18, 2024 at 12:01:01PM +0000, Kocher Emanuel, Bedag wrote:
> Hi Salvatore
> 
> On 18/09/2024 10:21, Salvatore Bonaccorso wrote:
> > Resending to the correct bug (#1082081)
> >
> > On Wed, Sep 18, 2024 at 10:19:28AM +0200, Salvatore Bonaccorso wrote:
> >> Hi David, hi Emanuel,
> >>
> >> On Wed, Sep 18, 2024 at 08:29:54AM +0200, David Prévot wrote:
> >>> Control: clone -1 -2
> >>> Control: reopen -2
> >>> Control: found -2 linux/6.1.106-3
> >>> Control: retitle -2 Performance issues on VM (virtio)
> >>>
> >>> Hi,
> >>>
> >>> Le Mon, Sep 16, 2024 at 11:37:07AM +0000, Kocher Emanuel, Bedag a écrit :
> >>> […]
> >>>> We still see issues with linux/6.1.106-3 in combination with virtio.
> >>> Thanks for the feedback.
> >>>
> >>>> Network performance is down to ~4 Mb/s measured with iperf3 instead of
> >>>> ~8 Gb/s (with linux/6.1.99-1 respectively 6.1.0-23-amd64). It is
> >>>> reproducible and booting the previous kernel resolves the issue.
> >>> Our network team also noticed performance issues, roughly 20 to 30%
> >>> increase speed on packages download once we reverted to 6.1.0-23-amd64
> >>> on a mirror we host. Not sure it’s exactly the same issue as the one
> >>> initially reported, hence the clone, but I agree there is still an issue
> >>> with the current stable kernel.
> >> Thanks for reporting back. I will try to reproduce it so we can test
> >> against newer 6.1.y kernels as well. Do you have reproducers available
> >> to show it?
> 
> Yes, we are able to reproduce it on Nutanix hypervisors (virtio 
> modules). There are different effects, e.g. copying a 1G file from an 
> NFS-Share to such a server results in input/output errors (after approx 
> 50s) and the NFS share going stale for a while, whereas on a healthy 
> server the process takes approx. 6s.
> 
> We see a lot of kernel messages like the one below (size varying between 
> 1368, 1380, 1398 and 1448):
> 
>      eth0: bad gso: type: 1, size: 1448
> 
> Tell me if you want me to test something.

Ah Nutanix, well that's not easily available to get ;-). Anyway, I
noticed there is the following commit queued for the next stable
series:

https://lore.kernel.org/all/20240910213553.839926-1-willemdebruijn.kernel@gmail.com/

I'm not yet sure and have to find a way to reproduce your both issues,
but in case you might check if applying the patch fixes your issue?

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]


#84056 — Bug#1082081: still issues in linux 6.1.106-3

FromFlorian La Roche <florian.laroche@googlemail.com>
Date2024-09-21 09:00 +0200
SubjectBug#1082081: still issues in linux 6.1.106-3
Message-ID<Jp2g1-dMtP-3@gated-at.bofh.it>
In reply to#84021
This patch is now included in 6.1.111 (commit
34aaedb052cb4cb068cb19a34cf6d9cedb9af87f in 6.1.11),
so the next kernel rebase to 6.1.x should also hopefully fix this issue.
I can also see quite a few virtio changes in 6.1.107.

best regards,

Florian La Roche

[toc] | [prev] | [next] | [standalone]


#84101 — Bug#1082081: Problem actually unreproducible for us

FromDavid Prevot <dprevot+debian@evolix.fr>
Date2024-09-25 15:50 +0200
SubjectBug#1082081: Problem actually unreproducible for us
Message-ID<JqAyZ-eK9v-1@gated-at.bofh.it>
In reply to#83697
Hi,

After further investigations, I now believe we’re not able to reproduce
the issue (the performance issues were actually random, and have other
causes that we’re currently investigating).

Emanuel, have you been able to test a kernel including the patch
previously pointed by Salvatore and Florian?

34aaedb052cb4cb068cb19a34cf6d9cedb9af87f
https://lore.kernel.org/all/20240910213553.839926-1-willemdebruijn.kernel@gmail.com/

I’ve built it last week following the team documentation.

https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4

It’s available at the following URL if needed.

https://download.evolix.net/b3e804f0aaf78b4dc217213942c20805079fd160/linux-image-6.1.0-0.a.test-amd64-unsigned_6.1.106-3a~test_amd64.deb

Regards,
-- 
David Prévot
Marseille (37 rue Guibal, Pôle Média, 13003) / Paris / Montréal
http://evolix.com | Twitter: @Evolix @EvolixNOC | http://blog.evolix.com

[toc] | [prev] | [next] | [standalone]


#84113 — Bug#1082081: Problem actually unreproducible for us

From"Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch>
Date2024-09-25 23:20 +0200
SubjectBug#1082081: Problem actually unreproducible for us
Message-ID<JqHAt-eOwp-11@gated-at.bofh.it>
In reply to#84101

[Multipart message — attachments visible in raw view] — view raw

Hi David,

On 25/09/2024 15:35, David Prevot wrote:

Hi,

After further investigations, I now believe we’re not able to reproduce
the issue (the performance issues were actually random, and have other
causes that we’re currently investigating).

Emanuel, have you been able to test a kernel including the patch
previously pointed by Salvatore and Florian?

34aaedb052cb4cb068cb19a34cf6d9cedb9af87f
https://lore.kernel.org/all/20240910213553.839926-1-willemdebruijn.kernel@gmail.com/

I’ve built it last week following the team documentation.

https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4

It’s available at the following URL if needed.

https://download.evolix.net/b3e804f0aaf78b4dc217213942c20805079fd160/linux-image-6.1.0-0.a.test-amd64-unsigned_6.1.106-3a~test_amd64.deb


This is unfortunate to hear - we actually still have the issue with the patched kernel and can't see any improvements at all with speed nor reliability.

Even a simple copy of the linux kernel from a nfs share to the homedir will interrupt after a while with an "Input/output error".

Turning GSO off did not help as some suggested:
echo options virtio_net gso=N >> /etc/modprobe.d/virtio.conf
rmmod virtio_net && modprobe virtio_net

As for turning LSO off on the hypervisor I would have to get a test host - it seems to have helped here: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/1131

If there's something we can do in order of testing we should be able to do so.

Regards,
Emanuel

[toc] | [prev] | [next] | [standalone]


#84190 — Bug#1082081: Problem actually unreproducible for us

FromSalvatore Bonaccorso <carnil@debian.org>
Date2024-10-04 22:30 +0200
SubjectBug#1082081: Problem actually unreproducible for us
Message-ID<JtX61-gWDc-1@gated-at.bofh.it>
In reply to#84113
Hi Emanuel,

On Wed, Sep 25, 2024 at 08:59:00PM +0000, Kocher Emanuel, Bedag wrote:
> Hi David,
> 
> On 25/09/2024 15:35, David Prevot wrote:
> 
> Hi,
> 
> After further investigations, I now believe we’re not able to reproduce
> the issue (the performance issues were actually random, and have other
> causes that we’re currently investigating).
> 
> Emanuel, have you been able to test a kernel including the patch
> previously pointed by Salvatore and Florian?
> 
> 34aaedb052cb4cb068cb19a34cf6d9cedb9af87f
> https://lore.kernel.org/all/20240910213553.839926-1-willemdebruijn.kernel@gmail.com/
> 
> I’ve built it last week following the team documentation.
> 
> https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4
> 
> It’s available at the following URL if needed.
> 
> https://download.evolix.net/b3e804f0aaf78b4dc217213942c20805079fd160/linux-image-6.1.0-0.a.test-amd64-unsigned_6.1.106-3a~test_amd64.deb
> 
> 
> This is unfortunate to hear - we actually still have the issue with the patched kernel and can't see any improvements at all with speed nor reliability.
> 
> Even a simple copy of the linux kernel from a nfs share to the homedir will interrupt after a while with an "Input/output error".
> 
> Turning GSO off did not help as some suggested:
> echo options virtio_net gso=N >> /etc/modprobe.d/virtio.conf
> rmmod virtio_net && modprobe virtio_net
> 
> As for turning LSO off on the hypervisor I would have to get a test host - it seems to have helped here: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/1131
> 
> If there's something we can do in order of testing we should be able to do so.

Since you still are affected by the issue: Can you please the newes
6.1.112-1 which was released as DSA 5782-1?

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]


#84241 — Bug#1082081: Problem actually unreproducible for us

From"Kocher Emanuel, Bedag" <emanuel.kocher@bedag.ch>
Date2024-10-08 10:10 +0200
SubjectBug#1082081: Problem actually unreproducible for us
Message-ID<Jvds5-99t-9@gated-at.bofh.it>
In reply to#84190
Hi Salvatore,

On 04/10/2024 22:24, Salvatore Bonaccorso wrote:
> Hi Emanuel,
>
> On Wed, Sep 25, 2024 at 08:59:00PM +0000, Kocher Emanuel, Bedag wrote:
>> Hi David,
>>
>> On 25/09/2024 15:35, David Prevot wrote:
>>
>> Hi,
>>
>> After further investigations, I now believe we’re not able to reproduce
>> the issue (the performance issues were actually random, and have other
>> causes that we’re currently investigating).
>>
>> Emanuel, have you been able to test a kernel including the patch
>> previously pointed by Salvatore and Florian?
>>
>> 34aaedb052cb4cb068cb19a34cf6d9cedb9af87f
>> https://lore.kernel.org/all/20240910213553.839926-1-willemdebruijn.kernel@gmail.com/
>>
>> I’ve built it last week following the team documentation.
>>
>> https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4
>>
>> It’s available at the following URL if needed.
>>
>> https://download.evolix.net/b3e804f0aaf78b4dc217213942c20805079fd160/linux-image-6.1.0-0.a.test-amd64-unsigned_6.1.106-3a~test_amd64.deb
>>
>>
>> This is unfortunate to hear - we actually still have the issue with the patched kernel and can't see any improvements at all with speed nor reliability.
>>
>> Even a simple copy of the linux kernel from a nfs share to the homedir will interrupt after a while with an "Input/output error".
>>
>> Turning GSO off did not help as some suggested:
>> echo options virtio_net gso=N >> /etc/modprobe.d/virtio.conf
>> rmmod virtio_net && modprobe virtio_net
>>
>> As for turning LSO off on the hypervisor I would have to get a test host - it seems to have helped here: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/1131
>>
>> If there's something we can do in order of testing we should be able to do so.
> Since you still are affected by the issue: Can you please the newes
> 6.1.112-1 which was released as DSA 5782-1?

I can confirm that 6.1.112-1 resolves the issue and the network 
performance is back to normal.

Tested on different virtual machines and upgrading the rest now, thank 
you and the team!

Regards,
Emanuel

[toc] | [prev] | [next] | [standalone]


#84273 — Bug#1082081: Problem actually unreproducible for us

FromSalvatore Bonaccorso <carnil@debian.org>
Date2024-10-12 19:10 +0200
SubjectBug#1082081: Problem actually unreproducible for us
Message-ID<JwNMR-191I-1@gated-at.bofh.it>
In reply to#84241
Source: linux
Source-Version: 6.1.112-1

On Tue, Oct 08, 2024 at 08:00:27AM +0000, Kocher Emanuel, Bedag wrote:
> Hi Salvatore,
> 
> On 04/10/2024 22:24, Salvatore Bonaccorso wrote:
> > Hi Emanuel,
> >
> > On Wed, Sep 25, 2024 at 08:59:00PM +0000, Kocher Emanuel, Bedag wrote:
> >> Hi David,
> >>
> >> On 25/09/2024 15:35, David Prevot wrote:
> >>
> >> Hi,
> >>
> >> After further investigations, I now believe we’re not able to reproduce
> >> the issue (the performance issues were actually random, and have other
> >> causes that we’re currently investigating).
> >>
> >> Emanuel, have you been able to test a kernel including the patch
> >> previously pointed by Salvatore and Florian?
> >>
> >> 34aaedb052cb4cb068cb19a34cf6d9cedb9af87f
> >> https://lore.kernel.org/all/20240910213553.839926-1-willemdebruijn.kernel@gmail.com/
> >>
> >> I’ve built it last week following the team documentation.
> >>
> >> https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#id-1.6.6.4
> >>
> >> It’s available at the following URL if needed.
> >>
> >> https://download.evolix.net/b3e804f0aaf78b4dc217213942c20805079fd160/linux-image-6.1.0-0.a.test-amd64-unsigned_6.1.106-3a~test_amd64.deb
> >>
> >>
> >> This is unfortunate to hear - we actually still have the issue with the patched kernel and can't see any improvements at all with speed nor reliability.
> >>
> >> Even a simple copy of the linux kernel from a nfs share to the homedir will interrupt after a while with an "Input/output error".
> >>
> >> Turning GSO off did not help as some suggested:
> >> echo options virtio_net gso=N >> /etc/modprobe.d/virtio.conf
> >> rmmod virtio_net && modprobe virtio_net
> >>
> >> As for turning LSO off on the hypervisor I would have to get a test host - it seems to have helped here: https://github.com/virtio-win/kvm-guest-drivers-windows/issues/1131
> >>
> >> If there's something we can do in order of testing we should be able to do so.
> > Since you still are affected by the issue: Can you please the newes
> > 6.1.112-1 which was released as DSA 5782-1?
> 
> I can confirm that 6.1.112-1 resolves the issue and the network 
> performance is back to normal.
> 
> Tested on different virtual machines and upgrading the rest now, thank 
> you and the team!

Many thanks for the confirmation! Closing the bug with the version
now.

Regards,
Salvatore

[toc] | [prev] | [next] | [standalone]


#84242 — Bug#1082081: Problem actually unreproducible for us

FromMarkus <markus@opsone.ch>
Date2024-10-08 11:00 +0200
SubjectBug#1082081: Problem actually unreproducible for us
Message-ID<Jveet-9s2-5@gated-at.bofh.it>
In reply to#84190
I had the same problem. Kernel 6.1.112-1 resolved the problem for me.

Regards,
Markus

[toc] | [prev] | [next] | [standalone]


#84274 — Bug#1082081: marked as done (Performance issues on VM (virtio))

From"Debian Bug Tracking System" <owner@bugs.debian.org>
Date2024-10-12 19:10 +0200
SubjectBug#1082081: marked as done (Performance issues on VM (virtio))
Message-ID<JwNMR-191I-9@gated-at.bofh.it>
In reply to#83697

[Multipart message — attachments visible in raw view] — view raw

Your message dated Sat, 12 Oct 2024 19:00:19 +0200
with message-id <ZwqrI_dPsUf3Sg-R@eldamar.lan>
and subject line Re: Bug#1082081: Problem actually unreproducible for us
has caused the Debian Bug report #1082081,
regarding Performance issues on VM (virtio)
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
1082081: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1082081
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [standalone]


Back to top | Article view | linux.debian.kernel


csiph-web