Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1283717 > unrolled thread

[PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

Started byIgor Mammedov <imammedo@redhat.com>
First post2015-12-04 12:30 +0100
Last post2015-12-04 15:40 +0100
Articles 6 — 3 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN Igor Mammedov <imammedo@redhat.com> - 2015-12-04 12:30 +0100
    Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory  regions above MAX_DMA32_PFN Ingo Molnar <mingo@kernel.org> - 2015-12-04 12:50 +0100
      Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory  regions above MAX_DMA32_PFN Igor Mammedov <imammedo@redhat.com> - 2015-12-04 13:20 +0100
      Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory  regions above MAX_DMA32_PFN Igor Mammedov <imammedo@redhat.com> - 2015-12-04 13:40 +0100
        Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory  regions above MAX_DMA32_PFN Paolo Bonzini <pbonzini@redhat.com> - 2015-12-04 13:40 +0100
          Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory  regions above MAX_DMA32_PFN Ingo Molnar <mingo@kernel.org> - 2015-12-04 15:40 +0100

#1283717 — [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

FromIgor Mammedov <imammedo@redhat.com>
Date2015-12-04 12:30 +0100
Subject[PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN
Message-ID<qBWIN-20f-9@gated-at.bofh.it>
when memory hotplug enabled system is booted with less
than 4GB of RAM and then later more RAM is hotplugged
32-bit devices stop functioning with following error:

 nommu_map_single: overflow 327b4f8c0+1522 of device mask ffffffff

the reason for this is that if x86_64 system were booted
with RAM less than 4GB, it doesn't enable SWIOTLB and
when memory is hotplugged beyond MAX_DMA32_PFN, devices
that expect 32-bit addresses can't handle 64-bit addresses.

Fix it by tracking max possible PFN when parsing
memory affinity structures from SRAT ACPI table and
enable SWIOTLB if there is hotpluggable memory
regions beyond MAX_DMA32_PFN.

It fixes KVM guests when they use emulated devices
(reproduces with ata_piix, e1000 and usb devices,
 RHBZ: 1275941, 1275977, 1271527)
It also fixes the HyperV, VMWare with emulated devices
which are affected by this issue as well.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 arch/x86/kernel/pci-swiotlb.c | 2 +-
 arch/x86/mm/srat.c            | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
index adf0392..7c577a1 100644
--- a/arch/x86/kernel/pci-swiotlb.c
+++ b/arch/x86/kernel/pci-swiotlb.c
@@ -88,7 +88,7 @@ int __init pci_swiotlb_detect_4gb(void)
 {
 	/* don't initialize swiotlb if iommu=off (no_iommu=1) */
 #ifdef CONFIG_X86_64
-	if (!no_iommu && max_pfn > MAX_DMA32_PFN)
+	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
 		swiotlb = 1;
 #endif
 	return swiotlb;
diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index c2aea63..a26bdbe 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -203,6 +203,9 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
 			(unsigned long long)start, (unsigned long long)end - 1);
 
+	if (max_possible_pfn < PFN_UP(end - 1))
+		max_possible_pfn = PFN_UP(end - 1);
+
 	return 0;
 out_err_bad_srat:
 	bad_srat();
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1283724 — Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

FromIngo Molnar <mingo@kernel.org>
Date2015-12-04 12:50 +0100
SubjectRe: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN
Message-ID<qBX2a-279-5@gated-at.bofh.it>
In reply to#1283717
* Igor Mammedov <imammedo@redhat.com> wrote:

> when memory hotplug enabled system is booted with less
> than 4GB of RAM and then later more RAM is hotplugged
> 32-bit devices stop functioning with following error:
> 
>  nommu_map_single: overflow 327b4f8c0+1522 of device mask ffffffff
> 
> the reason for this is that if x86_64 system were booted
> with RAM less than 4GB, it doesn't enable SWIOTLB and
> when memory is hotplugged beyond MAX_DMA32_PFN, devices
> that expect 32-bit addresses can't handle 64-bit addresses.
> 
> Fix it by tracking max possible PFN when parsing
> memory affinity structures from SRAT ACPI table and
> enable SWIOTLB if there is hotpluggable memory
> regions beyond MAX_DMA32_PFN.
> 
> It fixes KVM guests when they use emulated devices
> (reproduces with ata_piix, e1000 and usb devices,
>  RHBZ: 1275941, 1275977, 1271527)
> It also fixes the HyperV, VMWare with emulated devices
> which are affected by this issue as well.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  arch/x86/kernel/pci-swiotlb.c | 2 +-
>  arch/x86/mm/srat.c            | 3 +++
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
> index adf0392..7c577a1 100644
> --- a/arch/x86/kernel/pci-swiotlb.c
> +++ b/arch/x86/kernel/pci-swiotlb.c
> @@ -88,7 +88,7 @@ int __init pci_swiotlb_detect_4gb(void)
>  {
>  	/* don't initialize swiotlb if iommu=off (no_iommu=1) */
>  #ifdef CONFIG_X86_64
> -	if (!no_iommu && max_pfn > MAX_DMA32_PFN)
> +	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
>  		swiotlb = 1;

Ok, this series looks a lot better!

>  #endif
>  	return swiotlb;
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index c2aea63..a26bdbe 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -203,6 +203,9 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>  		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
>  			(unsigned long long)start, (unsigned long long)end - 1);
>  
> +	if (max_possible_pfn < PFN_UP(end - 1))
> +		max_possible_pfn = PFN_UP(end - 1);

Small nit, why not write this as something like:

	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));

?

I'd also move this second hunk to the first patch, because logically it belongs 
there (or into a third patch).

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283772 — Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

FromIgor Mammedov <imammedo@redhat.com>
Date2015-12-04 13:20 +0100
SubjectRe: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN
Message-ID<qBXvc-2xR-3@gated-at.bofh.it>
In reply to#1283724
On Fri, 4 Dec 2015 12:49:49 +0100
Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Igor Mammedov <imammedo@redhat.com> wrote:
> 
> > when memory hotplug enabled system is booted with less
> > than 4GB of RAM and then later more RAM is hotplugged
> > 32-bit devices stop functioning with following error:
> > 
> >  nommu_map_single: overflow 327b4f8c0+1522 of device mask ffffffff
> > 
> > the reason for this is that if x86_64 system were booted
> > with RAM less than 4GB, it doesn't enable SWIOTLB and
> > when memory is hotplugged beyond MAX_DMA32_PFN, devices
> > that expect 32-bit addresses can't handle 64-bit addresses.
> > 
> > Fix it by tracking max possible PFN when parsing
> > memory affinity structures from SRAT ACPI table and
> > enable SWIOTLB if there is hotpluggable memory
> > regions beyond MAX_DMA32_PFN.
> > 
> > It fixes KVM guests when they use emulated devices
> > (reproduces with ata_piix, e1000 and usb devices,
> >  RHBZ: 1275941, 1275977, 1271527)
> > It also fixes the HyperV, VMWare with emulated devices
> > which are affected by this issue as well.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  arch/x86/kernel/pci-swiotlb.c | 2 +-
> >  arch/x86/mm/srat.c            | 3 +++
> >  2 files changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
> > index adf0392..7c577a1 100644
> > --- a/arch/x86/kernel/pci-swiotlb.c
> > +++ b/arch/x86/kernel/pci-swiotlb.c
> > @@ -88,7 +88,7 @@ int __init pci_swiotlb_detect_4gb(void)
> >  {
> >  	/* don't initialize swiotlb if iommu=off (no_iommu=1) */
> >  #ifdef CONFIG_X86_64
> > -	if (!no_iommu && max_pfn > MAX_DMA32_PFN)
> > +	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
> >  		swiotlb = 1;
> 
> Ok, this series looks a lot better!
> 
> >  #endif
> >  	return swiotlb;
> > diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> > index c2aea63..a26bdbe 100644
> > --- a/arch/x86/mm/srat.c
> > +++ b/arch/x86/mm/srat.c
> > @@ -203,6 +203,9 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
> >  		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
> >  			(unsigned long long)start, (unsigned long long)end - 1);
> >  
> > +	if (max_possible_pfn < PFN_UP(end - 1))
> > +		max_possible_pfn = PFN_UP(end - 1);
> 
> Small nit, why not write this as something like:
> 
> 	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
> 
> ?
> 
> I'd also move this second hunk to the first patch, because logically it belongs 
> there (or into a third patch).
sure, I'll move it to the 1st patch

> 
> Thanks,
> 
> 	Ingo

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283783 — Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

FromIgor Mammedov <imammedo@redhat.com>
Date2015-12-04 13:40 +0100
SubjectRe: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN
Message-ID<qBXOy-2Eq-19@gated-at.bofh.it>
In reply to#1283724
On Fri, 4 Dec 2015 12:49:49 +0100
Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Igor Mammedov <imammedo@redhat.com> wrote:
> 
> > when memory hotplug enabled system is booted with less
> > than 4GB of RAM and then later more RAM is hotplugged
> > 32-bit devices stop functioning with following error:
> > 
> >  nommu_map_single: overflow 327b4f8c0+1522 of device mask ffffffff
> > 
> > the reason for this is that if x86_64 system were booted
> > with RAM less than 4GB, it doesn't enable SWIOTLB and
> > when memory is hotplugged beyond MAX_DMA32_PFN, devices
> > that expect 32-bit addresses can't handle 64-bit addresses.
> > 
> > Fix it by tracking max possible PFN when parsing
> > memory affinity structures from SRAT ACPI table and
> > enable SWIOTLB if there is hotpluggable memory
> > regions beyond MAX_DMA32_PFN.
> > 
> > It fixes KVM guests when they use emulated devices
> > (reproduces with ata_piix, e1000 and usb devices,
> >  RHBZ: 1275941, 1275977, 1271527)
> > It also fixes the HyperV, VMWare with emulated devices
> > which are affected by this issue as well.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  arch/x86/kernel/pci-swiotlb.c | 2 +-
> >  arch/x86/mm/srat.c            | 3 +++
> >  2 files changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c
> > index adf0392..7c577a1 100644
> > --- a/arch/x86/kernel/pci-swiotlb.c
> > +++ b/arch/x86/kernel/pci-swiotlb.c
> > @@ -88,7 +88,7 @@ int __init pci_swiotlb_detect_4gb(void)
> >  {
> >  	/* don't initialize swiotlb if iommu=off (no_iommu=1) */
> >  #ifdef CONFIG_X86_64
> > -	if (!no_iommu && max_pfn > MAX_DMA32_PFN)
> > +	if (!no_iommu && max_possible_pfn > MAX_DMA32_PFN)
> >  		swiotlb = 1;
> 
> Ok, this series looks a lot better!
> 
> >  #endif
> >  	return swiotlb;
> > diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> > index c2aea63..a26bdbe 100644
> > --- a/arch/x86/mm/srat.c
> > +++ b/arch/x86/mm/srat.c
> > @@ -203,6 +203,9 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
> >  		pr_warn("SRAT: Failed to mark hotplug range [mem %#010Lx-%#010Lx] in memblock\n",
> >  			(unsigned long long)start, (unsigned long long)end - 1);
> >  
> > +	if (max_possible_pfn < PFN_UP(end - 1))
> > +		max_possible_pfn = PFN_UP(end - 1);
> 
> Small nit, why not write this as something like:
> 
> 	max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
That doesn't pass strict type check:

include/linux/kernel.h:730:17: warning: comparison of distinct pointer types lacks a cast [enabled by default]
  (void) (&_max1 == &_max2);  \
                 ^
arch/x86/mm/srat.c:206:21: note: in expansion of macro ‘max’
  max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));

I can change max_possible_pfn to u64 to match PFN_UP(end - 1) type.

> 
> ?
> 
> I'd also move this second hunk to the first patch, because logically it belongs 
> there (or into a third patch).
> 
> Thanks,
> 
> 	Ingo

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283786 — Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

FromPaolo Bonzini <pbonzini@redhat.com>
Date2015-12-04 13:40 +0100
SubjectRe: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN
Message-ID<qBXOy-2Eq-25@gated-at.bofh.it>
In reply to#1283783

On 04/12/2015 13:33, Igor Mammedov wrote:
> That doesn't pass strict type check:
> 
> include/linux/kernel.h:730:17: warning: comparison of distinct pointer types lacks a cast [enabled by default]
>   (void) (&_max1 == &_max2);  \
>                  ^
> arch/x86/mm/srat.c:206:21: note: in expansion of macro ‘max’
>   max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
> 
> I can change max_possible_pfn to u64 to match PFN_UP(end - 1) type.

Sounds like a good idea anyway.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1283874 — Re: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN

FromIngo Molnar <mingo@kernel.org>
Date2015-12-04 15:40 +0100
SubjectRe: [PATCH v2 2/2] x86_64: enable SWIOTLB if system has SRAT memory regions above MAX_DMA32_PFN
Message-ID<qBZGG-3Ud-27@gated-at.bofh.it>
In reply to#1283786
* Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 04/12/2015 13:33, Igor Mammedov wrote:
> > That doesn't pass strict type check:
> > 
> > include/linux/kernel.h:730:17: warning: comparison of distinct pointer types lacks a cast [enabled by default]
> >   (void) (&_max1 == &_max2);  \
> >                  ^
> > arch/x86/mm/srat.c:206:21: note: in expansion of macro ‘max’
> >   max_possible_pfn = max(max_possible_pfn, PFN_UP(end - 1));
> > 
> > I can change max_possible_pfn to u64 to match PFN_UP(end - 1) type.
> 
> Sounds like a good idea anyway.

Yeah, agreed.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web