Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1397234 > unrolled thread

[PATCH 0/3] memory-hotplug: improve rezoning capability

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/3] memory-hotplug: improve rezoning capability Reza Arbab <arbab@linux.vnet.ibm.com> - 2016-05-09 20:00 +0200
    [PATCH 2/3] memory-hotplug: more general validation of zone during online Reza Arbab <arbab@linux.vnet.ibm.com> - 2016-05-09 20:00 +0200
      Re: [PATCH 2/3] memory-hotplug: more general validation of zone  during online Yasuaki Ishimatsu <yasu.isimatu@gmail.com> - 2016-05-10 20:10 +0200
        Re: [PATCH 2/3] memory-hotplug: more general validation of zone  during online Reza Arbab <arbab@linux.vnet.ibm.com> - 2016-05-10 22:40 +0200
          Re: [PATCH 2/3] memory-hotplug: more general validation of zone  during online Yasuaki Ishimatsu <yasu.isimatu@gmail.com> - 2016-05-11 17:20 +0200
            Re: [PATCH 2/3] memory-hotplug: more general validation of zone  during online Reza Arbab <arbab@linux.vnet.ibm.com> - 2016-05-11 21:30 +0200
    Re: [PATCH 0/3] memory-hotplug: improve rezoning capability Andrew Morton <akpm@linux-foundation.org> - 2016-05-09 22:20 +0200
      Re: [PATCH 0/3] memory-hotplug: improve rezoning capability Andrew Morton <akpm@linux-foundation.org> - 2016-05-09 22:20 +0200

#1397234 — [PATCH 0/3] memory-hotplug: improve rezoning capability

While it is currently possible to rezone memory when it is onlined, there are
implicit assumptions about the zones:

* To "online_kernel" a block into ZONE_NORMAL, it must currently
  be in ZONE_MOVABLE.

* To "online_movable" a block into ZONE_MOVABLE, it must currently
  be in (ZONE_MOVABLE - 1).

So on powerpc, where new memory is hotplugged into ZONE_DMA, these operations
do not work.

This patchset replaces the qualifications above with a more general
validation of zone movement.

Reza Arbab (3):
  memory-hotplug: add move_pfn_range()
  memory-hotplug: more general validation of zone during online
  memory-hotplug: use zone_can_shift() for sysfs valid_zones attribute

 drivers/base/memory.c          | 28 ++++++++++-------
 include/linux/memory_hotplug.h |  2 ++
 mm/memory_hotplug.c            | 70 ++++++++++++++++++++++++++++++++++--------
 3 files changed, 77 insertions(+), 23 deletions(-)

-- 
1.8.3.1

[toc] | [next] | [standalone]

#1397236 — [PATCH 2/3] memory-hotplug: more general validation of zone during online

When memory is onlined, we are only able to rezone from ZONE_MOVABLE to
ZONE_KERNEL, or from (ZONE_MOVABLE - 1) to ZONE_MOVABLE.

To be more flexible, use the following criteria instead; to online memory
from zone X into zone Y,

* Any zones between X and Y must be unused.
* If X is lower than Y, the onlined memory must lie at the end of X.
* If X is higher than Y, the onlined memory must lie at the start of X.

Add zone_can_shift() to make this determination.

Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
---
 include/linux/memory_hotplug.h |  2 ++
 mm/memory_hotplug.c            | 42 +++++++++++++++++++++++++++++++++++-------
 2 files changed, 37 insertions(+), 7 deletions(-)

diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index adbef58..7bff0f9 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -284,5 +284,7 @@ extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms,
 		unsigned long map_offset);
 extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
 					  unsigned long pnum);
+extern int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
+			  enum zone_type target);
 
 #endif /* __LINUX_MEMORY_HOTPLUG_H */
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 6b4b005..b63cc28 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1032,6 +1032,37 @@ static void node_states_set_node(int node, struct memory_notify *arg)
 	node_set_state(node, N_MEMORY);
 }
 
+int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
+		   enum zone_type target)
+{
+	struct zone *zone = page_zone(pfn_to_page(pfn));
+	enum zone_type idx = zone_idx(zone);
+	int i;
+
+	if (idx < target) {
+		/* pages must be at end of current zone */
+		if (pfn + nr_pages != zone_end_pfn(zone))
+			return 0;
+
+		/* no zones in use between current zone and target */
+		for (i = idx + 1; i < target; i++)
+			if (zone_is_initialized(zone - idx + i))
+				return 0;
+	}
+
+	if (target < idx) {
+		/* pages must be at beginning of current zone */
+		if (pfn != zone->zone_start_pfn)
+			return 0;
+
+		/* no zones in use between current zone and target */
+		for (i = target + 1; i < idx; i++)
+			if (zone_is_initialized(zone - idx + i))
+				return 0;
+	}
+
+	return target - idx;
+}
 
 /* Must be protected by mem_hotplug_begin() */
 int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_type)
@@ -1057,13 +1088,10 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ
 	    !can_online_high_movable(zone))
 		return -EINVAL;
 
-	if (online_type == MMOP_ONLINE_KERNEL &&
-	    zone_idx(zone) == ZONE_MOVABLE)
-		zone_shift = -1;
-
-	if (online_type == MMOP_ONLINE_MOVABLE &&
-	    zone_idx(zone) == ZONE_MOVABLE - 1)
-		zone_shift = 1;
+	if (online_type == MMOP_ONLINE_KERNEL)
+		zone_shift = zone_can_shift(pfn, nr_pages, ZONE_NORMAL);
+	else if (online_type == MMOP_ONLINE_MOVABLE)
+		zone_shift = zone_can_shift(pfn, nr_pages, ZONE_MOVABLE);
 
 	zone = move_pfn_range(zone_shift, pfn, pfn + nr_pages);
 	if (!zone)
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]

#1398374 — Re: [PATCH 2/3] memory-hotplug: more general validation of zone during online

On Mon,  9 May 2016 12:53:38 -0500
Reza Arbab <arbab@linux.vnet.ibm.com> wrote:

> When memory is onlined, we are only able to rezone from ZONE_MOVABLE to
> ZONE_KERNEL, or from (ZONE_MOVABLE - 1) to ZONE_MOVABLE.
> 
> To be more flexible, use the following criteria instead; to online memory
> from zone X into zone Y,
> 
> * Any zones between X and Y must be unused.

> * If X is lower than Y, the onlined memory must lie at the end of X.
> * If X is higher than Y, the onlined memory must lie at the start of X.

If memory address has hole, memory address gets uncotinuous. Then memory
cannot be changed the zone by above the two conditions. So the conditions
shouold be removed.

Thanks,
Yasuaki Ishimatsu

> 
> Add zone_can_shift() to make this determination.
> 
> Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com>
> ---
>  include/linux/memory_hotplug.h |  2 ++
>  mm/memory_hotplug.c            | 42 +++++++++++++++++++++++++++++++++++-------
>  2 files changed, 37 insertions(+), 7 deletions(-)
> 
> diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
> index adbef58..7bff0f9 100644
> --- a/include/linux/memory_hotplug.h
> +++ b/include/linux/memory_hotplug.h
> @@ -284,5 +284,7 @@ extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms,
>  		unsigned long map_offset);
>  extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
>  					  unsigned long pnum);
> +extern int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
> +			  enum zone_type target);
>  
>  #endif /* __LINUX_MEMORY_HOTPLUG_H */
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 6b4b005..b63cc28 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -1032,6 +1032,37 @@ static void node_states_set_node(int node, struct memory_notify *arg)
>  	node_set_state(node, N_MEMORY);
>  }
>  
> +int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
> +		   enum zone_type target)
> +{
> +	struct zone *zone = page_zone(pfn_to_page(pfn));
> +	enum zone_type idx = zone_idx(zone);
> +	int i;
> +
> +	if (idx < target) {
> +		/* pages must be at end of current zone */
> +		if (pfn + nr_pages != zone_end_pfn(zone))
> +			return 0;
> +
> +		/* no zones in use between current zone and target */
> +		for (i = idx + 1; i < target; i++)
> +			if (zone_is_initialized(zone - idx + i))
> +				return 0;
> +	}
> +
> +	if (target < idx) {
> +		/* pages must be at beginning of current zone */
> +		if (pfn != zone->zone_start_pfn)
> +			return 0;
> +
> +		/* no zones in use between current zone and target */
> +		for (i = target + 1; i < idx; i++)
> +			if (zone_is_initialized(zone - idx + i))
> +				return 0;
> +	}
> +
> +	return target - idx;
> +}
>  
>  /* Must be protected by mem_hotplug_begin() */
>  int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_type)
> @@ -1057,13 +1088,10 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ
>  	    !can_online_high_movable(zone))
>  		return -EINVAL;
>  
> -	if (online_type == MMOP_ONLINE_KERNEL &&
> -	    zone_idx(zone) == ZONE_MOVABLE)
> -		zone_shift = -1;
> -
> -	if (online_type == MMOP_ONLINE_MOVABLE &&
> -	    zone_idx(zone) == ZONE_MOVABLE - 1)
> -		zone_shift = 1;
> +	if (online_type == MMOP_ONLINE_KERNEL)
> +		zone_shift = zone_can_shift(pfn, nr_pages, ZONE_NORMAL);
> +	else if (online_type == MMOP_ONLINE_MOVABLE)
> +		zone_shift = zone_can_shift(pfn, nr_pages, ZONE_MOVABLE);
>  
>  	zone = move_pfn_range(zone_shift, pfn, pfn + nr_pages);
>  	if (!zone)
> -- 
> 1.8.3.1
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[toc] | [prev] | [next] | [standalone]

#1398497 — Re: [PATCH 2/3] memory-hotplug: more general validation of zone during online

On Tue, May 10, 2016 at 11:08:56AM -0700, Yasuaki Ishimatsu wrote:
>On Mon,  9 May 2016 12:53:38 -0500
>Reza Arbab <arbab@linux.vnet.ibm.com> wrote:
>> * If X is lower than Y, the onlined memory must lie at the end of X.
>> * If X is higher than Y, the onlined memory must lie at the start of X.
>
>If memory address has hole, memory address gets uncotinuous. Then memory
>cannot be changed the zone by above the two conditions. So the conditions
>shouold be removed.

I don't understand what you mean by this. Could you give an example?

-- 
Reza Arbab

[toc] | [prev] | [next] | [standalone]

#1399238 — Re: [PATCH 2/3] memory-hotplug: more general validation of zone during online

On Tue, 10 May 2016 15:39:43 -0500
Reza Arbab <arbab@linux.vnet.ibm.com> wrote:

> On Tue, May 10, 2016 at 11:08:56AM -0700, Yasuaki Ishimatsu wrote:
> >On Mon,  9 May 2016 12:53:38 -0500
> >Reza Arbab <arbab@linux.vnet.ibm.com> wrote:
> >> * If X is lower than Y, the onlined memory must lie at the end of X.
> >> * If X is higher than Y, the onlined memory must lie at the start of X.
> >
> >If memory address has hole, memory address gets uncotinuous. Then memory
> >cannot be changed the zone by above the two conditions. So the conditions
> >shouold be removed.
> 
> I don't understand what you mean by this. Could you give an example?

> +int zone_can_shift(unsigned long pfn, unsigned long nr_pages,
> +		   enum zone_type target)
> +{
<snip>
> +	if (idx < target) {
> +		/* pages must be at end of current zone */
> +		if (pfn + nr_pages != zone_end_pfn(zone))
> +			return 0;
<snip>
> +	if (target < idx) {
> +		/* pages must be at beginning of current zone */
> +		if (pfn != zone->zone_start_pfn)
> +			return 0;

According your patch, memory address must be continuous for changing zone.
So if memory address is uncontinuous as follows, memory address 0x180000000-0x1FFFFFFFF
can be changed from ZONE_NORMAL to ZONE_MOVABLE. But memory address 0x80000000-0xFFFFFFFF
can not be changed from ZONE_NORMAL to ZONE_MOVABLE since it does not meet
above condition.

Memory address
  0x80000000 -  0xFFFFFFFF
 0x180000000 - 0x1FFFFFFFF

Thanks,
Yasuaki Ishimatsu

[toc] | [prev] | [next] | [standalone]

#1399444 — Re: [PATCH 2/3] memory-hotplug: more general validation of zone during online

On Wed, May 11, 2016 at 08:17:41AM -0700, Yasuaki Ishimatsu wrote:
>On Tue, 10 May 2016 15:39:43 -0500
>Reza Arbab <arbab@linux.vnet.ibm.com> wrote:
><snip>
>> +	if (idx < target) {
>> +		/* pages must be at end of current zone */
>> +		if (pfn + nr_pages != zone_end_pfn(zone))
>> +			return 0;
><snip>
>> +	if (target < idx) {
>> +		/* pages must be at beginning of current zone */
>> +		if (pfn != zone->zone_start_pfn)
>> +			return 0;
>
>According your patch, memory address must be continuous for changing zone.
>So if memory address is uncontinuous as follows, memory address 0x180000000-0x1FFFFFFFF
>can be changed from ZONE_NORMAL to ZONE_MOVABLE. But memory address 0x80000000-0xFFFFFFFF
>can not be changed from ZONE_NORMAL to ZONE_MOVABLE since it does not meet
>above condition.
>
>Memory address
>  0x80000000 -  0xFFFFFFFF
> 0x180000000 - 0x1FFFFFFFF

Ah, I see. What do you think of this instead?

<snip>
+	if (idx < target) {
+		/* must be the last pages present in current zone */
+		for (i = pfn + nr_pages; i < zone_end_pfn(zone); i++)
+			if (pfn_present(i))
+				return 0;
<snip>
+	if (target < idx) {
+		/* must be the first pages present in current zone */
+		for (i = zone->zone_start_pfn; i < pfn; i++)
+			if (pfn_present(i))
+				return 0;

-- 
Reza Arbab

[toc] | [prev] | [next] | [standalone]

#1397336

On Mon,  9 May 2016 12:53:36 -0500 Reza Arbab <arbab@linux.vnet.ibm.com> wrote:

> While it is currently possible to rezone memory when it is onlined, there are
> implicit assumptions about the zones:
> 
> * To "online_kernel" a block into ZONE_NORMAL, it must currently
>   be in ZONE_MOVABLE.
> 
> * To "online_movable" a block into ZONE_MOVABLE, it must currently
>   be in (ZONE_MOVABLE - 1).
> 
> So on powerpc, where new memory is hotplugged into ZONE_DMA, these operations
> do not work.
> 
> This patchset replaces the qualifications above with a more general
> validation of zone movement.
> 

The patches look good from a first scan.  It's late for 4.6 so I'll
queue them for 4.7-rc1, unless there are convincing reasons otherwise?

Hopefully the other memory-hotplug developers will be able to find time
to review these.

[toc] | [prev] | [next] | [standalone]

#1397345

On Mon, 9 May 2016 13:11:58 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> On Mon,  9 May 2016 12:53:36 -0500 Reza Arbab <arbab@linux.vnet.ibm.com> wrote:
> 
> > While it is currently possible to rezone memory when it is onlined, there are
> > implicit assumptions about the zones:
> > 
> > * To "online_kernel" a block into ZONE_NORMAL, it must currently
> >   be in ZONE_MOVABLE.
> > 
> > * To "online_movable" a block into ZONE_MOVABLE, it must currently
> >   be in (ZONE_MOVABLE - 1).
> > 
> > So on powerpc, where new memory is hotplugged into ZONE_DMA, these operations
> > do not work.
> > 
> > This patchset replaces the qualifications above with a more general
> > validation of zone movement.
> > 
> 
> The patches look good from a first scan.  It's late for 4.6 so I'll
> queue them for 4.7-rc1, unless there are convincing reasons otherwise?

err, make that 4.8-rc1.

> Hopefully the other memory-hotplug developers will be able to find time
> to review these.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web