Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1695254
| From | Dennis Zhou <dennisz@fb.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH v2 19/23] percpu: update alloc path to only scan if contig hints are broken |
| Date | 2017-07-25 01:10 +0200 |
| Message-ID | <u6UEb-3ig-57@gated-at.bofh.it> (permalink) |
| References | <u6UE9-3ig-3@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
From: "Dennis Zhou (Facebook)" <dennisszhou@gmail.com>
Metadata is kept per block to keep track of where the contig hints are.
Scanning can be avoided when the contig hints are not broken. In that
case, left and right contigs have to be managed manually.
This patch changes the allocation path hint updating to only scan when
contig hints are broken.
Signed-off-by: Dennis Zhou <dennisszhou@gmail.com>
---
mm/percpu.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 56 insertions(+), 3 deletions(-)
diff --git a/mm/percpu.c b/mm/percpu.c
index aaad747..2bf2cfc 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -514,6 +514,10 @@ static void pcpu_block_refresh_hint(struct pcpu_chunk *chunk, int index)
* @chunk: chunk of interest
* @bit_off: chunk offset
* @bits: size of request
+ *
+ * Updates metadata for the allocation path. The metadata only has to be
+ * refreshed by a full scan iff the chunk's contig hint is broken. Block level
+ * scans are required if the block's contig hint is broken.
*/
static void pcpu_block_update_hint_alloc(struct pcpu_chunk *chunk, int bit_off,
int bits)
@@ -538,14 +542,56 @@ static void pcpu_block_update_hint_alloc(struct pcpu_chunk *chunk, int bit_off,
/*
* Update s_block.
+ * block->first_free must be updated if the allocation takes its place.
+ * If the allocation breaks the contig_hint, a scan is required to
+ * restore this hint.
*/
- pcpu_block_refresh_hint(chunk, s_index);
+ if (s_off == s_block->first_free)
+ s_block->first_free = find_next_zero_bit(
+ pcpu_index_alloc_map(chunk, s_index),
+ PCPU_BITMAP_BLOCK_BITS,
+ s_off + bits);
+
+ if (s_off >= s_block->contig_hint_start &&
+ s_off < s_block->contig_hint_start + s_block->contig_hint) {
+ /* block contig hint is broken - scan to fix it */
+ pcpu_block_refresh_hint(chunk, s_index);
+ } else {
+ /* update left and right contig manually */
+ s_block->left_free = min(s_block->left_free, s_off);
+ if (s_index == e_index)
+ s_block->right_free = min_t(int, s_block->right_free,
+ PCPU_BITMAP_BLOCK_BITS - e_off);
+ else
+ s_block->right_free = 0;
+ }
/*
* Update e_block.
*/
if (s_index != e_index) {
- pcpu_block_refresh_hint(chunk, e_index);
+ /*
+ * When the allocation is across blocks, the end is along
+ * the left part of the e_block.
+ */
+ e_block->first_free = find_next_zero_bit(
+ pcpu_index_alloc_map(chunk, e_index),
+ PCPU_BITMAP_BLOCK_BITS, e_off);
+
+ if (e_off == PCPU_BITMAP_BLOCK_BITS) {
+ /* reset the block */
+ e_block++;
+ } else {
+ if (e_off > e_block->contig_hint_start) {
+ /* contig hint is broken - scan to fix it */
+ pcpu_block_refresh_hint(chunk, e_index);
+ } else {
+ e_block->left_free = 0;
+ e_block->right_free =
+ min_t(int, e_block->right_free,
+ PCPU_BITMAP_BLOCK_BITS - e_off);
+ }
+ }
/* update in-between md_blocks */
for (block = s_block + 1; block < e_block; block++) {
@@ -555,7 +601,14 @@ static void pcpu_block_update_hint_alloc(struct pcpu_chunk *chunk, int bit_off,
}
}
- pcpu_chunk_refresh_hint(chunk);
+ /*
+ * The only time a full chunk scan is required is if the chunk
+ * contig hint is broken. Otherwise, it means a smaller space
+ * was used and therefore the chunk contig hint is still correct.
+ */
+ if (bit_off >= chunk->contig_bits_start &&
+ bit_off < chunk->contig_bits_start + chunk->contig_bits)
+ pcpu_chunk_refresh_hint(chunk);
}
/**
--
2.9.3
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH v2 00/23] percpu: replace percpu area map allocator with bitmap allocator Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
[PATCH v2 22/23] percpu: update pcpu_find_block_fit to use an iterator Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 22/23] percpu: update pcpu_find_block_fit to use an iterator Josef Bacik <josef@toxicpanda.com> - 2017-07-25 21:50 +0200
[PATCH v2 09/23] percpu: combine percpu address checks Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 09/23] percpu: combine percpu address checks Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:30 +0200
[PATCH v2 02/23] percpu: introduce start_offset to pcpu_chunk Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 02/23] percpu: introduce start_offset to pcpu_chunk Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:10 +0200
[PATCH v2 13/23] percpu: generalize bitmap (un)populated iterators Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 13/23] percpu: generalize bitmap (un)populated iterators Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:40 +0200
Re: [PATCH v2 13/23] percpu: generalize bitmap (un)populated iterators Tejun Heo <tj@kernel.org> - 2017-07-26 16:10 +0200
[PATCH v2 08/23] percpu: modify base_addr to be region specific Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 08/23] percpu: modify base_addr to be region specific Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:30 +0200
[PATCH v2 17/23] percpu: skip chunks if the alloc does not fit in the contig hint Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 17/23] percpu: skip chunks if the alloc does not fit in the contig hint Josef Bacik <josef@toxicpanda.com> - 2017-07-25 21:50 +0200
[PATCH v2 19/23] percpu: update alloc path to only scan if contig hints are broken Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 19/23] percpu: update alloc path to only scan if contig hints are broken Josef Bacik <josef@toxicpanda.com> - 2017-07-25 21:40 +0200
[PATCH v2 12/23] percpu: increase minimum percpu allocation size and align first regions Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:10 +0200
Re: [PATCH v2 12/23] percpu: increase minimum percpu allocation size and align first regions Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:40 +0200
[PATCH v2 01/23] percpu: setup_first_chunk enforce dynamic region must exist Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:20 +0200
Re: [PATCH v2 01/23] percpu: setup_first_chunk enforce dynamic region must exist Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:10 +0200
[PATCH v2 06/23] percpu: end chunk area maps page aligned for the populated bitmap Dennis Zhou <dennisz@fb.com> - 2017-07-25 01:20 +0200
Re: [PATCH v2 06/23] percpu: end chunk area maps page aligned for the populated bitmap Josef Bacik <josef@toxicpanda.com> - 2017-07-25 20:20 +0200
Re: [PATCH v2 14/23] percpu: replace area map allocator with bitmap allocator Josef Bacik <josef@toxicpanda.com> - 2017-07-25 21:20 +0200
csiph-web