Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1371437 > unrolled thread

[PATCH 00/27] block: cleanup direct access on .bi_vcnt & .bi_io_vec

Back to article view | Back to linux.kernel

Contents

  [PATCH 00/27] block: cleanup direct access on .bi_vcnt & .bi_io_vec Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:00 +0200
    [PATCH 02/27] block: drbd: use bio_get_base_vec() to retrieve the 1st bvec Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:00 +0200
    [PATCH 05/27] block: pktcdvd: use bio_get_base_vec() to retrive bvec table Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:00 +0200
    [PATCH 04/27] block: loop: use bio_get_base_vec() to retrive bvec table Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:00 +0200
    [PATCH 03/27] block: drbd: remove impossible failure handling Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:00 +0200
      Re: [PATCH 03/27] block: drbd: remove impossible failure handling Lars Ellenberg <lars.ellenberg@linbit.com> - 2016-04-05 14:50 +0200
    [PATCH 11/27] bcache: io.c: use bio_set_vec_table Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
      Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table Christoph Hellwig <hch@infradead.org> - 2016-04-05 14:50 +0200
        Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table Ming Lei <tom.leiming@gmail.com> - 2016-04-05 17:30 +0200
          Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table Christoph Hellwig <hch@infradead.org> - 2016-04-05 19:40 +0200
        Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table Kent Overstreet <kent.overstreet@gmail.com> - 2016-04-06 02:40 +0200
    [PATCH 12/27] bcache: journal.c: use bio_set_vec_table() Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
    [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
      Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly Greg Kroah-Hartman <gregkh@linuxfoundation.org> - 2016-04-05 15:00 +0200
      Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly Christoph Hellwig <hch@infradead.org> - 2016-04-05 15:10 +0200
        Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt  directly James Simmons <jsimmons@infradead.org> - 2016-04-10 16:40 +0200
          Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly Christoph Hellwig <hch@infradead.org> - 2016-04-10 16:50 +0200
            Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt  directly James Simmons <jsimmons@infradead.org> - 2016-04-10 18:10 +0200
              Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly Ming Lei <tom.leiming@gmail.com> - 2016-04-11 05:40 +0200
    [PATCH 10/27] bcache: debug: avoid to access .bi_io_vec directly Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
    [PATCH 07/27] block: floppy: use bio_add_page() Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
    [PATCH 09/27] target: use bio_is_full() Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
      Re: [PATCH 09/27] target: use bio_is_full() Christoph Hellwig <hch@infradead.org> - 2016-04-05 15:10 +0200
        Re: [PATCH 09/27] target: use bio_is_full() Ming Lei <tom.leiming@gmail.com> - 2016-04-07 06:10 +0200
    [PATCH 14/27] bcache: writeback: use bio_set_vec_table() Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
    [PATCH 13/27] bcache: movinggc: use bio_set_vec_table() Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:10 +0200
    [PATCH 06/27] block: floppy: use bio_set_vec_table() Ming Lei <tom.leiming@gmail.com> - 2016-04-05 14:20 +0200
      Re: [PATCH 06/27] block: floppy: use bio_set_vec_table() Christoph Hellwig <hch@infradead.org> - 2016-04-05 15:10 +0200

Page 1 of 2  [1] 2  Next page →

#1371437 — [PATCH 00/27] block: cleanup direct access on .bi_vcnt & .bi_io_vec

Hi Guys,

It is always not a good practice to access bio->bi_vcnt and
bio->bi_io_vec from drivers directly. Also this kind of direct
access will cause trouble when converting to multipage bvecs.

The 1st patch introduces the following 4 bio helpers which can be
used inside drivers for avoiding direct access to .bi_vcnt and .bi_io_vec.

	bio_pages()
	bio_is_full()
	bio_get_base_vec()
	bio_set_vec_table()

Both bio_pages() and bio_is_full() can be easy to convert to
multipage bvecs.

For bio_get_base_vec() and bio_set_vec_table(), they are often used
during initializing a new bio or in case of single bvec bio. With the
two new helpers, it becomes quite easy to audit access to .bi_io_vec
and .bi_vcnt.

Most of the other patches use the 4 helpers to clean up most of direct
access to .bi_vcnt and .bi_io_vec from drivers, except for MD and btrfs,
which two subsystems will be done in the future. 

Also bio_add_page() is used in floppy, dm-crypt and fs/logfs to
avoiding direct access to .bi_vcnt & .bi_io_vec.

Thanks,
Ming

Ming Lei (27):
  block: bio: introduce 4 helpers for cleanup
  block: drbd: use bio_get_base_vec() to retrieve the 1st bvec
  block: drbd: remove impossible failure handling
  block: loop: use bio_get_base_vec() to retrive bvec table
  block: pktcdvd: use bio_get_base_vec() to retrive bvec table
  block: floppy: use bio_set_vec_table()
  block: floppy: use bio_add_page()
  staging: lustre: avoid to use bio->bi_vcnt directly
  target: use bio_is_full()
  bcache: debug: avoid to access .bi_io_vec directly
  bcache: io.c: use bio_set_vec_table
  bcache: journal.c: use bio_set_vec_table()
  bcache: movinggc: use bio_set_vec_table()
  bcache: writeback: use bio_set_vec_table()
  bcache: super: use bio_set_vec_table()
  bcache: super: use bio_get_base_vec
  dm: crypt: use bio_add_page()
  dm: dm-io.c: use bio_get_base_vec()
  dm: dm.c: replace 'bio->bi_vcnt == 1' with !bio_multiple_segments
  dm: dm-bufio.c: use bio_set_vec_table()
  fs: logfs: use bio_set_vec_table()
  fs: logfs: convert to bio_add_page() in sync_request()
  fs: logfs: use bio_add_page() in __bdev_writeseg()
  fs: logfs: use bio_add_page() in do_erase()
  fs: logfs: remove unnecesary check
  kernel/power/swap.c: use bio_get_base_vec()
  mm: page_io.c: use bio_get_base_vec()

 drivers/block/drbd/drbd_bitmap.c            |   4 +-
 drivers/block/drbd/drbd_receiver.c          |  14 +---
 drivers/block/floppy.c                      |   9 +--
 drivers/block/loop.c                        |   5 +-
 drivers/block/pktcdvd.c                     |   3 +-
 drivers/md/bcache/debug.c                   |  11 ++-
 drivers/md/bcache/io.c                      |   3 +-
 drivers/md/bcache/journal.c                 |   3 +-
 drivers/md/bcache/movinggc.c                |   6 +-
 drivers/md/bcache/super.c                   |  28 +++++---
 drivers/md/bcache/writeback.c               |   4 +-
 drivers/md/dm-bufio.c                       |   3 +-
 drivers/md/dm-crypt.c                       |   8 +--
 drivers/md/dm-io.c                          |   7 +-
 drivers/md/dm.c                             |   3 +-
 drivers/staging/lustre/lustre/llite/lloop.c |   9 +--
 drivers/target/target_core_pscsi.c          |   2 +-
 fs/logfs/dev_bdev.c                         | 107 +++++++++++-----------------
 include/linux/bio.h                         |  28 ++++++++
 kernel/power/swap.c                         |  10 ++-
 mm/page_io.c                                |  18 ++++-
 21 files changed, 156 insertions(+), 129 deletions(-)

-- 
1.9.1

[toc] | [next] | [standalone]

#1371438 — [PATCH 02/27] block: drbd: use bio_get_base_vec() to retrieve the 1st bvec

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/block/drbd/drbd_bitmap.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/block/drbd/drbd_bitmap.c b/drivers/block/drbd/drbd_bitmap.c
index 92d6fc0..ccbd1e0 100644
--- a/drivers/block/drbd/drbd_bitmap.c
+++ b/drivers/block/drbd/drbd_bitmap.c
@@ -938,7 +938,9 @@ static void drbd_bm_endio(struct bio *bio)
 	struct drbd_bm_aio_ctx *ctx = bio->bi_private;
 	struct drbd_device *device = ctx->device;
 	struct drbd_bitmap *b = device->bitmap;
-	unsigned int idx = bm_page_to_idx(bio->bi_io_vec[0].bv_page);
+	/* single bvec bio */
+	const struct bio_vec *bvec = bio_get_base_vec(bio);
+	unsigned int idx = bm_page_to_idx(bvec->bv_page);
 
 	if ((ctx->flags & BM_AIO_COPY_PAGES) == 0 &&
 	    !bm_test_page_unchanged(b->bm_pages[idx]))
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371439 — [PATCH 05/27] block: pktcdvd: use bio_get_base_vec() to retrive bvec table

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/block/pktcdvd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c
index d06c62e..8f37435 100644
--- a/drivers/block/pktcdvd.c
+++ b/drivers/block/pktcdvd.c
@@ -1298,7 +1298,8 @@ try_next_bio:
 static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt)
 {
 	int f;
-	struct bio_vec *bvec = pkt->w_bio->bi_io_vec;
+	/* need to fix this usage after multipage bvecs */
+	struct bio_vec *bvec = bio_get_base_vec(pkt->w_bio);
 
 	bio_reset(pkt->w_bio);
 	pkt->w_bio->bi_iter.bi_sector = pkt->sector;
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371445 — [PATCH 04/27] block: loop: use bio_get_base_vec() to retrive bvec table

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/block/loop.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 423f4ca..2a94d3bb 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -477,7 +477,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
 		     loff_t pos, bool rw)
 {
 	struct iov_iter iter;
-	struct bio_vec *bvec;
+	const struct bio_vec *bvec;
 	struct bio *bio = cmd->rq->bio;
 	struct file *file = lo->lo_backing_file;
 	int ret;
@@ -485,7 +485,8 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
 	/* nomerge for loop request queue */
 	WARN_ON(cmd->rq->bio != cmd->rq->biotail);
 
-	bvec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter);
+	/* passed to iterate_bvec() */
+	bvec = bio_get_base_vec(bio);
 	iov_iter_bvec(&iter, ITER_BVEC | rw, bvec,
 		      bio_segments(bio), blk_rq_bytes(cmd->rq));
 
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371446 — [PATCH 03/27] block: drbd: remove impossible failure handling

For a non-cloned bio, bio_add_page() only returns failure when
the io vec table is full, but in that case, bio->bi_vcnt can't
be zero at all.

So remove the impossible failure handling.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/block/drbd/drbd_receiver.c | 14 +-------------
 1 file changed, 1 insertion(+), 13 deletions(-)

diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 050aaa1..1b0ed15 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1465,20 +1465,8 @@ next_bio:
 
 	page_chain_for_each(page) {
 		unsigned len = min_t(unsigned, data_size, PAGE_SIZE);
-		if (!bio_add_page(bio, page, len, 0)) {
-			/* A single page must always be possible!
-			 * But in case it fails anyways,
-			 * we deal with it, and complain (below). */
-			if (bio->bi_vcnt == 0) {
-				drbd_err(device,
-					"bio_add_page failed for len=%u, "
-					"bi_vcnt=0 (bi_sector=%llu)\n",
-					len, (uint64_t)bio->bi_iter.bi_sector);
-				err = -ENOSPC;
-				goto fail;
-			}
+		if (!bio_add_page(bio, page, len, 0))
 			goto next_bio;
-		}
 		data_size -= len;
 		sector += len >> 9;
 		--nr_pages;
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371515 — Re: [PATCH 03/27] block: drbd: remove impossible failure handling

On Tue, Apr 05, 2016 at 07:56:48PM +0800, Ming Lei wrote:
> For a non-cloned bio, bio_add_page() only returns failure when
> the io vec table is full, but in that case, bio->bi_vcnt can't
> be zero at all.
> 
> So remove the impossible failure handling.

Before the immutable bvecs,
we did in fact see this trigger in the wild.
On "strange" deployments.

But for the current implementation of bio_add_page(),
you are correct, this is impossible now.

Ack.

Thanks,

    Lars

[toc] | [prev] | [next] | [standalone]

#1371459 — [PATCH 11/27] bcache: io.c: use bio_set_vec_table

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/md/bcache/io.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
index 86a0bb8..1c48462 100644
--- a/drivers/md/bcache/io.c
+++ b/drivers/md/bcache/io.c
@@ -26,8 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
 
 	bio_init(bio);
 	bio->bi_flags		|= BIO_POOL_NONE << BIO_POOL_OFFSET;
-	bio->bi_max_vecs	 = bucket_pages(c);
-	bio->bi_io_vec		 = bio->bi_inline_vecs;
+	bio_set_vec_table(bio, bio->bi_inline_vecs, bucket_pages(c));
 
 	return bio;
 }
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371519 — Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table

On Tue, Apr 05, 2016 at 07:56:56PM +0800, Ming Lei wrote:
> diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
> index 86a0bb8..1c48462 100644
> --- a/drivers/md/bcache/io.c
> +++ b/drivers/md/bcache/io.c
> @@ -26,8 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
>  
>  	bio_init(bio);
>  	bio->bi_flags		|= BIO_POOL_NONE << BIO_POOL_OFFSET;
> -	bio->bi_max_vecs	 = bucket_pages(c);
> -	bio->bi_io_vec		 = bio->bi_inline_vecs;
> +	bio_set_vec_table(bio, bio->bi_inline_vecs, bucket_pages(c));

All this bcache code needs to move away from bio_init on a bio
embedded in a driver private structure toward properly using
bio_alloc / bio_alloc_bioset.  That will also fix the crash
with bcache over md that Shaohua reported, so I'd suggest to fast
track this part of the series.

[toc] | [prev] | [next] | [standalone]

#1371719 — Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table

On Tue, Apr 5, 2016 at 8:49 PM, Christoph Hellwig <hch@infradead.org> wrote:
> On Tue, Apr 05, 2016 at 07:56:56PM +0800, Ming Lei wrote:
>> diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
>> index 86a0bb8..1c48462 100644
>> --- a/drivers/md/bcache/io.c
>> +++ b/drivers/md/bcache/io.c
>> @@ -26,8 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
>>
>>       bio_init(bio);
>>       bio->bi_flags           |= BIO_POOL_NONE << BIO_POOL_OFFSET;
>> -     bio->bi_max_vecs         = bucket_pages(c);
>> -     bio->bi_io_vec           = bio->bi_inline_vecs;
>> +     bio_set_vec_table(bio, bio->bi_inline_vecs, bucket_pages(c));
>
> All this bcache code needs to move away from bio_init on a bio
> embedded in a driver private structure toward properly using
> bio_alloc / bio_alloc_bioset.  That will also fix the crash
> with bcache over md that Shaohua reported, so I'd suggest to fast
> track this part of the series.

I suggest to keep this usage for the following reasons:

- bio can be embedded into one biger instance, which is often allocated
dynamically, so one extra allocation for bio can be avoided.

- we should support arbitrary bio size by this way, at least bio_add_page()
supports this usage.  Also code gets lots of simplication with arbitrary bio
size support, such as prio_io(): bcache

BTW, the root cause for bcache crash still isn't clear now because
blk_bio_segment_split() should split big bio into proper size with
all queue's limits. Maybe the max segment limit isn't figured out correctly.

Thanks,
Ming Lei

[toc] | [prev] | [next] | [standalone]

#1371821 — Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table

On Tue, Apr 05, 2016 at 11:24:30PM +0800, Ming Lei wrote:
> - bio can be embedded into one biger instance, which is often allocated
> dynamically, so one extra allocation for bio can be avoided.

We can also do this the other way around with the bios front_pad,
which avoid the caller poking into bio details.

> - we should support arbitrary bio size by this way, at least bio_add_page()
> supports this usage.  Also code gets lots of simplication with arbitrary bio
> size support, such as prio_io(): bcache

There is no reason for not supporting huge bios in the core bio code,
in fact using bio_kmalloc you can already allocate huges bios
dynamically right now.  Except that you can't really use it, because the
layers below don't expect that.  Bios based drivers expect to be able to
call bio_clone and friends called on bios passed to them, and might
also make assumptions about the max number of bios segments for now.

> BTW, the root cause for bcache crash still isn't clear now because
> blk_bio_segment_split() should split big bio into proper size with
> all queue's limits. Maybe the max segment limit isn't figured out correctly.

The root cause is pretty simple:  The queue limits matter for request
based drivers, which are the only ones getting bios > BIO_MAX_PAGES
except for the buggy bcache use case.  You'll need to either adjust the
limit for all bio based drivers to or get rid of that one magic caller
not playing by the rules.

[toc] | [prev] | [next] | [standalone]

#1372099 — Re: [PATCH 11/27] bcache: io.c: use bio_set_vec_table

On Tue, Apr 05, 2016 at 05:49:02AM -0700, Christoph Hellwig wrote:
> On Tue, Apr 05, 2016 at 07:56:56PM +0800, Ming Lei wrote:
> > diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
> > index 86a0bb8..1c48462 100644
> > --- a/drivers/md/bcache/io.c
> > +++ b/drivers/md/bcache/io.c
> > @@ -26,8 +26,7 @@ struct bio *bch_bbio_alloc(struct cache_set *c)
> >  
> >  	bio_init(bio);
> >  	bio->bi_flags		|= BIO_POOL_NONE << BIO_POOL_OFFSET;
> > -	bio->bi_max_vecs	 = bucket_pages(c);
> > -	bio->bi_io_vec		 = bio->bi_inline_vecs;
> > +	bio_set_vec_table(bio, bio->bi_inline_vecs, bucket_pages(c));
> 
> All this bcache code needs to move away from bio_init on a bio
> embedded in a driver private structure toward properly using
> bio_alloc / bio_alloc_bioset.  That will also fix the crash
> with bcache over md that Shaohua reported, so I'd suggest to fast
> track this part of the series.

Why?

bio_init() is a publicly exported function, it's always been one and bcache is
ot the only driver to use it directly.

bios with > BIO_MAX_PAGES bvecs is a separate issue; I would argue that the bug
is in md's queue_limits; it uses blk_set_stacking_limits() which sets
max_segments = USHRT_MAX, which is wrong if it's going to clone the biovec.

[toc] | [prev] | [next] | [standalone]

#1371460 — [PATCH 12/27] bcache: journal.c: use bio_set_vec_table()

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/md/bcache/journal.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/md/bcache/journal.c b/drivers/md/bcache/journal.c
index 29eba72..bf8924f 100644
--- a/drivers/md/bcache/journal.c
+++ b/drivers/md/bcache/journal.c
@@ -453,8 +453,7 @@ static void do_journal_discard(struct cache *ca)
 						ca->sb.d[ja->discard_idx]);
 		bio->bi_bdev		= ca->bdev;
 		bio->bi_rw		= REQ_WRITE|REQ_DISCARD;
-		bio->bi_max_vecs	= 1;
-		bio->bi_io_vec		= bio->bi_inline_vecs;
+		bio_set_vec_table(bio, bio->bi_inline_vecs, 1);
 		bio->bi_iter.bi_size	= bucket_bytes(ca);
 		bio->bi_end_io		= journal_discard_endio;
 
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371462 — [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/staging/lustre/lustre/llite/lloop.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
index b725fc1..67323db 100644
--- a/drivers/staging/lustre/lustre/llite/lloop.c
+++ b/drivers/staging/lustre/lustre/llite/lloop.c
@@ -302,19 +302,20 @@ static unsigned int loop_get_bio(struct lloop_device *lo, struct bio **req)
 	}
 
 	/* TODO: need to split the bio, too bad. */
-	LASSERT(first->bi_vcnt <= LLOOP_MAX_SEGMENTS);
+	LASSERT(bio_pages(first) <= LLOOP_MAX_SEGMENTS);
 
 	rw = first->bi_rw;
 	bio = &lo->lo_bio;
 	while (*bio && (*bio)->bi_rw == rw) {
+		unsigned curr_cnt = bio_pages(*bio);
 		CDEBUG(D_INFO, "bio sector %llu size %u count %u vcnt%u\n",
 		       (unsigned long long)(*bio)->bi_iter.bi_sector,
 		       (*bio)->bi_iter.bi_size,
-		       page_count, (*bio)->bi_vcnt);
-		if (page_count + (*bio)->bi_vcnt > LLOOP_MAX_SEGMENTS)
+		       page_count, curr_cnt);
+		if (page_count + curr_cnt > LLOOP_MAX_SEGMENTS)
 			break;
 
-		page_count += (*bio)->bi_vcnt;
+		page_count += curr_cnt;
 		count++;
 		bio = &(*bio)->bi_next;
 	}
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

#1371532 — Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

On Tue, Apr 05, 2016 at 07:56:53PM +0800, Ming Lei wrote:
> Signed-off-by: Ming Lei <tom.leiming@gmail.com>

A bit more of a commit message is always nice :)

Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

[toc] | [prev] | [next] | [standalone]

#1371549 — Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

The lloop driver should be removed entirely - use the loop driver
instead.

[toc] | [prev] | [next] | [standalone]

#1374796 — Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

> The lloop driver should be removed entirely - use the loop driver
> instead.

I talked with Andreas last week at our annual Lustre users group meeting 
about this. The reason I was told for existance is that some users were
using files on a Lustre file system with the loop back device. The 
performance was really bad at the time so a lloop was developed to 
overcome those limitations. Its been a long time so perhaps its time
to look at the default loop driver again to see if can perform now. If
it doesn't we will go the route of reworking the lloop driver in the
spirit of the cryptoloop device.

[toc] | [prev] | [next] | [standalone]

#1374797 — Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

On Sun, Apr 10, 2016 at 03:37:42PM +0100, James Simmons wrote:
> 
> > The lloop driver should be removed entirely - use the loop driver
> > instead.
> 
> I talked with Andreas last week at our annual Lustre users group meeting 
> about this. The reason I was told for existance is that some users were
> using files on a Lustre file system with the loop back device. The 
> performance was really bad at the time so a lloop was developed to 
> overcome those limitations. Its been a long time so perhaps its time
> to look at the default loop driver again to see if can perform now. If
> it doesn't we will go the route of reworking the lloop driver in the
> spirit of the cryptoloop device.

The loop driver now supports using AIO/DIO on any file systems that
implements ->read_iter and ->write_iter. If lustre doesn't support
those or doesn't have proper performance using them it should be
addressed in the file system.

Note that the dio mode in the loop device is not the default and you
need to manually enabled it, keep that in mind when testing.

[toc] | [prev] | [next] | [standalone]

#1374812 — Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

> On Sun, Apr 10, 2016 at 03:37:42PM +0100, James Simmons wrote:
> > 
> > > The lloop driver should be removed entirely - use the loop driver
> > > instead.
> > 
> > I talked with Andreas last week at our annual Lustre users group meeting 
> > about this. The reason I was told for existance is that some users were
> > using files on a Lustre file system with the loop back device. The 
> > performance was really bad at the time so a lloop was developed to 
> > overcome those limitations. Its been a long time so perhaps its time
> > to look at the default loop driver again to see if can perform now. If
> > it doesn't we will go the route of reworking the lloop driver in the
> > spirit of the cryptoloop device.
> 
> The loop driver now supports using AIO/DIO on any file systems that
> implements ->read_iter and ->write_iter. If lustre doesn't support
> those or doesn't have proper performance using them it should be
> addressed in the file system.
> 
> Note that the dio mode in the loop device is not the default and you
> need to manually enabled it, keep that in mind when testing.

This is excellent news. The only sad thing is that most lustre users
are running distros that use kernels before the AIO/DIO enhancements
were landed :-( We will have to keep a copy around for those guys. But
first I need to test the performance of the loop back driver this
week before this can be dropped.

[toc] | [prev] | [next] | [standalone]

#1375448 — Re: [PATCH 08/27] staging: lustre: avoid to use bio->bi_vcnt directly

On Mon, Apr 11, 2016 at 12:02 AM, James Simmons <jsimmons@infradead.org> wrote:
>
>> On Sun, Apr 10, 2016 at 03:37:42PM +0100, James Simmons wrote:
>> >
>> > > The lloop driver should be removed entirely - use the loop driver
>> > > instead.
>> >
>> > I talked with Andreas last week at our annual Lustre users group meeting
>> > about this. The reason I was told for existance is that some users were
>> > using files on a Lustre file system with the loop back device. The
>> > performance was really bad at the time so a lloop was developed to
>> > overcome those limitations. Its been a long time so perhaps its time
>> > to look at the default loop driver again to see if can perform now. If
>> > it doesn't we will go the route of reworking the lloop driver in the
>> > spirit of the cryptoloop device.
>>
>> The loop driver now supports using AIO/DIO on any file systems that
>> implements ->read_iter and ->write_iter. If lustre doesn't support
>> those or doesn't have proper performance using them it should be
>> addressed in the file system.
>>
>> Note that the dio mode in the loop device is not the default and you
>> need to manually enabled it, keep that in mind when testing.
>
> This is excellent news. The only sad thing is that most lustre users
> are running distros that use kernels before the AIO/DIO enhancements
> were landed :-( We will have to keep a copy around for those guys. But
> first I need to test the performance of the loop back driver this
> week before this can be dropped.

Considered that this cleanup patch for lustre loop is quite simple and
straightforward, I suggest to keep this cleanup patch as so and do the
dropping in another patchset. Christoph, are you OK with that?

Thanks,
Ming Lei

[toc] | [prev] | [next] | [standalone]

#1371464 — [PATCH 10/27] bcache: debug: avoid to access .bi_io_vec directly

Instead we use standard iterator way to do that.

Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
 drivers/md/bcache/debug.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache/debug.c
index 8b1f1d5..d1ad49d 100644
--- a/drivers/md/bcache/debug.c
+++ b/drivers/md/bcache/debug.c
@@ -106,8 +106,8 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
 {
 	char name[BDEVNAME_SIZE];
 	struct bio *check;
-	struct bio_vec bv, *bv2;
-	struct bvec_iter iter;
+	struct bio_vec bv, cbv, *bv2;
+	struct bvec_iter iter, citer = { 0 };
 	int i;
 
 	check = bio_clone(bio, GFP_NOIO);
@@ -119,9 +119,13 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
 
 	submit_bio_wait(READ_SYNC, check);
 
+	citer.bi_size = UINT_MAX;
 	bio_for_each_segment(bv, bio, iter) {
 		void *p1 = kmap_atomic(bv.bv_page);
-		void *p2 = page_address(check->bi_io_vec[iter.bi_idx].bv_page);
+		void *p2;
+
+		cbv = bio_iter_iovec(check, citer);
+		p2 = page_address(cbv.bv_page);
 
 		cache_set_err_on(memcmp(p1 + bv.bv_offset,
 					p2 + bv.bv_offset,
@@ -132,6 +136,7 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio)
 				 (uint64_t) bio->bi_iter.bi_sector);
 
 		kunmap_atomic(p1);
+		bio_advance_iter(check, &citer, bv.bv_len);
 	}
 
 	bio_for_each_segment_all(bv2, check, i)
-- 
1.9.1

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web