Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1257808

[PATCH] md/raid5: fix locking in handle_stripe_clean_event()

Show all headers | View raw

After commit 566c09c53455 ("raid5: relieve lock contention in get_active_stripe()")
__find_stripe() is called under conf->hash_locks + hash.
But handle_stripe_clean_event() calls remove_hash() under
conf->device_lock.

Under some cirscumstances the hash chain can be circuited,
and we get an infinite loop with disabled interrupts and locked hash
lock in __find_stripe(). This leads to hard lockup on multiple CPUs
and following system crash.

I was able to reproduce this behavior on raid6 over 6 ssd disks.
The devices_handle_discard_safely option should be set to enable trim
support. The following script was used:

for i in `seq 1 32`; do
    dd if=/dev/zero of=large$i bs=10M count=100 &
done

Signed-off-by: Roman Gushchin <klamm@yandex-team.ru>
Cc: Neil Brown <neilb@suse.de>
Cc: Shaohua Li <shli@kernel.org>
Cc: linux-raid@vger.kernel.org
Cc: <stable@vger.kernel.org> # 3.10 - 3.19
---
 drivers/md/raid5.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index e421016..5fa7549 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -3060,6 +3060,8 @@ static void handle_stripe_clean_event(struct r5conf *conf,
 		}
 	if (!discard_pending &&
 	    test_bit(R5_Discard, &sh->dev[sh->pd_idx].flags)) {
+		int hash = sh->hash_lock_index;
+
 		clear_bit(R5_Discard, &sh->dev[sh->pd_idx].flags);
 		clear_bit(R5_UPTODATE, &sh->dev[sh->pd_idx].flags);
 		if (sh->qd_idx >= 0) {
@@ -3073,9 +3075,9 @@ static void handle_stripe_clean_event(struct r5conf *conf,
 		 * no updated data, so remove it from hash list and the stripe
 		 * will be reinitialized
 		 */
-		spin_lock_irq(&conf->device_lock);
+		spin_lock_irq(conf->hash_locks + hash);
 		remove_hash(sh);
-		spin_unlock_irq(&conf->device_lock);
+		spin_unlock_irq(conf->hash_locks + hash);
 		if (test_bit(STRIPE_SYNC_REQUESTED, &sh->state))
 			set_bit(STRIPE_HANDLE, &sh->state);
 
-- 
2.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

[PATCH] md/raid5: fix locking in handle_stripe_clean_event() Roman Gushchin <klamm@yandex-team.ru> - 2015-10-28 10:00 +0100
  Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Neil Brown <neilb@suse.de> - 2015-10-29 01:40 +0100
    Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Roman Gushchin <klamm@yandex-team.ru> - 2015-10-29 15:30 +0100
      Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Greg KH <greg@kroah.com> - 2015-10-29 22:30 +0100
      Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Neil Brown <neilb@suse.de> - 2015-10-30 02:40 +0100
        Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Roman Gushchin <klamm@yandex-team.ru> - 2015-10-30 15:10 +0100
          Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Shaohua Li <shli@kernel.org> - 2015-10-30 17:30 +0100
            Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Neil Brown <neilb@suse.de> - 2015-10-30 23:20 +0100
              Re: [PATCH] md/raid5: fix locking in handle_stripe_clean_event() Roman Gushchin <klamm@yandex-team.ru> - 2015-10-31 13:30 +0100

csiph-web