Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1202041

[PATCH 3.13.y-ckt 41/53] st: null pointer dereference panic caused by use after kref_put by st_open

From Kamal Mostafa <kamal@canonical.com>
Newsgroups linux.kernel
Subject [PATCH 3.13.y-ckt 41/53] st: null pointer dereference panic caused by use after kref_put by st_open
Date 2015-08-06 22:50 +0200
Message-ID <pUAgY-3gz-43@gated-at.bofh.it> (permalink)
References <pUA7f-34H-5@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

3.13.11-ckt25 -stable review patch.  If anyone has any objections, please let me know.

------------------

From: "Seymour, Shane M" <shane.seymour@hp.com>

commit e7ac6c6666bec0a354758a1298d3231e4a635362 upstream.

Two SLES11 SP3 servers encountered similar crashes simultaneously
following some kind of SAN/tape target issue:

...
qla2xxx [0000:81:00.0]-801c:3: Abort command issued nexus=3:0:2 --  1 2002.
qla2xxx [0000:81:00.0]-801c:3: Abort command issued nexus=3:0:2 --  1 2002.
qla2xxx [0000:81:00.0]-8009:3: DEVICE RESET ISSUED nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800c:3: do_reset failed for cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800f:3: DEVICE RESET FAILED: Task management failed nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-8009:3: TARGET RESET ISSUED nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800c:3: do_reset failed for cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-800f:3: TARGET RESET FAILED: Task management failed nexus=3:0:2 cmd=ffff882f89c2c7c0.
qla2xxx [0000:81:00.0]-8012:3: BUS RESET ISSUED nexus=3:0:2.
qla2xxx [0000:81:00.0]-802b:3: BUS RESET SUCCEEDED nexus=3:0:2.
qla2xxx [0000:81:00.0]-505f:3: Link is operational (8 Gbps).
qla2xxx [0000:81:00.0]-8018:3: ADAPTER RESET ISSUED nexus=3:0:2.
qla2xxx [0000:81:00.0]-00af:3: Performing ISP error recovery - ha=ffff88bf04d18000.
 rport-3:0-0: blocked FC remote port time out: removing target and saving binding
qla2xxx [0000:81:00.0]-505f:3: Link is operational (8 Gbps).
qla2xxx [0000:81:00.0]-8017:3: ADAPTER RESET SUCCEEDED nexus=3:0:2.
 rport-2:0-0: blocked FC remote port time out: removing target and saving binding
sg_rq_end_io: device detached
BUG: unable to handle kernel NULL pointer dereference at 00000000000002a8
IP: [<ffffffff8133b268>] __pm_runtime_idle+0x28/0x90
PGD 7e6586f067 PUD 7e5af06067 PMD 0 [1739975.390354] Oops: 0002 [#1] SMP
CPU 0
...
Supported: No, Proprietary modules are loaded [1739975.390463]
Pid: 27965, comm: ABCD Tainted: PF           X 3.0.101-0.29-default #1 HP ProLiant DL580 Gen8
RIP: 0010:[<ffffffff8133b268>]  [<ffffffff8133b268>] __pm_runtime_idle+0x28/0x90
RSP: 0018:ffff8839dc1e7c68  EFLAGS: 00010202
RAX: 0000000000000000 RBX: ffff883f0592fc00 RCX: 0000000000000090
RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000138
RBP: 0000000000000138 R08: 0000000000000010 R09: ffffffff81bd39d0
R10: 00000000000009c0 R11: ffffffff81025790 R12: 0000000000000001
R13: ffff883022212b80 R14: 0000000000000004 R15: ffff883022212b80
FS:  00007f8e54560720(0000) GS:ffff88407f800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000000002a8 CR3: 0000007e6ced6000 CR4: 00000000001407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process ABCD (pid: 27965, threadinfo ffff8839dc1e6000, task ffff883592e0c640)
Stack:
 ffff883f0592fc00 00000000fffffffa 0000000000000001 ffff883022212b80
 ffff883eff772400 ffffffffa03fa309 0000000000000000 0000000000000000
 ffffffffa04003a0 ffff883f063196c0 ffff887f0379a930 ffffffff8115ea1e
Call Trace:
 [<ffffffffa03fa309>] st_open+0x129/0x240 [st]
 [<ffffffff8115ea1e>] chrdev_open+0x13e/0x200
 [<ffffffff811588a8>] __dentry_open+0x198/0x310
 [<ffffffff81167d74>] do_last+0x1f4/0x800
 [<ffffffff81168fe9>] path_openat+0xd9/0x420
 [<ffffffff8116946c>] do_filp_open+0x4c/0xc0
 [<ffffffff8115a00f>] do_sys_open+0x17f/0x250
 [<ffffffff81468d92>] system_call_fastpath+0x16/0x1b
 [<00007f8e4f617fd0>] 0x7f8e4f617fcf
Code: eb d3 90 48 83 ec 28 40 f6 c6 04 48 89 6c 24 08 4c 89 74 24 20 48 89 fd 48 89 1c 24 4c 89 64 24 10 41 89 f6 4c 89 6c 24 18 74 11 <f0> ff 8f 70 01 00 00 0f 94 c0 45 31 ed 84 c0 74 2b 4c 8d a5 a0
RIP  [<ffffffff8133b268>] __pm_runtime_idle+0x28/0x90
 RSP <ffff8839dc1e7c68>
CR2: 00000000000002a8

Analysis reveals the cause of the crash to be due to STp->device
being NULL. The pointer was NULLed via scsi_tape_put(STp) when it
calls scsi_tape_release(). In st_open() we jump to err_out after
scsi_block_when_processing_errors() completes and returns the
device as offline (sdev_state was SDEV_DEL):

1180 /* Open the device. Needs to take the BKL only because of incrementing the SCSI host
1181    module count. */
1182 static int st_open(struct inode *inode, struct file *filp)
1183 {
1184         int i, retval = (-EIO);
1185         int resumed = 0;
1186         struct scsi_tape *STp;
1187         struct st_partstat *STps;
1188         int dev = TAPE_NR(inode);
1189         char *name;
...
1217         if (scsi_autopm_get_device(STp->device) < 0) {
1218                 retval = -EIO;
1219                 goto err_out;
1220         }
1221         resumed = 1;
1222         if (!scsi_block_when_processing_errors(STp->device)) {
1223                 retval = (-ENXIO);
1224                 goto err_out;
1225         }
...
1264  err_out:
1265         normalize_buffer(STp->buffer);
1266         spin_lock(&st_use_lock);
1267         STp->in_use = 0;
1268         spin_unlock(&st_use_lock);
1269         scsi_tape_put(STp); <-- STp->device = 0 after this
1270         if (resumed)
1271                 scsi_autopm_put_device(STp->device);
1272         return retval;

The ref count for the struct scsi_tape had already been reduced
to 1 when the .remove method of the st module had been called.
The kref_put() in scsi_tape_put() caused scsi_tape_release()
to be called:

0266 static void scsi_tape_put(struct scsi_tape *STp)
0267 {
0268         struct scsi_device *sdev = STp->device;
0269
0270         mutex_lock(&st_ref_mutex);
0271         kref_put(&STp->kref, scsi_tape_release); <-- calls this
0272         scsi_device_put(sdev);
0273         mutex_unlock(&st_ref_mutex);
0274 }

In scsi_tape_release() the struct scsi_device in the struct
scsi_tape gets set to NULL:

4273 static void scsi_tape_release(struct kref *kref)
4274 {
4275         struct scsi_tape *tpnt = to_scsi_tape(kref);
4276         struct gendisk *disk = tpnt->disk;
4277
4278         tpnt->device = NULL; <<<---- where the dev is nulled
4279
4280         if (tpnt->buffer) {
4281                 normalize_buffer(tpnt->buffer);
4282                 kfree(tpnt->buffer->reserved_pages);
4283                 kfree(tpnt->buffer);
4284         }
4285
4286         disk->private_data = NULL;
4287         put_disk(disk);
4288         kfree(tpnt);
4289         return;
4290 }

Although the problem was reported on SLES11.3 the problem appears
in linux-next as well.

The crash is fixed by reordering the code so we no longer access
the struct scsi_tape after the kref_put() is done on it in st_open().

Signed-off-by: Shane Seymour <shane.seymour@hp.com>
Signed-off-by: Darren Lavender <darren.lavender@hp.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.com>
Acked-by: Kai Mäkisara <kai.makisara@kolumbus.fi>
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Signed-off-by: Kamal Mostafa <kamal@canonical.com>
---
 drivers/scsi/st.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index ff44b3c..9903f1d 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -1262,9 +1262,9 @@ static int st_open(struct inode *inode, struct file *filp)
 	spin_lock(&st_use_lock);
 	STp->in_use = 0;
 	spin_unlock(&st_use_lock);
-	scsi_tape_put(STp);
 	if (resumed)
 		scsi_autopm_put_device(STp->device);
+	scsi_tape_put(STp);
 	return retval;
 
 }
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[3.13.y-ckt stable] Linux 3.13.11-ckt25 stable review Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:40 +0200
  [PATCH 3.13.y-ckt 37/53] USB: serial: Destroy serial_minors IDR on module exit Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 44/53] genirq: Prevent resend to interrupts marked IRQ_NESTED_THREAD Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 01/53] x86/asm/entry/64: Fold the 'test_in_nmi' macro into its only user Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 47/53] bridge: mdb: zero out the local br_ip variable before use Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 49/53] net: graceful exit from netif_alloc_netdev_queues() Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 45/53] ip_tunnel: fix ipv4 pmtu check to honor inner ip header df Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 24/53] iio: adc: at91_adc: allow to use full range of startup time Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 29/53] USB: cp210x: add ID for Aruba Networks controllers Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 28/53] USB: option: add 2020:4000 ID Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 11/53] Btrfs: use kmem_cache_free when freeing entry in inode cache Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 50/53] net: dsa: Fix off-by-one in switch address parsing Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 10/53] sg_start_req(): make sure that there's not too many elements in iovec Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 25/53] ALSA: usb-audio: Add MIDI support for Steinberg MI2/MI4 Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 38/53] Btrfs: fix memory leak in the extent_same ioctl Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 13/53] Btrfs: fix fsync data loss after append write Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 19/53] hpfs: kstrdup() out of memory handling Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 34/53] ARM: dts: mx23: fix iio-hwmon support Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 46/53] bridge: mdb: start delete timer for temp static entries Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 41/53] st: null pointer dereference panic caused by use after kref_put by st_open Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 40/53] s390/process: fix sfpc inline assembly Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 33/53] drm: add a check for x/y in drm_mode_setcrtc Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 15/53] ext4: be more strict when migrating to non-extent based file Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 02/53] x86/asm/entry/64: Remove a redundant jump Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 35/53] tracing: Have branch tracer use recursive field of task struct Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 27/53] dm btree remove: fix bug in redistribute3 Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 32/53] s390/sclp: clear upper register halves in _sclp_print_early Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 30/53] dm btree: silence lockdep lock inversion in dm_btree_del() Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 26/53] iio: tmp006: Check channel info on write Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 36/53] drivers: net: cpsw: fix crash while accessing second slave ethernet interface Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 31/53] usb: musb: host: rely on port_mode to call musb_start() Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 22/53] iio: inv-mpu: Specify the expected format/precision for write channels Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 42/53] drm/radeon: add a dpm quirk for Sapphire Radeon R9 270X 2GB GDDR5 Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 39/53] ARC: make sure instruction_pointer() returns unsigned value Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 23/53] iio: DAC: ad5624r_spi: fix bit shift of output data value Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 14/53] ext4: fix reservation release on invalidatepage for delalloc fs Kamal Mostafa <kamal@canonical.com> - 2015-08-06 22:50 +0200
  [PATCH 3.13.y-ckt 07/53] x86/nmi/64: Reorder nested NMI checks Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 21/53] freeing unlinked file indefinitely delayed Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 08/53] x86/nmi/64: Use DF to avoid userspace RSP confusing nested NMI detection Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 16/53] ext4: correctly migrate a file with a hole at the beginning Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 06/53] x86/nmi/64: Improve nested NMI comments Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 05/53] x86/nmi/64: Switch stacks on userspace NMI entry Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 20/53] 9p: don't leave a half-initialized inode sitting around Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 12/53] Btrfs: fix race between caching kthread and returning inode to inode cache Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 03/53] x86/nmi: Enable nested do_nmi handling for 64-bit kernels Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 18/53] ACPI / PNP: Reserve ACPI resources at the fs_initcall_sync stage Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 17/53] ext4: replace open coded nofail allocation in ext4_free_blocks() Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200
  [PATCH 3.13.y-ckt 09/53] KEYS: ensure we free the assoc array edit if edit is valid Kamal Mostafa <kamal@canonical.com> - 2015-08-06 23:00 +0200

csiph-web