Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1689548

Re: [PATCH] nvme: Acknowledge completion queue on each iteration

Show all headers | View raw

Hi Keith,

On 7/17/2017 6:45 PM, Keith Busch wrote:
> On Mon, Jul 17, 2017 at 06:36:23PM -0400, Sinan Kaya wrote:
>> Code is moving the completion queue doorbell after processing all completed
>> events and sending callbacks to the block layer on each iteration.
>>
>> This is causing a performance drop when a lot of jobs are queued towards
>> the HW. Move the completion queue doorbell on each loop instead and allow new
>> jobs to be queued by the HW.
> 
> That doesn't make sense. Aggregating doorbell writes should be much more
> efficient for high depth workloads.
> 

Problem is that code is throttling the HW as HW cannot queue more completions until
SW get a chance to clear it. 

As an example:

for each in N
(
	blk_layer()
)
ring door bell

HW cannot queue new job until N x blk_layer operations are processed and queue
element ownership is passed to the HW after the loop. HW is just sitting idle
there if no queue entries are available.

Sinan

-- 
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH] nvme: Acknowledge completion queue on each iteration Sinan Kaya <okaya@codeaurora.org> - 2017-07-18 00:40 +0200
  Re: [PATCH] nvme: Acknowledge completion queue on each iteration Keith Busch <keith.busch@intel.com> - 2017-07-18 00:40 +0200
    Re: [PATCH] nvme: Acknowledge completion queue on each iteration Sinan Kaya <okaya@codeaurora.org> - 2017-07-18 00:50 +0200
      Re: [PATCH] nvme: Acknowledge completion queue on each iteration Keith Busch <keith.busch@intel.com> - 2017-07-18 01:00 +0200
        Re: [PATCH] nvme: Acknowledge completion queue on each iteration okaya@codeaurora.org - 2017-07-18 01:10 +0200
          Re: [PATCH] nvme: Acknowledge completion queue on each iteration Keith Busch <keith.busch@intel.com> - 2017-07-18 16:40 +0200
            Re: [PATCH] nvme: Acknowledge completion queue on each iteration Sinan Kaya <okaya@codeaurora.org> - 2017-07-18 21:00 +0200
              Re: [PATCH] nvme: Acknowledge completion queue on each iteration Keith Busch <keith.busch@intel.com> - 2017-07-18 23:30 +0200
  Re: [PATCH] nvme: Acknowledge completion queue on each iteration Sagi Grimberg <sagi@grimberg.me> - 2017-07-19 11:30 +0200
    Re: [PATCH] nvme: Acknowledge completion queue on each iteration Sinan Kaya <okaya@codeaurora.org> - 2017-07-19 12:40 +0200

csiph-web