Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1229519

Re: [PATCH 00/15] RDS: connection scalability and performance improvements

Show all headers | View raw

On 9/20/2015 1:37 AM, Sagi Grimberg wrote:
> On 9/20/2015 2:04 AM, Santosh Shilimkar wrote:
>> This series addresses RDS connection bottlenecks on massive workloads and
>> improve the RDMA performance almost by 3X. RDS TCP also gets a small gain
>> of about 12%.
>>
>> RDS is being used in massive systems with high scalability where several
>> hundred thousand end points and tens of thousands of local processes
>> are operating in tens of thousand sockets. Being RC(reliable connection),
>> socket bind and release happens very often and any inefficiencies in
>> bind hash look ups hurts the overall system performance. RDS bin
>> hash-table
>> uses global spin-lock which is the biggest bottleneck. To make matter
>> worst,
>> it uses rcu inside global lock for hash buckets.
>> This is being addressed by simply using per bucket rw lock which makes
>> the
>> locking simple and very efficient. The hash table size is also scaled up
>> accordingly.
>>
>> For RDS RDMA improvement, the completion handling is revamped so that we
>> can do batch completions. Both send and receive completion handlers are
>> split logically to achieve the same. RDS 8K messages being one of the
>> key usecase, mr pool is adapted to have the 8K mrs along with default 1M
>> mrs. And while doing this, few fixes and couple of bottlenecks seen with
>> rds_sendmsg() are addressed.
>
> Hi Santosh,
>
> I think that can get a more effective code review if you CC the
> Linux-rdma mailing list.
>
I will do that from next time. Thanks Sagi !!

Regards,
Santosh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

[PATCH 00/15] RDS: connection scalability and performance improvements Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 13/15] RDS: mark rds_ib_fmr_wq static Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 06/15] RDS: defer the over_batch work to send worker Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 04/15] RDS: Use per-bucket rw lock for bind hash-table Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 08/15] RDS: ack more receive completions to improve performance Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 15/15] RDS: split mr pool to improve 8K messages performance Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 14/15] RDS: use max_mr from HCA caps than max_fmr Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 12/15] RDS: use already available pool handle from ibmr Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 03/15] RDS: fix rds_sock reference bug while doing bind Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 07/15] RDS: use rds_send_xmit() state instead of RDS_LL_SEND_FULL Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 05/15] RDS: increase size of hash-table to 8K Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
    RE: [PATCH 05/15] RDS: increase size of hash-table to 8K David Laight <David.Laight@ACULAB.COM> - 2015-09-21 10:40 +0200
      Re: [PATCH 05/15] RDS: increase size of hash-table to 8K santosh shilimkar <santosh.shilimkar@oracle.com> - 2015-09-21 18:00 +0200
    Re: [PATCH 05/15] RDS: increase size of hash-table to 8K David Miller <davem@davemloft.net> - 2015-09-22 01:10 +0200
      Re: [PATCH 05/15] RDS: increase size of hash-table to 8K santosh shilimkar <santosh.shilimkar@oracle.com> - 2015-09-22 02:00 +0200
        Re: [PATCH 05/15] RDS: increase size of hash-table to 8K santosh shilimkar <santosh.shilimkar@oracle.com> - 2015-09-23 20:10 +0200
  [PATCH 02/15] RDS: make socket bind/release locking scheme simple and more efficient Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:10 +0200
  [PATCH 10/15] RDS: handle rds_ibdev release case instead of crashing the kernel Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:20 +0200
  [PATCH 01/15] RDS: use kfree_rcu in rds_ib_remove_ipaddr Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:20 +0200
  [PATCH 11/15] RDS: fix the rds_ib_fmr_wq kick call Santosh Shilimkar <santosh.shilimkar@oracle.com> - 2015-09-20 01:20 +0200
  Re: [PATCH 00/15] RDS: connection scalability and performance  improvements Sagi Grimberg <sagig@dev.mellanox.co.il> - 2015-09-20 10:40 +0200
    Re: [PATCH 00/15] RDS: connection scalability and performance  improvements santosh shilimkar <santosh.shilimkar@oracle.com> - 2015-09-21 18:10 +0200

csiph-web