Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.networking > #431 > unrolled thread

Recommended bandwidth measureing tool for 10G

Back to article view | Back to comp.os.linux.networking

Contents

  Recommended bandwidth measureing tool for 10G General Schvantzkoph <schvantzkoph@yahoo.com> - 2011-07-26 23:38 +0000
    Re: Recommended bandwidth measureing tool for 10G Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-07-27 08:12 +0000
      Re: Recommended bandwidth measureing tool for 10G General Schvantzkoph <schvantzkoph@yahoo.com> - 2011-07-27 12:02 +0000
        Re: Recommended bandwidth measureing tool for 10G Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-07-27 15:13 +0000
          Re: Recommended bandwidth measureing tool for 10G General Schvantzkoph <schvantzkoph@yahoo.com> - 2011-07-27 15:36 +0000
            Re: Recommended bandwidth measureing tool for 10G Jorgen Grahn <grahn+nntp@snipabacken.se> - 2011-07-27 20:42 +0000
          Re: Recommended bandwidth measureing tool for 10G Rick Jones <rick.jones2@hp.com> - 2011-07-27 16:33 +0000
            Re: Recommended bandwidth measureing tool for 10G General Schvantzkoph <schvantzkoph@yahoo.com> - 2011-07-27 23:31 +0000
              Re: Recommended bandwidth measureing tool for 10G Rick Jones <rick.jones2@hp.com> - 2011-07-27 23:01 +0000

#431 — Recommended bandwidth measureing tool for 10G

What's the best tool to measure the bandwidth between NIC cards. I need to 
measure the bandwidth between 10G NICs.

[toc] | [next] | [standalone]

#432

On Tue, 2011-07-26, General Schvantzkoph wrote:
> What's the best tool to measure the bandwidth between NIC cards. I need to 
> measure the bandwidth between 10G NICs.

What do you mean?  That you have two NICs, and want to measure the
mean bits/s are going from A to B over a certain time period?

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#434

On Wed, 27 Jul 2011 08:12:47 +0000, Jorgen Grahn wrote:

> On Tue, 2011-07-26, General Schvantzkoph wrote:
>> What's the best tool to measure the bandwidth between NIC cards. I need
>> to measure the bandwidth between 10G NICs.
> 
> What do you mean?  That you have two NICs, and want to measure the mean
> bits/s are going from A to B over a certain time period?
> 
> /Jorgen

Yes that's it exactly. I'm going to try netperf, that looks promising.

[toc] | [prev] | [next] | [standalone]

#435

On Wed, 2011-07-27, General Schvantzkoph wrote:
> On Wed, 27 Jul 2011 08:12:47 +0000, Jorgen Grahn wrote:
>
>> On Tue, 2011-07-26, General Schvantzkoph wrote:
>>> What's the best tool to measure the bandwidth between NIC cards. I need
>>> to measure the bandwidth between 10G NICs.
>> 
>> What do you mean?  That you have two NICs, and want to measure the mean
>> bits/s are going from A to B over a certain time period?
>> 
>> /Jorgen
>
> Yes that's it exactly. I'm going to try netperf, that looks promising.

Isn't netperf a traffic generator? If you just want to see what's
happening in terms of bits/s, you can use e.g. ifstat(1), or even
use ifconfig and calculate the average yourself.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#436

On Wed, 27 Jul 2011 15:13:05 +0000, Jorgen Grahn wrote:

> On Wed, 2011-07-27, General Schvantzkoph wrote:
>> On Wed, 27 Jul 2011 08:12:47 +0000, Jorgen Grahn wrote:
>>
>>> On Tue, 2011-07-26, General Schvantzkoph wrote:
>>>> What's the best tool to measure the bandwidth between NIC cards. I
>>>> need to measure the bandwidth between 10G NICs.
>>> 
>>> What do you mean?  That you have two NICs, and want to measure the
>>> mean bits/s are going from A to B over a certain time period?
>>> 
>>> /Jorgen
>>
>> Yes that's it exactly. I'm going to try netperf, that looks promising.
> 
> Isn't netperf a traffic generator? If you just want to see what's
> happening in terms of bits/s, you can use e.g. ifstat(1), or even use
> ifconfig and calculate the average yourself.
> 
> /Jorgen

ifconfig just tells you the configuration it doesn't do any measurements. 
ifstat gives you stats but it doesn't generate traffic which is necessary 
for measuring the actual bandwidth of the interfaces.

[toc] | [prev] | [next] | [standalone]

#439

On Wed, 2011-07-27, General Schvantzkoph wrote:
> On Wed, 27 Jul 2011 15:13:05 +0000, Jorgen Grahn wrote:
>
>> On Wed, 2011-07-27, General Schvantzkoph wrote:
>>> On Wed, 27 Jul 2011 08:12:47 +0000, Jorgen Grahn wrote:
>>>
>>>> On Tue, 2011-07-26, General Schvantzkoph wrote:
>>>>> What's the best tool to measure the bandwidth between NIC cards. I
>>>>> need to measure the bandwidth between 10G NICs.
>>>> 
>>>> What do you mean?  That you have two NICs, and want to measure the
>>>> mean bits/s are going from A to B over a certain time period?
>>>> 
>>>> /Jorgen
>>>
>>> Yes that's it exactly. I'm going to try netperf, that looks promising.
>> 
>> Isn't netperf a traffic generator? If you just want to see what's
>> happening in terms of bits/s, you can use e.g. ifstat(1), or even use
>> ifconfig and calculate the average yourself.
>> 
>> /Jorgen
>
> ifconfig just tells you the configuration it doesn't do any measurements. 

The one I use on Linux does.

> ifstat gives you stats but it doesn't generate traffic which is necessary 
> for measuring the actual bandwidth of the interfaces.

I interpreted your earlier answer as "I have traffic going and just
want to find out how much there already is".

I also think that with 10Gbit interfaces, you'll hit a limit not in
the /interfaces/ but in the system bus/IP stack/userspace of the
machines themselves.  (Not that it's not interesting to find out.)

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#438

Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:
> On Wed, 2011-07-27, General Schvantzkoph wrote:
> > On Wed, 27 Jul 2011 08:12:47 +0000, Jorgen Grahn wrote:
> >
> >> On Tue, 2011-07-26, General Schvantzkoph wrote:
> >>> What's the best tool to measure the bandwidth between NIC
> >>> cards. I need to measure the bandwidth between 10G NICs.
> >> 
> >> What do you mean?  That you have two NICs, and want to measure
> >> the mean bits/s are going from A to B over a certain time period?
> >> 
> >
> > Yes that's it exactly. I'm going to try netperf, that looks
> > promising.

> Isn't netperf a traffic generator? If you just want to see what's
> happening in terms of bits/s, you can use e.g. ifstat(1), or even
> use ifconfig and calculate the average yourself.

That is correct, netperf is a traffic generator not a traffic monitor.
It will generate traffic between A and B and tell you about the
result.

As there is more to networking than just bandwidth, I'll suggest
TCP_RR tests in addition to TCP_STREAM.

happy benchmarking,

rick jones
-- 
It is not a question of half full or empty - the glass has a leak.
The real question is "Can it be patched?"
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

[toc] | [prev] | [next] | [standalone]

#442

On Wed, 27 Jul 2011 16:33:46 +0000, Rick Jones wrote:

> Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:
>> On Wed, 2011-07-27, General Schvantzkoph wrote:
>> > On Wed, 27 Jul 2011 08:12:47 +0000, Jorgen Grahn wrote:
>> >
>> >> On Tue, 2011-07-26, General Schvantzkoph wrote:
>> >>> What's the best tool to measure the bandwidth between NIC cards. I
>> >>> need to measure the bandwidth between 10G NICs.
>> >> 
>> >> What do you mean?  That you have two NICs, and want to measure the
>> >> mean bits/s are going from A to B over a certain time period?
>> >> 
>> >> 
>> > Yes that's it exactly. I'm going to try netperf, that looks
>> > promising.
> 
>> Isn't netperf a traffic generator? If you just want to see what's
>> happening in terms of bits/s, you can use e.g. ifstat(1), or even use
>> ifconfig and calculate the average yourself.
> 
> That is correct, netperf is a traffic generator not a traffic monitor.
> It will generate traffic between A and B and tell you about the result.
> 
> As there is more to networking than just bandwidth, I'll suggest TCP_RR
> tests in addition to TCP_STREAM.
> 
> happy benchmarking,
> 
> rick jones

What I want to do is be able to generate traffic at wire speed. I'm 
designing a piece of hardware that has a couple of 10G ports on it and we 
need to be able to test it under maximum load. 

[toc] | [prev] | [next] | [standalone]

#443

General Schvantzkoph <schvantzkoph@yahoo.com> wrote:
> What I want to do is be able to generate traffic at wire speed. I'm
> designing a piece of hardware that has a couple of 10G ports on it
> and we need to be able to test it under maximum load.

Assuming your piece of hardware has sufficient CPU "oomph" (I'm
assuming it runs Linux given the news group) then netperf should be
able to generate traffic at wire speed.

Some, but probably not all the factors involved:

*) CPU "oomph" 
*) NIC driver model efficiency/driver path length
*) Stateless offloads in the NIC - both on transmit and receive side
*) Width of path to the NIC (PCIe Gen1/Gen2 x4/x8 etc)
*) Use of a 9000 byte (JumboFrame) vs 1500 byte MTU

When you get to the point of running, joining netperf-talk@netperf.org
might be a good idea.

rick jones

Some boiler-plate I trot-out from time to time:

Looking from the standpoint of the defining IEEE specifications, there
is nothing in 10 Gigabit Ethernet that makes data transmission any
easier on the host than it was for 1 Gigabit Ethernet, or for that
matter 100 or 10 Megabit Ethernet.  It takes just as many CPU cycles
to send a frame through the interface for all of those.

Now, as time has passed, NIC vendors have learned, or been taught by
system vendors :) how to do things *beyond* the IEEE
specifications.

In the time of 100 Megabit Ethernet, it became possible to have fewer
than one interrupt per packet.

In the time of 1 Gigabit Ethernet, various interrupt coalescing
schemes took things farther than they went with 100 Megabit Ethernet.
Also, the mass-market NICs started to support ChecKsum Offload (CKO -
something first done by the then major systems vendors with their FDDI
NICs in the early 1990's) and some started supporting maximum frame
sizes of 9000 bytes - what Alteon dubbed "Jumbo Frames" - a name that
has stuck to this day.

Any 10 Gigabit Ethernet NIC worth its silicon will have all those
features, plus support for directing interrupts to multiple cores and
using multiple packet queues.  This can spread the work across
multiple cores - but only when there are multiple "flows" (eg TCP
connections).  The 10 Gigabit Ethernet NICs also provide support for
TCP Segmentation Offload (TSO) and newer ones also include Large
Receive Offload (LRO - sometimes called Transparant Packet
Aggregation).

Those stateless offloads can very dramatically lower the CPU overhead
of data transfer.  However...

Stateless offloads such as CKO, TSO and LRO really only come into play
when the traffic is a "bulk transfer" - when the application(s)
involved send rather more than an MSS's worth of data at one time.
There are many applications which do so, but not all applications do.
CKO, TSO, and LRO do little or nothing for applications making
discrete, small sends - those applications are basically back to 10
Megabit Ethenet days when it comes to how much CPU will be consumed
sending/receiving their traffic through the NIC.


-- 
denial, anger, bargaining, depression, acceptance, rebirth...
                                     where do you want to be today?
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com  but NOT BOTH...

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.networking

csiph-web