Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #11951 > unrolled thread

How to map NIC to cpu socket?

Back to article view | Back to comp.os.linux.misc

Contents

  How to map NIC to cpu socket? chrippa08@gmail.com - 2014-09-05 06:01 -0700
    Re: How to map NIC to cpu socket? Joe Beanfish <joebeanfish@nospam.duh> - 2014-09-05 13:09 +0000
      Re: How to map NIC to cpu socket? Robert Heller <heller@deepsoft.com> - 2014-09-05 08:51 -0500
        Re: How to map NIC to cpu socket? chrippa08@gmail.com - 2014-09-05 07:49 -0700
          Re: How to map NIC to cpu socket? Robert Heller <heller@deepsoft.com> - 2014-09-05 11:15 -0500
          Re: How to map NIC to cpu socket? "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2014-09-05 12:15 -0400
          Re: How to map NIC to cpu socket? William Unruh <unruh@invalid.ca> - 2014-09-05 20:48 +0000
            Re: How to map NIC to cpu socket? chrippa08@gmail.com - 2014-09-05 23:34 -0700
              Re: How to map NIC to cpu socket? Wayne <nospam@all.invalid> - 2014-09-06 18:52 -0400
                Re: How to map NIC to cpu socket? "David W. Hodgins" <dwhodgins@nomail.afraid.org> - 2014-09-06 20:32 -0400
                  Re: How to map NIC to cpu socket? Aragorn <thorongil@telenet.be.invalid> - 2014-09-07 14:19 +0200
                    Re: How to map NIC to cpu socket? Wayne <nospam@all.invalid> - 2014-09-08 04:41 -0400
                      Re: How to map NIC to cpu socket? Aragorn <thorongil@telenet.be.invalid> - 2014-09-08 14:07 +0200
                Re: How to map NIC to cpu socket? chrippa08@gmail.com - 2014-09-08 03:05 -0700

#11951 — How to map NIC to cpu socket?

Hi,

I have a number of NICs in a machine and two cpus and hence two cpu sockets.

How do I find out which NIC is connected to which cpu (socket) without physically opening the machine?

Brs

[toc] | [next] | [standalone]

#11952

On Fri, 05 Sep 2014 06:01:22 -0700, chrippa08 wrote:
> Hi,
> 
> I have a number of NICs in a machine and two cpus and hence two cpu
> sockets.
> 
> How do I find out which NIC is connected to which cpu (socket) without
> physically opening the machine?
> 
> Brs

What makes you think there's a direct one to one connection between
cpu socket and nic? In the general case either cpu can talk to either nic.

[toc] | [prev] | [next] | [standalone]

#11956

At Fri, 5 Sep 2014 13:09:30 +0000 (UTC) Joe Beanfish <joebeanfish@nospam.duh> wrote:

> 
> On Fri, 05 Sep 2014 06:01:22 -0700, chrippa08 wrote:
> > Hi,
> > 
> > I have a number of NICs in a machine and two cpus and hence two cpu
> > sockets.
> > 
> > How do I find out which NIC is connected to which cpu (socket) without
> > physically opening the machine?
> > 
> > Brs
> 
> What makes you think there's a direct one to one connection between
> cpu socket and nic? In the general case either cpu can talk to either nic.

Unless the OP is really talking about a blade chassis with two blade systems 
plugged in.  In which what he means by 'cpus' is 'blades' and by 'cpu sockets' 
he means 'blade sockets'.  But I really doubt this is what he is talking 
about.  Generally, a motherboard with multiple processor sockets will have all 
of the sockets connected to a support chip (or chips) that implements a SMP 
relationship amoungst the processors (2, 3, 4, 8, etc.), and with multi-core 
processors, you end up with n X m total cores (n == # sockets, m # 
cores/socket).  This whole 'array' pf cores is connected to the RAM controller 
and the various bus controllers (PCI, USB, whatever).  The NICs are on the PCI 
bus, either internaly in the system support 'chip' (or chips), or externally 
in a PCI(-<mumble>) card slot.  Thus all cores can talk to all memory and all 
PCI devices.  (Well, really, the O/S kernel sorts out who can talk to what at 
any given instance in time, depending on how things are scheduled.)

>                                                                                                                

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller@deepsoft.com       -- Webhosting Services
 

[toc] | [prev] | [next] | [standalone]

#11960

> > > Hi,
> 
> > > 
> 
> > > I have a number of NICs in a machine and two cpus and hence two cpu
> 
> > > sockets.
> 
> > > 
> 
> > > How do I find out which NIC is connected to which cpu (socket) without
> 
> > > physically opening the machine?
> 
> > > 
> 
> > > Brs
> 
> > 
> 
> > What makes you think there's a direct one to one connection between
> 
> > cpu socket and nic? In the general case either cpu can talk to either nic.
> 
> 
> 
> Unless the OP is really talking about a blade chassis with two blade systems 
> 
> plugged in.  In which what he means by 'cpus' is 'blades' and by 'cpu sockets' 
> 
> he means 'blade sockets'.  But I really doubt this is what he is talking 
> 
> about.  Generally, a motherboard with multiple processor sockets will have all 
> 
> of the sockets connected to a support chip (or chips) that implements a SMP 
> 
> relationship amoungst the processors (2, 3, 4, 8, etc.), and with multi-core 
> 
> processors, you end up with n X m total cores (n == # sockets, m # 
> 
> cores/socket).  This whole 'array' pf cores is connected to the RAM controller 
> 
> and the various bus controllers (PCI, USB, whatever).  The NICs are on the PCI 
> 
> bus, either internaly in the system support 'chip' (or chips), or externally 
> 
> in a PCI(-<mumble>) card slot.  Thus all cores can talk to all memory and all 
> 
> PCI devices.  (Well, really, the O/S kernel sorts out who can talk to what at 
> 
> any given instance in time, depending on how things are scheduled.)
> 
> 

I am talking about a single blade, with two physical sockets and a CPU mounted in each.

Here is the ouptut of lscpu

lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                40
On-line CPU(s) list:   0-39
Thread(s) per core:    2
Core(s) per socket:    10
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 62
Stepping:              4
CPU MHz:               1200.000
BogoMIPS:              5601.29
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              25600K
NUMA node0 CPU(s):     0-9,20-29
NUMA node1 CPU(s):     10-19,30-39

Network cards:

lspci |grep -i ethernet
05:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
05:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
21:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
21:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 

How do I map the NIC to socket?

[toc] | [prev] | [next] | [standalone]

#11962

At Fri, 5 Sep 2014 07:49:50 -0700 (PDT) chrippa08@gmail.com wrote:

> 
> > > > Hi,
> > 
> > > > 
> > 
> > > > I have a number of NICs in a machine and two cpus and hence two cpu
> > 
> > > > sockets.
> > 
> > > > 
> > 
> > > > How do I find out which NIC is connected to which cpu (socket) without
> > 
> > > > physically opening the machine?
> > 
> > > > 
> > 
> > > > Brs
> > 
> > > 
> > 
> > > What makes you think there's a direct one to one connection between
> > 
> > > cpu socket and nic? In the general case either cpu can talk to either nic.
> > 
> > 
> > 
> > Unless the OP is really talking about a blade chassis with two blade systems 
> > 
> > plugged in.  In which what he means by 'cpus' is 'blades' and by 'cpu sockets' 
> > 
> > he means 'blade sockets'.  But I really doubt this is what he is talking 
> > 
> > about.  Generally, a motherboard with multiple processor sockets will have all 
> > 
> > of the sockets connected to a support chip (or chips) that implements a SMP 
> > 
> > relationship amoungst the processors (2, 3, 4, 8, etc.), and with multi-core 
> > 
> > processors, you end up with n X m total cores (n == # sockets, m # 
> > 
> > cores/socket).  This whole 'array' pf cores is connected to the RAM controller 
> > 
> > and the various bus controllers (PCI, USB, whatever).  The NICs are on the PCI 
> > 
> > bus, either internaly in the system support 'chip' (or chips), or externally 
> > 
> > in a PCI(-<mumble>) card slot.  Thus all cores can talk to all memory and all 
> > 
> > PCI devices.  (Well, really, the O/S kernel sorts out who can talk to what at 
> > 
> > any given instance in time, depending on how things are scheduled.)
> > 
> > 
> 
> I am talking about a single blade, with two physical sockets and a CPU mounted in each.
> 
> Here is the ouptut of lscpu
> 
> lscpu
> Architecture:          x86_64
> CPU op-mode(s):        32-bit, 64-bit
> Byte Order:            Little Endian
> CPU(s):                40
> On-line CPU(s) list:   0-39
> Thread(s) per core:    2
> Core(s) per socket:    10
> Socket(s):             2
> NUMA node(s):          2
> Vendor ID:             GenuineIntel
> CPU family:            6
> Model:                 62
> Stepping:              4
> CPU MHz:               1200.000
> BogoMIPS:              5601.29
> Virtualization:        VT-x
> L1d cache:             32K
> L1i cache:             32K
> L2 cache:              256K
> L3 cache:              25600K
> NUMA node0 CPU(s):     0-9,20-29
> NUMA node1 CPU(s):     10-19,30-39
> 
> Network cards:
> 
> lspci |grep -i ethernet
> 05:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
> 05:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
> 21:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
> 21:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
> 
> How do I map the NIC to socket?

Meaningless question. There is no mapping of NIC to *CPU* socket. All of the
CPUs can 'see' all of the NICs.

*Unless* you mean mapping RJ45 connectors to NICs. That is a completely
*different* can of worms. I'm afraid you are going to have to 'easter egg' it.
That is, pick a RJ45 connector and plug in a CAT5 (10/100BaseT network) or
CAT6 (1000BaseT network) in and connect it to a router or switch. Then attempt
to ifup/ifconfig each of the ethernet devs (eg eth0, ... eth3 or em1, ... em4,
or whatever). When the lucky port comes up and works, you then know which NIC
that RJ45 connector is connected to. Note the MAC address, etc. and move on to
the next RJ45 connector.  Note: the *documentation* (wow what a concept: you 
could just *read* the little booklet that came with the system) might tell you 
how the RJ45 connectors map to NICs.

>                                            

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller@deepsoft.com       -- Webhosting Services
  

[toc] | [prev] | [next] | [standalone]

#11963

On Fri, 05 Sep 2014 10:49:50 -0400, <chrippa08@gmail.com> wrote:

> lspci |grep -i ethernet
> 05:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
> 05:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
> 21:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
> 21:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
> How do I map the NIC to socket?

You can't. The nics are attched to the pci bus, which either cpu can
access. As all of the nics have the same model, the same kernel module
will be used for all of them. Which cpu is used while executing the
code in the module, is up to the kernel. It may use both, or just one
of them.

Regards, Dave Hodgins

-- 
Change nomail.afraid.org to ody.ca to reply by email.
(nomail.afraid.org has been set up specifically for
use in usenet. Feel free to use it yourself.)

[toc] | [prev] | [next] | [standalone]

#11967

On 2014-09-05, chrippa08@gmail.com <chrippa08@gmail.com> wrote:
>> > > Hi,
>> 

Stop using google as your news provider. They stick in a blank line
between every line in the quote. It is unreadable. 

>
> I am talking about a single blade, with two physical sockets and a CPU mounted in each.
>

As stated, cpus and wireless cards attach to buses, so anthing can talk
to anything. Which talks to which is determined by software not hardware
(unelss you have a really weird system). 

> Here is the ouptut of lscpu
>
> lscpu
> Architecture:          x86_64
> CPU op-mode(s):        32-bit, 64-bit
> Byte Order:            Little Endian
> CPU(s):                40
> On-line CPU(s) list:   0-39
> Thread(s) per core:    2
> Core(s) per socket:    10
> Socket(s):             2
> NUMA node(s):          2
> Vendor ID:             GenuineIntel
> CPU family:            6
> Model:                 62
> Stepping:              4
> CPU MHz:               1200.000
> BogoMIPS:              5601.29
> Virtualization:        VT-x
> L1d cache:             32K
> L1i cache:             32K
> L2 cache:              256K
> L3 cache:              25600K
> NUMA node0 CPU(s):     0-9,20-29
> NUMA node1 CPU(s):     10-19,30-39
>
> Network cards:
>
> lspci |grep -i ethernet
> 05:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port
> 05:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
> 21:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
> 21:00.1 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port 
>
> How do I map the NIC to socket?

You do not. the OS software does that. What do you want to do? Why ask
us how to accomplish your objective in some specific manner  rather than
telling us what your objective is and let us suggest possibilities that
might actually work.

[toc] | [prev] | [next] | [standalone]

#11979

Each socket/cpu seem to have its own PCIe interface and NICs can be connected to eihter PCIe. 
The two sockets/cpus are connected via QPI. 
So traffic coming in on one NIC could potentially be destined to the "other" cpu and hence need to go via the QPI which reduces the throughput.

Maybe the following picture is appropriate:

http://www.theinquirer.net/IMG/174/119174/sandybrdiag1-540x540.png?1288098400

So when implementing a packet handling application it will perform better if the application is running on the cpu/socket where the NIC is directly connected to over PCIe.

So in order to know on which cpu/socket to launch the packet handling application for a NIC, how do I map the NIC to cpu/socket?

I found the following, but it was not that straightforward to understand:
http://dpdk.org/ml/archives/dev/2013-December/000988.html

[toc] | [prev] | [next] | [standalone]

#11994

On 9/6/2014 2:34 AM, chrippa08@gmail.com wrote:
> 
> Each socket/cpu seem to have its own PCIe interface and NICs can be connected to eihter PCIe. 
> The two sockets/cpus are connected via QPI. 
> So traffic coming in on one NIC could potentially be destined to the "other" cpu and hence need to go via the QPI which reduces the throughput.
> 
> Maybe the following picture is appropriate:
> 
> http://www.theinquirer.net/IMG/174/119174/sandybrdiag1-540x540.png?1288098400
> 
> So when implementing a packet handling application it will perform better if the application is running on the cpu/socket where the NIC is directly connected to over PCIe.
> 
> So in order to know on which cpu/socket to launch the packet handling application for a NIC, how do I map the NIC to cpu/socket?
> 
> I found the following, but it was not that straightforward to understand:
> http://dpdk.org/ml/archives/dev/2013-December/000988.html
> 

After reading that link, and the Intel DDIO FAQ, I think I
understand what you want.

Intel DDIO sends NIC data direct to a NUMA node's cache, not
main memory.  If the NIC is attached to the NUMA node (socket)
that uses it, throughput may be doubled (no RAM access needed
at all).  Otherwise, the NIC must send the data to a remote
NUMA node, which looses at least some of the benefit.

To make this work well, you would need to know which CPU was
processing the packets from which NIC.  You could then attach the
appropriate network to the appropriate NIC, to gain considerable
throughput.

The archived email you linked to explains how to tell to which
NUMA node (socket) a particular NIC (say eth0) is attached:

   $ cat /sys/class/net/eth0/device/numa_node

(Since my system doesn't have NUMA, it returns "-1" for me.)

The other calculations in that email only are needed if you are
running an old kernel version.

Intel has a short PDF showing how to visually determine this for xeon;
probably it's the same for other models:
<http://www.intel.com/content/dam/www/public/us/en/documents/application-notes/xeon-e5-ddio-appl-notes.pdf>

Info about NUMA:
<https://software.intel.com/en-us/articles/optimizing-applications-for-numa?wapkw=optimizing%20software%20applications%20for%20numa>

<http://www.akkadia.org/drepper/cpumemory.pdf>

-- 
Wayne

[toc] | [prev] | [next] | [standalone]

#11995

On Sat, 06 Sep 2014 18:52:00 -0400, Wayne <nospam@all.invalid> wrote:

> Info about NUMA:
> <https://software.intel.com/en-us/articles/optimizing-applications-for-numa?wapkw=optimizing%20software%20applications%20for%20numa>

Thanks for that link. I was not aware of numa, and don't have it on any
of the systems I have access to, hence my earlier comment about the nic
being accessed over the pci bus, by any cpu.

The op's question does make sense now, assuming the system in use does
support numa.

Regards, Dave Hodgins

-- 
Change nomail.afraid.org to ody.ca to reply by email.
(nomail.afraid.org has been set up specifically for
use in usenet. Feel free to use it yourself.)

[toc] | [prev] | [next] | [standalone]

#11997

On Sunday 07 September 2014 02:32, David W. Hodgins conveyed the 
following to comp.os.linux.misc...

> On Sat, 06 Sep 2014 18:52:00 -0400, Wayne <nospam@all.invalid> wrote:
> 
>> Info about NUMA:
>> <https://software.intel.com/en-us/articles/optimizing-applications-
for-numa?wapkw=optimizing%20software%20applications%20for%20numa>
> 
> Thanks for that link. I was not aware of numa, and don't have it on
> any of the systems I have access to, hence my earlier comment about
> the nic being accessed over the pci bus, by any cpu.
> 
> The op's question does make sense now, assuming the system in use does
> support numa.

I own a NUMA machine - not in use at the moment - based upon a Tyan 
motherboard and second generation AMD Opteron processors, and it has two 
NICs, on on each node [*].  

As I understand it, the kernel will handle the NUMA layout based upon 
the information it receives from the BIOS.  Linux has already had 
excellent NUMA support for a very long time.  If I'm not mistaken, then 
that was already implemented somewhere early on during the lifetime of 
the 2.6 kernel generation.  

If one builds one's own kernels from sources, then one can choose 
whether NUMA should be used or whether a flat memory model is desirable, 
but distro kernels are typically compiled with NUMA support, in which 
case Linux will detect whether the system really /is/ a NUMA 
architecture - this also depends on BIOS settings, as the BIOS can 
disable the NUMA layout and implement plain SMP instead - and if the 
system isn't configured for NUMA, then Linux will simply treat the whole 
memory layout and processor architecture as being a single NUMA node, 
which works just as well as the flat memory model.  Performance might be 
marginally less than optimal with this approach, but the difference will 
be smaller than that of the performance penalty associated with using 
generic "lowest common denominator" x86(-64) code rather than processor 
brand- and processor type-specific register optimizations.


[*] It's a bit ironic that such an article should appear on Intel's
    website, given that AMD has been implementing all their own AMD64
    processors as NUMA from the start, while Intel waited with
    implementing NUMA on x86-64 until the Core i series and kept on
    implementing all their multisocket x86-64 processors as plain old
    SMP platforms up until and including the Core 2 series.  (Intel
    did already have experience at building NUMA boards, but they were
    very rare in the x86 market.)

-- 
= Aragorn =

         http://www.linuxcounter.net - registrant #223157

[toc] | [prev] | [next] | [standalone]

#12010

On 9/7/2014 8:19 AM, Aragorn wrote:
> On Sunday 07 September 2014 02:32, David W. Hodgins conveyed the 
> following to comp.os.linux.misc...
> 
>> On Sat, 06 Sep 2014 18:52:00 -0400, Wayne <nospam@all.invalid> wrote:
>>
>>> Info about NUMA:
>>> <https://software.intel.com/en-us/articles/optimizing-applications-
> for-numa?wapkw=optimizing%20software%20applications%20for%20numa>
>>
>> Thanks for that link. I was not aware of numa, and don't have it on
>> any of the systems I have access to, hence my earlier comment about
>> the nic being accessed over the pci bus, by any cpu.
>>
>> The op's question does make sense now, assuming the system in use does
>> support numa.
> 
> I own a NUMA machine - not in use at the moment - based upon a Tyan 
> motherboard and second generation AMD Opteron processors, and it has two 
> NICs, on on each node [*].  
> 
> As I understand it, the kernel will handle the NUMA layout based upon 
> the information it receives from the BIOS.  Linux has already had 
> excellent NUMA support for a very long time.  If I'm not mistaken, then 
> that was already implemented somewhere early on during the lifetime of 
> the 2.6 kernel generation.  

What I don't get is that NICs don't pass data directly to applications.
So unless you have a kernel (or part of one) running on every
NUMA node, the packet headers from all but one NIC will still need
to be accessed by the remote node, where the kernel is.  So remote
access can't be avoided, is that right?

-- 
Wayne

[toc] | [prev] | [next] | [standalone]

#12015

On Monday 08 September 2014 10:41, Wayne conveyed the following to 
comp.os.linux.misc...

> On 9/7/2014 8:19 AM, Aragorn wrote:
>
>> On Sunday 07 September 2014 02:32, David W. Hodgins conveyed the
>> following to comp.os.linux.misc...
>> 
>>> On Sat, 06 Sep 2014 18:52:00 -0400, Wayne <nospam@all.invalid>
>>> wrote:
>>>
>>>> Info about NUMA:
>>>> <https://software.intel.com/en-us/articles/optimizing-applications-
>> for-numa?wapkw=optimizing%20software%20applications%20for%20numa>
>>>
>>> Thanks for that link. I was not aware of numa, and don't have it on
>>> any of the systems I have access to, hence my earlier comment about
>>> the nic being accessed over the pci bus, by any cpu.
>>>
>>> The op's question does make sense now, assuming the system in use
>>> does support numa.
>> 
>> I own a NUMA machine - not in use at the moment - based upon a Tyan
>> motherboard and second generation AMD Opteron processors, and it has
>> two NICs, one on each node [*].
>> 
>> As I understand it, the kernel will handle the NUMA layout based upon
>> the information it receives from the BIOS.  Linux has already had
>> excellent NUMA support for a very long time.  If I'm not mistaken,
>> then that was already implemented somewhere early on during the
>> lifetime of the 2.6 kernel generation.
> 
> What I don't get is that NICs don't pass data directly to
> applications.

In a monolithic kernel like Linux, that is not possible.  The whole 
networking stack runs as part of the kernel.  

Something like the GNU Mach/HURD on the other hand might be able to do 
just that, because that's a microkernel-based system and thus it runs 
its entire networking stack in userspace.  However, I don't have any 
experience with GNU/HURD, and I'm also not a kernel hacker, so don't 
take my word for it.

> So unless you have a kernel (or part of one) running on every NUMA
> node, the packet headers from all but one NIC will still need
> to be accessed by the remote node, where the kernel is.  So remote
> access can't be avoided, is that right?

The kernel will normally try as much as possible to run userspace 
processes on the same node as where the proces' data is stored in 
memory.  However, each CPU - by which I mean "each core", because a 
single node may contain more than one core - has four privilege rings, 
and thus of course also a kernel ring.

Now, provided that the BIOS is set up to initialize the APICs in a 
discrete way - which is the best choice for NUMA systems with NICS 
connected to different nodes - the kernel will have to execute some of 
its code on either node.  By consequence, you will always have a bit of 
remote access going on, given that the kernel itself runs in a single 
address space and will thus most likely be contained within the RAM of a 
single NUMA node.

Now, say that you start up a web browser.  This will typically connect 
to the WAN-side of the machine.  However, the kernel does not know about 
that.  What the kernel does know is what load each of the nodes (and 
their respective cores) are bearing at the moment, and how much memory 
load is available in the RAM of either node.  It will thus try to 
balance the load by putting the application in the most suitable node, 
and having its code execute on the processor cores of that node.  Now, 
if that node requires access to the networking stack, then the kernel 
will check on what interface is connected to which node and will thus 
run the pertinent code where it is needed.

Remote access is inevitable, and this is one of the main reasons why 
such remote access is possible in NUMA systems.  The Linux NUMA 
scheduling is pretty good, so I wouldn't worry about it. ;-)

-- 
= Aragorn =

         http://www.linuxcounter.net - registrant #223157

[toc] | [prev] | [next] | [standalone]

#12013

If "/sys/class/net/eth0/device/numa_node" is available it is straightforward to use it together with lscpu to map numa socket to core.

However, if not available, the link http://dpdk.org/ml/archives/dev/2013-December/000988.html , says to use "lspci -t" as a fallback to map socket to core, however, can someone explain this so it's easier to understand?

Also, http://www.intel.com/content/dam/www/public/us/en/documents/application-notes/xeon-e5-ddio-appl-notes.pdf talks about how to map the socket to core using /proc/cpuinfo and apicid, can someone explain this with an example?

Brs

[toc] | [prev] | [standalone]

Back to top | Article view | comp.os.linux.misc

csiph-web