Discussion:
More fine-grained NUMA knobs
John Baldwin
2016-03-23 00:13:05 UTC
Permalink
Some of the I/O device affinity stuff such as bus_get_domain() (and the
bus_get_cpus() I have in review in D5519) are useful on their own right even
if the VM system is not doing NUMA-aware allocations. I think it would be
useful to be able to enable these two "prongs" of NUMA awareness
independently. To that end, I have a little strawman patch that adds two
new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA. I actually think it is
probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
default MAXMEMDOM to, say, 8 (quad-socket haswell)).

You can see the simple patch at:

https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts

Thoughts?
--
John Baldwin
Adrian Chadd
2016-03-23 03:15:16 UTC
Permalink
+1 by me!


-a
Post by John Baldwin
Some of the I/O device affinity stuff such as bus_get_domain() (and the
bus_get_cpus() I have in review in D5519) are useful on their own right even
if the VM system is not doing NUMA-aware allocations. I think it would be
useful to be able to enable these two "prongs" of NUMA awareness
independently. To that end, I have a little strawman patch that adds two
new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA. I actually think it is
probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
default MAXMEMDOM to, say, 8 (quad-socket haswell)).
https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
Thoughts?
--
John Baldwin
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
Jack Vogel
2016-03-23 06:39:48 UTC
Permalink
Cool! Glad to see this happening :)
Post by Adrian Chadd
+1 by me!
-a
Post by John Baldwin
Some of the I/O device affinity stuff such as bus_get_domain() (and the
bus_get_cpus() I have in review in D5519) are useful on their own right
even
Post by John Baldwin
if the VM system is not doing NUMA-aware allocations. I think it would
be
Post by John Baldwin
useful to be able to enable these two "prongs" of NUMA awareness
independently. To that end, I have a little strawman patch that adds two
new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA. I actually think it
is
Post by John Baldwin
probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
default MAXMEMDOM to, say, 8 (quad-socket haswell)).
https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
Thoughts?
--
John Baldwin
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-arch
Konstantin Belousov
2016-03-23 07:23:16 UTC
Permalink
Post by John Baldwin
Some of the I/O device affinity stuff such as bus_get_domain() (and the
bus_get_cpus() I have in review in D5519) are useful on their own right even
if the VM system is not doing NUMA-aware allocations. I think it would be
useful to be able to enable these two "prongs" of NUMA awareness
independently. To that end, I have a little strawman patch that adds two
new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA. I actually think it is
probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
default MAXMEMDOM to, say, 8 (quad-socket haswell)).
https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
Thoughts?
I do not like that you check both defined(xxx_NUMA) and MAXMEMDOM > 1.
IMO the > 1 part should be dropped. I do not see why would it be useful
even as optimization.

Otherwise, this looks good.

Loading...