[mvapich-discuss] HWLOC warnings

[David Winslow]

We just upgraded our servers from Centos 6.5 to 7.2. With the upgrade, we recompiled MVAPICH2.2.2b using the same method as before. We have two types of serves: Older Dells and fairly new Supermicros. When I run the below osu_bw on the Dell machines, it works as it always did; however, on our Supermicros with AMD Opteron Processor 6344 (48 cores), we now get a HWLOC message. Our software appears to work, but this error never showed up before to upgrade of the OS.

I’m not sure what is wrong; I see this error has occurred with others and may be a kernel bug or BIOS problems. I’ve tried updating the kernel to the latest from Centos (3.10.0-327.3.1.el7.x86_64) and that didn’t solve it. I’ve even upgraded it to 4.4.3. I still see a similar error when I run hwloc-info.

Question. If i compile with "—without-hwloc”, I suspect that the error will go away but how would CPUs then be mapped? Where would MVAPICH2 get the information? Is running without hwloc problematic? 


mpirun -demux select -np 2 -hostfile /home/david.winslow/DISTRIBUTED_COMPUTING/hostfile -genv MV2_ENABLE_AFFINITY=0 -genv IPATH_NO_CPUAFFINITY=1 -genv MV2_DEBUG_SHOW_BACKTRACE=1 /opt/mvapich2-2.2b-install-psm/libexec/mvapich2/osu_bw
****************************************************************************
* hwloc 1.11.0rc3-git has encountered what looks like an error from the operating system.
*
* L3 (cpuset 0x000003f0) intersects with NUMANode (P#0 cpuset 0x0000003f) without inclusion!
* Error occurred in topology.c line 983
*
* The following FAQ entry in the hwloc documentation may help:
*   What should I do when hwloc reports "operating system" warnings?
* Otherwise please report this error message to the hwloc user's mailing list,
* along with the output+tarball generated by the hwloc-gather-topology script.
****************************************************************************
Authorized uses only. All activity may be monitored and reported.
****************************************************************************
* hwloc 1.11.0rc3-git has encountered what looks like an error from the operating system.
*
* L3 (cpuset 0x000003f0) intersects with NUMANode (P#0 cpuset 0x0000003f) without inclusion!
* Error occurred in topology.c line 983
*
* The following FAQ entry in the hwloc documentation may help:
*   What should I do when hwloc reports "operating system" warnings?
* Otherwise please report this error message to the hwloc user's mailing list,
* along with the output+tarball generated by the hwloc-gather-topology script.
****************************************************************************
# OSU MPI Bandwidth Test
# Size      Bandwidth (MB/s)
1                       0.96
2                       1.93
4                       4.53
8                       9.21
16                     17.20
32                     34.98
64                     69.60
128                   136.12
256                   273.36
512                   513.37
1024                  874.99
2048                 1369.63
4096                 2143.63
8192                 1836.13
16384                2776.28
32768                2837.68
65536                2814.84
131072               2849.19
262144               2871.47
524288               2881.64
1048576              2886.59
2097152              2888.57
4194304              2891.03

output of mpirun —version

HYDRA build details:
    Version:                                 3.1.4
    Release Date:                            Thu Nov 12 06:32:40 EST 2015
    CC:                              gcc
    CXX:                             g++
    F77:
    F90:
    Configure options:                       '--disable-option-checking' '--prefix=/opt/mvapich2-2.2b-install-psm' '--with-device=ch3:psm' '--disable-fortran' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'CFLAGS= -DNDEBUG -DNVALGRIND -O2' 'LDFLAGS= -L/lib -Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib' 'LIBS=-libverbs -lpsm_infinipath -lm -lpthread ' 'CPPFLAGS= -I/opt/mvapich2-2.2b/src/mpid/ch3/channels/psm/include -I/opt/mvapich2-2.2b/src/mpid/ch3/channels/psm/include -I/opt/mvapich2-2.2b/src/util/wrappers -I/opt/mvapich2-2.2b/src/util/wrappers -I/opt/mvapich2-2.2b/src/mpl/include -I/opt/mvapich2-2.2b/src/mpl/include -I/opt/mvapich2-2.2b/src/openpa/src -I/opt/mvapich2-2.2b/src/openpa/src -D_REENTRANT -I/opt/mvapich2-2.2b/src/mpi/romio/include -I/include -I/include'
    Process Manager:                         pmi
    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist
    Topology libraries available:            hwloc
    Resource management kernels available:   user slurm ll lsf sge pbs cobalt
    Checkpointing libraries available:
    Demux engines available:                 poll select


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20160106/2eb7f5f0/attachment-0001.html>