[mvapich-discuss] ptmalloc warning MVAPICH2-2.2a
Trey Dockendorf
treydock at tamu.edu
Mon Nov 2 13:28:53 EST 2015
Below [1] are the my build steps. The last warning I saw was with PETSc
3.4.5 test programs, that are from PETSc source. I have had to
set MV2_ENABLE_AFFINITY=0 to avoid affinity errors, unsure if that has any
impact.
I ran osu_bw and received the same warning [2].
The output from mpiname -a:
MVAPICH2 2.2a Mon Aug 17 20:00:00 EDT 2015 ch3:mrail
Compilation
CC: gcc -DNDEBUG -DNVALGRIND -O2
CXX: g++ -DNDEBUG -DNVALGRIND -O2
F77: gfortran -O2
FC: gfortran -O2
Configuration
--enable-shared --enable-threads=multiple --with-device=ch3:mrail
--with-rdma=gen2 --with-pm=slurm --with-pmi=pmi2 --enable-ckpt --with-hwloc
--with-ibverbs=/usr --with-slurm=/usr --prefix=/apps/gcc-5_2/mvapich2/2.2a
[1]:
module load gcc/5.2.0
cd mvapich2-2.2a
patch -p1 < /home/treydock/support/cr_inline_build_fail.patch
./configure --enable-shared --enable-threads=multiple \
--with-device=ch3:mrail --with-rdma=gen2 \
--with-pm=slurm --with-pmi=pmi2 \
--enable-ckpt --with-hwloc \
--with-ibverbs=/usr --with-slurm=/usr \
--prefix=/apps/gcc-5_2/mvapich2/2.2a
make -j4
make install
[2]:
$ srun -n 2 -N 2 -p admin --qos admin --constraint='core8&ib_ddr'
/apps/gcc-5_2/mvapich2/2.2a/libexec/mvapich2/osu_bw
srun: job 8700040 queued and waiting for resources
srun: job 8700040 has been allocated resources
WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing without
InfiniBand registration cache support.
# OSU MPI Bandwidth Test
# Size Bandwidth (MB/s)
1 1.07
2 2.16
4 4.30
8 8.43
16 17.03
32 31.31
64 62.78
128 122.03
256 235.78
512 419.07
1024 740.01
2048 1134.13
4096 1409.16
8192 1488.48
16384 1287.30
32768 1441.78
65536 1521.44
131072 1478.69
262144 1546.63
524288 1562.09
1048576 1624.74
2097152 1578.08
4194304 1571.00
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning Technologies
Phone: (979)458-2396
Email: treydock at tamu.edu
Jabber: treydock at tamu.edu
On Mon, Nov 2, 2015 at 12:01 PM, Jonathan Perkins <
perkinjo at cse.ohio-state.edu> wrote:
> Hello Trey. This is a bit surprising. Can you provide us more
> information on how you built mvapich2 and whether or not OMB runs emit this
> error or if the are more specific applications that produce the warning
> message?
>
> Please send us the output of `mpiname -a' as well.
>
> On Mon, Nov 2, 2015 at 12:58 PM Trey Dockendorf <treydock at tamu.edu> wrote:
>
>> --===============4849056478641154220==
>> Content-Type: multipart/alternative;
>> boundary="001a1136e43681345205239283a1"
>>
>> --001a1136e43681345205239283a1
>> Content-Type: text/plain; charset="UTF-8"
>>
>>
>> I'm testing MVAPICH2-2.2a and have noticed that all applications run
>> result
>> in the following warning:
>>
>> WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing
>> without
>> InfiniBand registration cache support.
>>
>> Is there a way to resolve this, or do I need to rebuild
>> with --disable-registration-cache?
>>
>> Thanks,
>> - Trey
>>
>> =============================
>>
>> Trey Dockendorf
>> Systems Analyst I
>> Texas A&M University
>> Academy for Advanced Telecommunications and Learning Technologies
>> Phone: (979)458-2396
>> Email: treydock at tamu.edu
>> Jabber: treydock at tamu.edu
>>
>> --001a1136e43681345205239283a1
>> Content-Type: text/html; charset="UTF-8"
>> Content-Transfer-Encoding: quoted-printable
>>
>> <div dir=3D"ltr">I'm testing MVAPICH2-2.2a and have noticed that all
>> ap=
>> plications run result in the following
>> warning:<div><br></div><div>WARNING:=
>> Error in initializing MVAPICH2 ptmalloc library.Continuing without
>> InfiniB=
>> and registration cache support.</div><div><br></div><div>Is there a way
>> to =
>> resolve this, or do I need to rebuild
>> with=C2=A0--disable-registration-cach=
>> e?</div><div><br></div><div>Thanks,</div><div>- Trey<br><div><br
>> clear=3D"a=
>> ll"><div><div class=3D"gmail_signature"><div
>> dir=3D"ltr"><div>=3D=3D=3D=3D=
>>
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>> </div><div><br></div><div>Trey Dockendorf=C2=A0</div><div>Systems Analyst
>> I=
>> =C2=A0</div><div>Texas A&M University=C2=A0</div><div>Academy for
>> Advan=
>> ced Telecommunications and Learning Technologies=C2=A0</div><div>Phone:
>> (97=
>> 9)458-2396=C2=A0</div><div>Email: <a href=3D"mailto:treydock at tamu.edu"
>> targ=
>> et=3D"_blank">treydock at tamu.edu</a>=C2=A0</div><div>Jabber: <a
>> href=3D"mail=
>> to:treydock at tamu.edu" target=3D"_blank">treydock at tamu.edu
>> </a></div></div></=
>> div></div>
>> </div></div></div>
>>
>> --001a1136e43681345205239283a1--
>>
>> --===============4849056478641154220==
>> Content-Type: text/plain; charset="us-ascii"
>> MIME-Version: 1.0
>> Content-Transfer-Encoding: 7bit
>> Content-Disposition: inline
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>> --===============4849056478641154220==--
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20151102/34d814d4/attachment-0001.html>
More information about the mvapich-discuss
mailing list