[mvapich-discuss] ptmalloc warning MVAPICH2-2.2a
Trey Dockendorf
treydock at tamu.edu
Mon Nov 2 13:58:58 EST 2015
The patch was provided to me around Oct 21st, and is attached.
GCC is 5.2.0 and BLCR is 0.8.6
IB stack is from OS provided packages (CentOS 6.5).
Thanks,
- Trey
=============================
Trey Dockendorf
Systems Analyst I
Texas A&M University
Academy for Advanced Telecommunications and Learning Technologies
Phone: (979)458-2396
Email: treydock at tamu.edu
Jabber: treydock at tamu.edu
On Mon, Nov 2, 2015 at 12:53 PM, Jonathan Perkins <
perkinjo at cse.ohio-state.edu> wrote:
> Thanks for the requested info. Can you also share the contents of the
> patch? We will try to reproduce this locally and get back to you.
>
> The version of gcc and blcr used may also be helpful.
>
> On Mon, Nov 2, 2015 at 1:29 PM Trey Dockendorf <treydock at tamu.edu> wrote:
>
>> Below [1] are the my build steps. The last warning I saw was with PETSc
>> 3.4.5 test programs, that are from PETSc source. I have had to
>> set MV2_ENABLE_AFFINITY=0 to avoid affinity errors, unsure if that has any
>> impact.
>>
>> I ran osu_bw and received the same warning [2].
>>
>> The output from mpiname -a:
>>
>> MVAPICH2 2.2a Mon Aug 17 20:00:00 EDT 2015 ch3:mrail
>>
>> Compilation
>> CC: gcc -DNDEBUG -DNVALGRIND -O2
>> CXX: g++ -DNDEBUG -DNVALGRIND -O2
>> F77: gfortran -O2
>> FC: gfortran -O2
>>
>> Configuration
>> --enable-shared --enable-threads=multiple --with-device=ch3:mrail
>> --with-rdma=gen2 --with-pm=slurm --with-pmi=pmi2 --enable-ckpt --with-hwloc
>> --with-ibverbs=/usr --with-slurm=/usr --prefix=/apps/gcc-5_2/mvapich2/2.2a
>>
>> [1]:
>>
>> module load gcc/5.2.0
>>
>> cd mvapich2-2.2a
>>
>> patch -p1 < /home/treydock/support/cr_inline_build_fail.patch
>>
>> ./configure --enable-shared --enable-threads=multiple \
>> --with-device=ch3:mrail --with-rdma=gen2 \
>> --with-pm=slurm --with-pmi=pmi2 \
>> --enable-ckpt --with-hwloc \
>> --with-ibverbs=/usr --with-slurm=/usr \
>> --prefix=/apps/gcc-5_2/mvapich2/2.2a
>>
>> make -j4
>>
>> make install
>>
>> [2]:
>> $ srun -n 2 -N 2 -p admin --qos admin --constraint='core8&ib_ddr'
>> /apps/gcc-5_2/mvapich2/2.2a/libexec/mvapich2/osu_bw
>> srun: job 8700040 queued and waiting for resources
>> srun: job 8700040 has been allocated resources
>> WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing
>> without InfiniBand registration cache support.
>> # OSU MPI Bandwidth Test
>> # Size Bandwidth (MB/s)
>> 1 1.07
>> 2 2.16
>> 4 4.30
>> 8 8.43
>> 16 17.03
>> 32 31.31
>> 64 62.78
>> 128 122.03
>> 256 235.78
>> 512 419.07
>> 1024 740.01
>> 2048 1134.13
>> 4096 1409.16
>> 8192 1488.48
>> 16384 1287.30
>> 32768 1441.78
>> 65536 1521.44
>> 131072 1478.69
>> 262144 1546.63
>> 524288 1562.09
>> 1048576 1624.74
>> 2097152 1578.08
>> 4194304 1571.00
>>
>> =============================
>>
>> Trey Dockendorf
>> Systems Analyst I
>> Texas A&M University
>> Academy for Advanced Telecommunications and Learning Technologies
>> Phone: (979)458-2396
>> Email: treydock at tamu.edu
>> Jabber: treydock at tamu.edu
>>
>> On Mon, Nov 2, 2015 at 12:01 PM, Jonathan Perkins <
>> perkinjo at cse.ohio-state.edu> wrote:
>>
>>> Hello Trey. This is a bit surprising. Can you provide us more
>>> information on how you built mvapich2 and whether or not OMB runs emit this
>>> error or if the are more specific applications that produce the warning
>>> message?
>>>
>>> Please send us the output of `mpiname -a' as well.
>>>
>>> On Mon, Nov 2, 2015 at 12:58 PM Trey Dockendorf <treydock at tamu.edu>
>>> wrote:
>>>
>>>> --===============4849056478641154220==
>>>> Content-Type: multipart/alternative;
>>>> boundary="001a1136e43681345205239283a1"
>>>>
>>>> --001a1136e43681345205239283a1
>>>> Content-Type: text/plain; charset="UTF-8"
>>>>
>>>>
>>>> I'm testing MVAPICH2-2.2a and have noticed that all applications run
>>>> result
>>>> in the following warning:
>>>>
>>>> WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing
>>>> without
>>>> InfiniBand registration cache support.
>>>>
>>>> Is there a way to resolve this, or do I need to rebuild
>>>> with --disable-registration-cache?
>>>>
>>>> Thanks,
>>>> - Trey
>>>>
>>>> =============================
>>>>
>>>> Trey Dockendorf
>>>> Systems Analyst I
>>>> Texas A&M University
>>>> Academy for Advanced Telecommunications and Learning Technologies
>>>> Phone: (979)458-2396
>>>> Email: treydock at tamu.edu
>>>> Jabber: treydock at tamu.edu
>>>>
>>>> --001a1136e43681345205239283a1
>>>> Content-Type: text/html; charset="UTF-8"
>>>> Content-Transfer-Encoding: quoted-printable
>>>>
>>>> <div dir=3D"ltr">I'm testing MVAPICH2-2.2a and have noticed that
>>>> all ap=
>>>> plications run result in the following
>>>> warning:<div><br></div><div>WARNING:=
>>>> Error in initializing MVAPICH2 ptmalloc library.Continuing without
>>>> InfiniB=
>>>> and registration cache support.</div><div><br></div><div>Is there a way
>>>> to =
>>>> resolve this, or do I need to rebuild
>>>> with=C2=A0--disable-registration-cach=
>>>> e?</div><div><br></div><div>Thanks,</div><div>- Trey<br><div><br
>>>> clear=3D"a=
>>>> ll"><div><div class=3D"gmail_signature"><div
>>>> dir=3D"ltr"><div>=3D=3D=3D=3D=
>>>>
>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
>>>> </div><div><br></div><div>Trey Dockendorf=C2=A0</div><div>Systems
>>>> Analyst I=
>>>> =C2=A0</div><div>Texas A&M University=C2=A0</div><div>Academy for
>>>> Advan=
>>>> ced Telecommunications and Learning Technologies=C2=A0</div><div>Phone:
>>>> (97=
>>>> 9)458-2396=C2=A0</div><div>Email: <a href=3D"mailto:treydock at tamu.edu"
>>>> targ=
>>>> et=3D"_blank">treydock at tamu.edu</a>=C2=A0</div><div>Jabber: <a
>>>> href=3D"mail=
>>>> to:treydock at tamu.edu" target=3D"_blank">treydock at tamu.edu
>>>> </a></div></div></=
>>>> div></div>
>>>> </div></div></div>
>>>>
>>>> --001a1136e43681345205239283a1--
>>>>
>>>> --===============4849056478641154220==
>>>> Content-Type: text/plain; charset="us-ascii"
>>>> MIME-Version: 1.0
>>>> Content-Transfer-Encoding: 7bit
>>>> Content-Disposition: inline
>>>>
>>>> _______________________________________________
>>>> mvapich-discuss mailing list
>>>> mvapich-discuss at cse.ohio-state.edu
>>>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>>
>>>> --===============4849056478641154220==--
>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20151102/8689a3b2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cr_inline_build_fail.patch
Type: application/octet-stream
Size: 505 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20151102/8689a3b2/attachment-0001.obj>
More information about the mvapich-discuss
mailing list