[mvapich-discuss] MVAPICH2-0.9.8 internal errors
David Gunter
dog at lanl.gov
Thu May 10 14:18:35 EDT 2007
I had a chance to download mvapich2-0.9.8p2 this morning and retest
and yes, the problem still exists. I am running the IMB 3.0 suite
and I see the following type of behavior:
#----------------------------------------------------------------
# Benchmarking Allreduce
# #processes = 2
# ( 2 additional processes waiting in MPI_Barrier)
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
0 1000 0.03 0.03 0.03
4 1000 1.79 1.79 1.79
8 1000 1.81 1.82 1.81
16 1000 1.84 1.84 1.84
32 1000 1.93 1.93 1.93
64 1000 1.97 1.97 1.97
128 1000 2.12 2.13 2.13
256 1000 2.39 2.39 2.39
512 1000 2.87 2.88 2.87
1024 1000 3.90 3.90 3.90
2048 1000 5.97 5.97 5.97
4096 1000 10.38 10.38 10.38
8192 1000 19.20 19.20 19.20
16384 1000 36.35 36.35 36.35
32768 1000 69.93 69.93 69.93
65536 640 142.94 142.94 142.94
131072 320 281.42 281.42 281.42
262144 160 607.62 607.66 607.64
524288 80 1662.42 1662.94 1662.68
1048576 40 3636.55 3636.75 3636.65
2097152 20 8966.41 8974.30 8970.36
4194304 10 17831.90 17837.21 17834.56
Internal error: communicator is already on free list
Internal error: communicator is already on free list
Wherever a test uses all the processes in a communicator (4 in this
run) then there are no internal error messages. The number of such
messages printed is always equal to the number of processes that did
not participate in a particular test, i.e. the number that always
wait in MPI_Barrier.
Again, if I reconfigure with --disable-debuginfo then I also don't
see these error messages.
-david
--
David Gunter
HPC-4: Parallel Tools Team
Los Alamos National Laboratory
On May 9, 2007, at 2:02 PM, Dhabaleswar Panda wrote:
> Hi David,
>
> Do you see this problem with MVAPICH2 0.9.8p2 version? We had several
> fixes to the released version during the last months. This latest
> version (0.9.8p2) is available from the mvapich page. This version is
> also included and is being tested with OFED 1.2. We have also done
> testing of this latest version with OFED 1.1.
>
> Please let us know if you still see the error with this latest version
> (p2) on your system and we will investigate this issue further.
>
> Thanks,
>
> DK
>
>
>> I am curious to know if you were able to reproduce this problem,
>> whether it has been fixed or not.
>>
>> Thanks,
>> david
>> --
>> David Gunter
>> HPC-4: HPC Environments: Parallel Tools Team
>> Los Alamos National Laboratory
>>
>>
>> On Mar 22, 2007, at 11:20 AM, David Gunter wrote:
>>
>>> I am using the released 0.9.8 version.
>>>
>>> -david
>>>
>>> On Mar 22, 2007, at 9:37 AM, Dhabaleswar Panda wrote:
>>>
>>>> Hi David,
>>>>
>>>> Thanks for this information. One more question - are you using
>>>> MVAPICH2 0.9.8 `released' version or the `branch' version (with
>>>> some
>>>> recent fixes). If you can let us know this information, it will
>>>> help
>>>> us.
>>>>
>>>> Thanks,
>>>>
>>>> DK
>>>>
>>>>
>>>>> I have recompiled mvapich2 without using the --enable-debuginfo
>>>>> flag
>>>>> and the problem has gone away. However, I wish to have debuginfo
>>>>> available to our TotalView users so hopefully this can be
>>>>> resolved.
>>>>>
>>>>> Here is the configuration that generates the error message I saw
>>>>> previously:
>>>>>
>>>>> ./configure --prefix=/opt/mvapich2/mvapich2-0.9.8-gcc/ib --with-
>>>>> openib=/usr/local/ofed --enable-romio --with-file-system=ufs+nfs
>>>>> +panfs --enable-sharedlibs=gcc --enable-debuginfo --enable-fast --
>>>>> with-mpe
>>>>>
>>>>> OFED is ofed-1.1
>>>>>
>>>>> CFLAGS=-D_X86_64_ -D_SMP_ -DUSE_HEADER_CACHING -DONE_SIDED -
>>>>> DMPID_USE_SEQUENCE_NUMBERS -I/usr/local/ofed/include -O2 -
>>>>> D_SHMEM_COLL_
>>>>>
>>>>> CC=/usr/bin/gcc
>>>>> CXX=/usr/bin/g++
>>>>> FC=/usr/bin/gfortran
>>>>> F77=/usr/bin/gfortran
>>>>> F90=/usr/bin/gfortran
>>>>>
>>>>> The test was run on 32 process, 16 process, 8 process and 4
>>>>> process -
>>>>> all of which generated this error message.
>>>>>
>>>>> Thanks,
>>>>> david
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> David Gunter
>>>>> HPC-4: HPC Environments: Parallel Tools Team
>>>>> Los Alamos National Laboratory
>>>>>
>>>>>
>>>>> On Mar 21, 2007, at 4:50 PM, wei huang wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Thanks for letting us know this problem.
>>>>>>
>>>>>> We will try to reproduce the problem on our cluster. To help
>>>>>> us on
>>>>>> looking
>>>>>> into this problem, would you please let us know the following:
>>>>>>
>>>>>> 1) The exact CFLAGS you are using when configuring mvapich2 (are
>>>>>> you using
>>>>>> default scripts provided by us?)
>>>>>> 2) Any runtime environment variables you have setup up?
>>>>>> 3) On how many nodes do you run the test?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> Regards,
>>>>>> Wei Huang
>>>>>>
>>>>>> 774 Dreese Lab, 2015 Neil Ave,
>>>>>> Dept. of Computer Science and Engineering
>>>>>> Ohio State University
>>>>>> OH 43210
>>>>>> Tel: (614)292-8501
>>>>>>
>>>>>>
>>>>>> On Wed, 21 Mar 2007, David Gunter wrote:
>>>>>>
>>>>>>> I have built mvapich2 for an OFED-based IB Opteron cluster.
>>>>>>> When
>>>>>>> running the Intel MPI Benchmarks (IMB3) I keep seeing the
>>>>>>> following
>>>>>>> errors messages in many spots, although the tests run to
>>>>>>> completion:
>>>>>>>
>>>>>>> Internal error: communicator is already on free list
>>>>>>>
>>>>>>> What is this referring to?
>>>>>>>
>>>>>>> Thanks.
>>>>>>> --david
>>>>>>>
>>>>>>> --
>>>>>>> David Gunter
>>>>>>> HPC-4: HPC Environments: Parallel Tools Team
>>>>>>> Los Alamos National Laboratory
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --Apple-Mail-1-849288998
>>>>> Content-Transfer-Encoding: quoted-printable
>>>>> Content-Type: text/html;
>>>>> charset=ISO-8859-1
>>>>>
>>>>> <HTML><BODY style=3D"word-wrap: break-word; -khtml-nbsp-mode:
>>>>> space; =
>>>>> -khtml-line-break: after-white-space; ">I have recompiled
>>>>> mvapich2 =
>>>>> without using the --enable-debuginfo flag and the problem has
>>>>> gone =
>>>>> away.=A0 However, I wish to have debuginfo available to our
>>>>> TotalView =
>>>>> users so hopefully this can be resolved.<DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV>Here is the
>>>>> configuration =
>>>>> that generates the error message I saw previously:</DIV><DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV>./configure =
>>>>> --prefix=3D/opt/mvapich2/mvapich2-0.9.8-gcc/ib=A0--with-openib=3D/
>>>>> usr/loca=
>>>>> l/ofed --enable-romio --with-file-system=3Dufs+nfs+panfs =
>>>>> --enable-sharedlibs=3Dgcc --enable-debuginfo --enable-fast =
>>>>> --with-mpe</DIV><DIV><BR class=3D"khtml-block-placeholder"></
>>>>> DIV><DIV>OFED=
>>>>> is ofed-1.1</DIV><DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV>CFLAGS=3D-D_X86_64_ -
>>>>> D_SMP_ =
>>>>> -DUSE_HEADER_CACHING=A0 -DONE_SIDED -
>>>>> DMPID_USE_SEQUENCE_NUMBERS=A0 =
>>>>> -I/usr/local/ofed/include -O2=A0=A0 -D_SHMEM_COLL_</DIV><DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV>CC=3D/usr/bin/gcc</
>>>>> DIV><DIV>C=
>>>>> XX=3D/usr/bin/g++</DIV><DIV>FC=3D/usr/bin/gfortran</
>>>>> DIV><DIV>F77=3D/usr/bi=
>>>>> n/gfortran</DIV><DIV>F90=3D/usr/bin/gfortran</DIV><DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV>The test was run on
>>>>> 32 =
>>>>> process, 16 process, 8 process and 4 process - all of which
>>>>> generated =
>>>>> this error message.</DIV><DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV>Thanks,</
>>>>> DIV><DIV>david</DIV>=
>>>>> <DIV><BR class=3D"khtml-block-placeholder"></DIV><DIV><BR =
>>>>> class=3D"khtml-block-placeholder"></DIV><DIV><BR><DIV> <SPAN =
>>>>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>>>>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>>>>> Helvetica; =
>>>>> font-size: 12px; font-style: normal; font-variant: normal; font-
>>>>> weight: =
>>>>> normal; letter-spacing: normal; line-height: normal; text-align:
>>>>> auto; =
>>>>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>>>>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>>>>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>>>>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>>>>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>>>>> Helvetica; =
>>>>> font-size: 12px; font-style: normal; font-variant: normal; font-
>>>>> weight: =
>>>>> normal; letter-spacing: normal; line-height: normal; text-align:
>>>>> auto; =
>>>>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>>>>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>>>>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>>>>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>>>>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>>>>> Helvetica; =
>>>>> font-size: 12px; font-style: normal; font-variant: normal; font-
>>>>> weight: =
>>>>> normal; letter-spacing: normal; line-height: normal; text-align:
>>>>> auto; =
>>>>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>>>>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>>>>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>>>>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>>>>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>>>>> Helvetica; =
>>>>> font-size: 12px; font-style: normal; font-variant: normal; font-
>>>>> weight: =
>>>>> normal; letter-spacing: normal; line-height: normal; text-align:
>>>>> auto; =
>>>>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>>>>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>>>>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>>>>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>>>>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>>>>> Helvetica; =
>>>>> font-size: 12px; font-style: normal; font-variant: normal; font-
>>>>> weight: =
>>>>> normal; letter-spacing: normal; line-height: normal; text-align:
>>>>> auto; =
>>>>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>>>>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>>>>> white-space: normal; widows: 2; word-spacing: 0px; =
>>>>> "><DIV>--</DIV><DIV>David Gunter</DIV><DIV>HPC-4: HPC
>>>>> Environments: =
>>>>> Parallel Tools Team</DIV><DIV>Los Alamos National =
>>>>> Laboratory</DIV></SPAN></SPAN><BR =
>>>>> class=3D"Apple-interchange-newline"></SPAN></SPAN></SPAN> =
>>>>> </DIV><BR><DIV><DIV>On Mar 21, 2007, at 4:50 PM, wei huang =
>>>>> wrote:</DIV><BR class=3D"Apple-interchange-newline"><BLOCKQUOTE =
>>>>> type=3D"cite"><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; ">Hi,</DIV><DIV
>>>>> style=3D"margin-top:=
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> min-height: 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Thanks
>>>>> for =
>>>>> letting us know this problem.</DIV><DIV style=3D"margin-top:
>>>>> 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>>>>> height: =
>>>>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right:
>>>>> 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; ">We will try to reproduce
>>>>> the =
>>>>> problem on our cluster. To help us on looking</DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">into this problem, would you please let us
>>>>> know the =
>>>>> following:</DIV><DIV style=3D"margin-top: 0px; margin-right:
>>>>> 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">1) The exact CFLAGS you are using when
>>>>> configuring =
>>>>> mvapich2 (are you using</DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>>>>> ">default =
>>>>> scripts provided by us?)</DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">2)
>>>>> Any =
>>>>> runtime environment variables you have setup up?</DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">3) On how many nodes do you run the test?</
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>>>>> style=3D"margin-top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> ">Thanks.</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">Regards,</DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Wei =
>>>>> Huang</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">774 Dreese Lab, 2015 Neil Ave,</DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">Dept. of Computer Science and Engineering</
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">Ohio State University</DIV><DIV
>>>>> style=3D"margin-top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>>>>> ">OH =
>>>>> 43210</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; ">Tel: (614)292-8501</
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>>>>> style=3D"margin-top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> min-height: 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On
>>>>> Wed, 21 =
>>>>> Mar 2007, David Gunter wrote:</DIV><DIV style=3D"margin-top:
>>>>> 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>>>>> height: =
>>>>> 14px; "><BR></DIV> <BLOCKQUOTE type=3D"cite"><DIV style=3D"margin-
>>>>> top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I
>>>>> have =
>>>>> built mvapich2 for an OFED-based IB Opteron cluster.<SPAN =
>>>>> class=3D"Apple-converted-space">=A0 </SPAN>When</DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">running the Intel MPI Benchmarks (IMB3) I
>>>>> keep =
>>>>> seeing the following</DIV><DIV style=3D"margin-top: 0px; margin-
>>>>> right: =
>>>>> 0px; margin-bottom: 0px; margin-left: 0px; ">errors messages in
>>>>> many =
>>>>> spots, although the tests run to completion:</DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>>>>> style=3D"margin-top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>>>>> ">Internal =
>>>>> error: communicator is already on free list</DIV><DIV
>>>>> style=3D"margin-top:=
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> min-height: 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">What
>>>>> is this =
>>>>> referring to?</DIV><DIV style=3D"margin-top: 0px; margin-right:
>>>>> 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">Thanks.</DIV><DIV style=3D"margin-top: 0px; =
>>>>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> ">--david</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">--</DIV><DIV style=3D"margin-top: 0px; margin-
>>>>> right: =
>>>>> 0px; margin-bottom: 0px; margin-left: 0px; ">David Gunter</
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; ">HPC-4: HPC Environments: Parallel Tools =
>>>>> Team</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; ">Los Alamos National =
>>>>> Laboratory</DIV><DIV style=3D"margin-top: 0px; margin-right:
>>>>> 0px; =
>>>>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>>>>> DIV><DIV =
>>>>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>>>>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>>>>> style=3D"margin-top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> min-height: 14px; "><BR></DIV> </BLOCKQUOTE><DIV style=3D"margin-
>>>>> top: =
>>>>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>>>>> min-height: 14px; "><BR></DIV> =
>>>>> </BLOCKQUOTE></DIV><BR></DIV></BODY></HTML>=
>>>>>
>>>>> --Apple-Mail-1-849288998--
>>>>>
>>>>> --===============1750880957==
>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>> MIME-Version: 1.0
>>>>> Content-Transfer-Encoding: 7bit
>>>>> Content-Disposition: inline
>>>>>
>>>>> _______________________________________________
>>>>> mvapich-discuss mailing list
>>>>> mvapich-discuss at cse.ohio-state.edu
>>>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>>>
>>>>> --===============1750880957==--
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> mvapich-discuss mailing list
>>> mvapich-discuss at cse.ohio-state.edu
>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>> --Apple-Mail-2-709635751
>> Content-Transfer-Encoding: quoted-printable
>> Content-Type: text/html;
>> charset=ISO-8859-1
>>
>> <HTML><BODY style=3D"word-wrap: break-word; -khtml-nbsp-mode:
>> space; =
>> -khtml-line-break: after-white-space; ">I am curious to know if
>> you were =
>> able to reproduce this problem, whether it has been fixed or =
>> not.<DIV><BR =
>> class=3D"khtml-block-placeholder"></DIV><DIV>Thanks,</
>> DIV><DIV>david</DIV>=
>> <DIV><DIV><SPAN class=3D"Apple-style-span" style=3D"border-
>> collapse: =
>> separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-
>> family: =
>> Helvetica; font-size: 12px; font-style: normal; font-variant:
>> normal; =
>> font-weight: normal; letter-spacing: normal; line-height: normal; =
>> text-align: auto; -khtml-text-decorations-in-effect: none; text-
>> indent: =
>> 0px; -apple-text-size-adjust: auto; text-transform: none; orphans:
>> 2; =
>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>> Helvetica; =
>> font-size: 12px; font-style: normal; font-variant: normal; font-
>> weight: =
>> normal; letter-spacing: normal; line-height: normal; text-align:
>> auto; =
>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>> Helvetica; =
>> font-size: 12px; font-style: normal; font-variant: normal; font-
>> weight: =
>> normal; letter-spacing: normal; line-height: normal; text-align:
>> auto; =
>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>> Helvetica; =
>> font-size: 12px; font-style: normal; font-variant: normal; font-
>> weight: =
>> normal; letter-spacing: normal; line-height: normal; text-align:
>> auto; =
>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>> white-space: normal; widows: 2; word-spacing: 0px; "><SPAN =
>> class=3D"Apple-style-span" style=3D"border-collapse: separate; =
>> border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family:
>> Helvetica; =
>> font-size: 12px; font-style: normal; font-variant: normal; font-
>> weight: =
>> normal; letter-spacing: normal; line-height: normal; text-align:
>> auto; =
>> -khtml-text-decorations-in-effect: none; text-indent: 0px; =
>> -apple-text-size-adjust: auto; text-transform: none; orphans: 2; =
>> white-space: normal; widows: 2; word-spacing: 0px; =
>> "><DIV>--</DIV><DIV>David Gunter</DIV><DIV>HPC-4: HPC Environments: =
>> Parallel Tools Team</DIV><DIV>Los Alamos National =
>> Laboratory</DIV></SPAN></SPAN><BR =
>> class=3D"Apple-interchange-newline"></SPAN></SPAN></SPAN> =
>> </DIV><BR><DIV><DIV>On Mar 22, 2007, at 11:20 AM, David Gunter =
>> wrote:</DIV><BR class=3D"Apple-interchange-newline"><BLOCKQUOTE =
>> type=3D"cite"><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">I am using the released
>> 0.9.8 =
>> version.</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">-david</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">On Mar 22, 2007, at 9:37
>> AM, =
>> Dhabaleswar Panda wrote:</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV> <BLOCKQUOTE type=3D"cite"><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Hi =
>> David,</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Thanks for this information. One more question
>> - are =
>> you using</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">MVAPICH2 0.9.8 `released' =
>> version or the `branch' version (with some</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>> ">recent =
>> fixes). If you can let us know this information, it will help</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">us.</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Thanks,</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">DK</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV> <BLOCKQUOTE =
>> type=3D"cite"><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">I have recompiled mvapich2 =
>> without using the --enable-debuginfo flag</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>> ">and the =
>> problem has gone away.<SPAN class=3D"Apple-converted-space">=A0 =
>> </SPAN>However, I wish to have debuginfo</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">available to our TotalView users so hopefully this can be =
>> resolved.</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Here is the configuration that generates the
>> error =
>> message I saw</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">previously:</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">./configure --prefix=3D/opt/mvapich2/mvapich2-0.9.8-gcc/ib =
>> --with-</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">openib=3D/usr/local/ofed =
>> --enable-romio --with-file-system=3Dufs+nfs</DIV><DIV
>> style=3D"margin-top:=
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">
>> +panfs =
>> --enable-sharedlibs=3Dgcc --enable-debuginfo --enable-fast --</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">with-mpe</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">OFED is ofed-1.1</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">CFLAGS=3D-D_X86_64_ -D_SMP_ -DUSE_HEADER_CACHING<SPAN =
>> class=3D"Apple-converted-space">=A0 </SPAN>-DONE_SIDED -</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">DMPID_USE_SEQUENCE_NUMBERS<SPAN =
>> class=3D"Apple-converted-space">=A0 </SPAN>-I/usr/local/ofed/
>> include -O2 =
>> <SPAN class=3D"Apple-converted-space">=A0 </SPAN>-</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">D_SHMEM_COLL_</DIV><DIV style=3D"margin-top:
>> 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">CC=3D/usr/bin/gcc</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">CXX=3D/usr/bin/g++</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">FC=3D/usr/bin/gfortran</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">F77=3D/usr/bin/gfortran</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">F90=3D/usr/bin/gfortran</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">The test was run on 32
>> process, =
>> 16 process, 8 process and 4 process -</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>> ">all of =
>> which generated this error message.</DIV><DIV style=3D"margin-top:
>> 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Thanks,</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">david</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">--</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">David Gunter</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">HPC-4: HPC Environments: Parallel Tools =
>> Team</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Los Alamos National =
>> Laboratory</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On
>> Mar =
>> 21, 2007, at 4:50 PM, wei huang wrote:</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> min-height: 14px; "><BR></DIV> <BLOCKQUOTE type=3D"cite"><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Hi,</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Thanks for letting us know
>> this =
>> problem.</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">We will try to reproduce the problem on our
>> cluster. =
>> To help us on</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">looking</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">into this problem, would you please let us
>> know the =
>> following:</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">1) The exact CFLAGS you are using when
>> configuring =
>> mvapich2 (are</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">you using</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">default scripts provided by us?)</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">2) Any runtime environment variables you have
>> setup =
>> up?</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">3) On how many nodes do
>> you run =
>> the test?</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Thanks.</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Regards,</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Wei Huang</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">774 Dreese Lab, 2015 Neil =
>> Ave,</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Dept. of Computer Science
>> and =
>> Engineering</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Ohio State University</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">OH 43210</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Tel: =
>> (614)292-8501</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">On
>> Wed, =
>> 21 Mar 2007, David Gunter wrote:</DIV><DIV style=3D"margin-top:
>> 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV> <BLOCKQUOTE type=3D"cite"><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I
>> have =
>> built mvapich2 for an OFED-based IB Opteron cluster.<SPAN =
>> class=3D"Apple-converted-space">=A0 </SPAN>When</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">running the Intel MPI Benchmarks (IMB3) I keep =
>> seeing the following</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">errors messages in
>> many =
>> spots, although the tests run to completion:</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>> ">Internal =
>> error: communicator is already on free list</DIV><DIV
>> style=3D"margin-top:=
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> min-height: 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">What is
>> this =
>> referring to?</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Thanks.</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">--david</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">--</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">David Gunter</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">HPC-4: HPC Environments: Parallel Tools =
>> Team</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Los Alamos National =
>> Laboratory</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> min-height: 14px; "><BR></DIV> </BLOCKQUOTE><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> min-height: 14px; "><BR></DIV> </BLOCKQUOTE><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> min-height: 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =
>> ">--Apple-Mail-1-849288998</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">Content-Transfer-Encoding: quoted-printable</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Content-Type: text/html;</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; "><SPAN class=3D"Apple-tab-span" =
>> style=3D"white-space:pre"> </SPAN>charset=3DISO-8859-1</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> "><HTML><BODY style=3D3D"word-wrap: break-word; =
>> -khtml-nbsp-mode: space; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-khtml-line-break: after-white-space; ">I have recompiled
>> mvapich2 =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">without using the =
>> --enable-debuginfo flag and the problem has gone =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">away.=3DA0 However, I wish to have debuginfo =
>> available to our TotalView =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">users so =
>> hopefully this can be resolved.<DIV><BR =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></
>> DIV><DIV>Here is =
>> the configuration =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">that generates the
>> error =
>> message I saw previously:</DIV><DIV><BR =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></DIV><DIV>./
>> configur=
>> e =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =
>> ">--prefix=3D3D/opt/mvapich2/mvapich2-0.9.8-gcc/ib=3DA0--with-
>> openib=3D3D/=
>> usr/loca=3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">l/ofed --enable-romio =
>> --with-file-system=3D3Dufs+nfs+panfs =3D</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">--enable-sharedlibs=3D3Dgcc --enable-debuginfo --enable-fast =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =
>> ">--with-mpe</DIV><DIV><BR =
>> class=3D3D"khtml-block-placeholder"></
>> DIV><DIV>OFED=3D</DIV=
>>> <DIV style=3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>>> 0px; =
>> margin-left: 0px; "><SPAN class=3D"Apple-converted-space">=A0</
>> SPAN>is =
>> ofed-1.1</DIV><DIV><BR =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></
>> DIV><DIV>CFLAGS=3D3=
>> D-D_X86_64_ -D_SMP_ =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-DUSE_HEADER_CACHING=3DA0 -DONE_SIDED -
>> DMPID_USE_SEQUENCE_NUMBERS=3DA0 =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">-I/usr/local/ofed/include =
>> -O2=3DA0=3DA0 -D_SHMEM_COLL_</DIV><DIV><BR =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></
>> DIV><DIV>CC=3D3D/us=
>> r/bin/gcc</DIV><DIV>C=3D</DIV><DIV style=3D"margin-
>> top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">XX=3D3D/usr/bin/g++</DIV><DIV>FC=3D3D/usr/bin/
>> gfortran</D=
>> IV><DIV>F77=3D3D/usr/bi=3D</DIV><DIV style=3D"margin-top:
>> 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">n/gfortran</DIV><DIV>F90=3D3D/usr/bin/gfortran</
>> DIV>&l=
>> t;DIV><BR =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></
>> DIV><DIV>The test =
>> was run on 32 =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">process, 16 process, 8
>> process =
>> and 4 process - all of which generated =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>> ">this =
>> error message.</DIV><DIV><BR =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></
>> DIV><DIV>Thanks,<=
>> ;/DIV><DIV>david</DIV>=3D</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> "><DIV><BR =
>> class=3D3D"khtml-block-placeholder"></
>> DIV><DIV><BR =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"khtml-block-placeholder"></
>> DIV><DIV><BR>=
>> <DIV> <SPAN =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"Apple-style-span" style=3D3D"border-collapse:
>> separate; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">border-spacing: 0px 0px;
>> color: =
>> rgb(0, 0, 0); font-family: Helvetica; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">font-size: 12px; font-style: normal; font-variant: normal; =
>> font-weight: =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">normal; letter-spacing:
>> normal; =
>> line-height: normal; text-align: auto; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-khtml-text-decorations-in-effect: none; text-indent: 0px; =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">-apple-text-size-adjust: auto; text-transform:
>> none; =
>> orphans: 2; =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">white-space: normal;
>> widows: 2; =
>> word-spacing: 0px; "><SPAN =3D</DIV><DIV style=3D"margin-
>> top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"Apple-style-span" style=3D3D"border-collapse:
>> separate; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">border-spacing: 0px 0px;
>> color: =
>> rgb(0, 0, 0); font-family: Helvetica; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">font-size: 12px; font-style: normal; font-variant: normal; =
>> font-weight: =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">normal; letter-spacing:
>> normal; =
>> line-height: normal; text-align: auto; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-khtml-text-decorations-in-effect: none; text-indent: 0px; =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">-apple-text-size-adjust: auto; text-transform:
>> none; =
>> orphans: 2; =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">white-space: normal;
>> widows: 2; =
>> word-spacing: 0px; "><SPAN =3D</DIV><DIV style=3D"margin-
>> top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"Apple-style-span" style=3D3D"border-collapse:
>> separate; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">border-spacing: 0px 0px;
>> color: =
>> rgb(0, 0, 0); font-family: Helvetica; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">font-size: 12px; font-style: normal; font-variant: normal; =
>> font-weight: =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">normal; letter-spacing:
>> normal; =
>> line-height: normal; text-align: auto; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-khtml-text-decorations-in-effect: none; text-indent: 0px; =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">-apple-text-size-adjust: auto; text-transform:
>> none; =
>> orphans: 2; =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">white-space: normal;
>> widows: 2; =
>> word-spacing: 0px; "><SPAN =3D</DIV><DIV style=3D"margin-
>> top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"Apple-style-span" style=3D3D"border-collapse:
>> separate; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">border-spacing: 0px 0px;
>> color: =
>> rgb(0, 0, 0); font-family: Helvetica; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">font-size: 12px; font-style: normal; font-variant: normal; =
>> font-weight: =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">normal; letter-spacing:
>> normal; =
>> line-height: normal; text-align: auto; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-khtml-text-decorations-in-effect: none; text-indent: 0px; =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">-apple-text-size-adjust: auto; text-transform:
>> none; =
>> orphans: 2; =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">white-space: normal;
>> widows: 2; =
>> word-spacing: 0px; "><SPAN =3D</DIV><DIV style=3D"margin-
>> top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">class=3D3D"Apple-style-span" style=3D3D"border-collapse:
>> separate; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">border-spacing: 0px 0px;
>> color: =
>> rgb(0, 0, 0); font-family: Helvetica; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">font-size: 12px; font-style: normal; font-variant: normal; =
>> font-weight: =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">normal; letter-spacing:
>> normal; =
>> line-height: normal; text-align: auto; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">-khtml-text-decorations-in-effect: none; text-indent: 0px; =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">-apple-text-size-adjust: auto; text-transform:
>> none; =
>> orphans: 2; =3D</DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">white-space: normal;
>> widows: 2; =
>> word-spacing: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">"><DIV>--</DIV><DIV>David =
>> Gunter</DIV><DIV>HPC-4: HPC Environments: =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Parallel Tools Team</DIV><DIV>Los
>> Alamos =
>> National =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =
>> ">Laboratory</DIV></SPAN></SPAN><BR =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; =
>> ">class=3D3D"Apple-interchange-newline"></SPAN></
>> SPAN></=
>> SPAN> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =
>> "></DIV><BR><DIV><DIV>On Mar 21, 2007, at
>> 4:50 =
>> PM, wei huang =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">wrote:</DIV><BR =
>> class=3D3D"Apple-interchange-newline"><BLOCKQUOTE =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">type=3D3D"cite"><DIV style=3D3D"margin-
>> top: =
>> 0px; margin-right: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">margin-bottom: 0px; margin-left: 0px; ">Hi,</DIV><DIV =
>> style=3D3D"margin-top:=3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><SPAN =
>> class=3D"Apple-converted-space">=A0</SPAN>0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">min-height: 14px; "><BR></DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">Thanks for =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">letting us know this problem.</
>> DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; min-height: =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">14px; "><BR></DIV><DIV =
>> style=3D3D"margin-top: 0px; margin-right: 0px; =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">margin-bottom: 0px; margin-left: 0px; ">We
>> will =
>> try to reproduce the =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">problem
>> on =
>> our cluster. To help us on looking</DIV><DIV =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; ">into this problem, would you please let us know the =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">following:</
>> DIV><DIV =
>> style=3D3D"margin-top: 0px; margin-right: 0px; =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; ">1) The exact CFLAGS you are using when configuring =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">mvapich2 (are you using</DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">default =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">scripts provided by us?)</DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">2) Any =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">runtime environment variables you have setup =
>> up?</DIV><DIV =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">style=3D3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-left: 0px; ">3)
>> On how =
>> many nodes do you run the test?</DIV><DIV =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; min-height: 14px; "><BR></DIV><DIV =
>> style=3D3D"margin-top: =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">">Thanks.</DIV><DIV
>> style=3D3D"margin-top:=
>> 0px; margin-right: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">margin-bottom: 0px; margin-left: 0px; min-height: 14px; =
>> "><BR></DIV><DIV =3D</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">style=3D3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-left: 0px; =
>> ">Regards,</DIV><DIV style=3D3D"margin-top: 0px; =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">margin-right: 0px; margin-bottom: 0px; margin-
>> left: =
>> 0px; ">Wei =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Huang</DIV><DIV =
>> style=3D3D"margin-top: 0px; margin-right: 0px; =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; ">774 Dreese Lab, 2015 Neil Ave,</DIV><DIV =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; ">Dept. of Computer Science and Engineering</
>> DIV><DIV =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">style=3D3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">margin-left: 0px; ">Ohio State University</DIV><DIV =
>> style=3D3D"margin-top: =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">OH =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">43210</DIV><DIV =
>> style=3D3D"margin-top: 0px; margin-right: 0px; =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">margin-bottom: 0px; margin-left: 0px; ">Tel: =
>> (614)292-8501</DIV><DIV =3D</DIV><DIV style=3D"margin-
>> top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">style=3D3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D3D"margin-top: =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">min-height: 14px; "><BR></DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">On Wed, 21 =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Mar 2007, David Gunter wrote:</
>> DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; min-height: =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">14px; "><BR></DIV>
>> <BLOCKQUOTE =
>> type=3D3D"cite"><DIV style=3D3D"margin-top: =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">I have =3D</DIV><DIV style=3D"margin-top:
>> 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">built =
>> mvapich2 for an OFED-based IB Opteron cluster.<SPAN =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">class=3D3D"Apple-converted-space">=3DA0 =
>> </SPAN>When</DIV><DIV =3D</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">style=3D3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-left: 0px;
>> ">running =
>> the Intel MPI Benchmarks (IMB3) I keep =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;
>> ">seeing =
>> the following</DIV><DIV style=3D3D"margin-top: 0px; =
>> margin-right: =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">0px; margin-bottom: 0px; =
>> margin-left: 0px; ">errors messages in many =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">spots, although the tests run to =
>> completion:</DIV><DIV =3D</DIV><DIV style=3D"margin-top:
>> 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">style=3D3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D3D"margin-top: =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">Internal =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">error: communicator is already on free =
>> list</DIV><DIV style=3D3D"margin-top:=3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; "><SPAN class=3D"Apple-converted-space">=A0</
>> SPAN>0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D3D"margin-top: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">What is this =3D</
>> DIV><DIV=
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">referring to?</DIV><DIV =
>> style=3D3D"margin-top: 0px; margin-right: 0px; =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">margin-bottom: 0px; margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; ">Thanks.</DIV><DIV style=3D3D"margin-top: 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">">--david</DIV><DIV style=3D3D"margin-top: 0px; =
>> margin-right: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">style=3D3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">margin-left: 0px; ">--</DIV><DIV style=3D3D"margin-
>> top: =
>> 0px; margin-right: =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">0px; margin-bottom:
>> 0px; =
>> margin-left: 0px; ">David Gunter</DIV><DIV =3D</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">style=3D3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">margin-
>> left: =
>> 0px; ">HPC-4: HPC Environments: Parallel Tools =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Team</DIV><DIV style=3D3D"margin-top:
>> 0px; =
>> margin-right: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">margin-bottom: 0px; =
>> margin-left: 0px; ">Los Alamos National =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Laboratory</DIV><DIV
>> style=3D3D"margin-top: =
>> 0px; margin-right: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">margin-bottom: 0px; margin-left: 0px; min-height: 14px; =
>> "><BR></DIV><DIV =3D</DIV><DIV style=3D"margin-
>> top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">style=3D3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">margin-left: 0px; min-
>> height: =
>> 14px; "><BR></DIV><DIV style=3D3D"margin-top: =
>> =3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; ">0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; =3D</DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">min-height: 14px; "><BR></DIV> =
>> </BLOCKQUOTE><DIV style=3D3D"margin-top: =3D</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; =3D</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">min-height: 14px; =
>> "><BR></DIV> =3D</DIV><DIV style=3D"margin-top: 0px; =
>> margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> "></BLOCKQUOTE></DIV><BR></DIV></
>> BODY></H=
>> TML>=3D</DIV><DIV style=3D"margin-top: 0px; margin-right: 0px; =
>> margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><BR></
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">--Apple-Mail-1-849288998--</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">--=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D1750880957=3D=3D</
>> DIV><DI=
>> V style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Content-Type: text/plain; =
>> charset=3D"us-ascii"</DIV><DIV style=3D"margin-top: 0px; margin-
>> right: =
>> 0px; margin-bottom: 0px; margin-left: 0px; ">MIME-Version: 1.0</
>> DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Content-Transfer-Encoding: 7bit</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">Content-Disposition: inline</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">_______________________________________________</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">mvapich-discuss mailing list</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; "><A =
>> href=3D"mailto:mvapich-discuss at cse.ohio-state.edu">mvapich-
>> discuss at cse.ohi=
>> o-state.edu</A></DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; "><A =
>> href=3D"http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-
>> discuss">h=
>> ttp://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss</
>> A></DIV><D=
>> IV style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">--=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D1750880957=3D=3D--
>> </DIV><=
>> DIV style=3D"margin-top: 0px; margin-right: 0px; margin-bottom:
>> 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV> </BLOCKQUOTE><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV> </BLOCKQUOTE><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; min-height: 14px; "><BR></DIV><DIV
>> style=3D"margin-top: =
>> 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; =
>> ">_______________________________________________</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; ">mvapich-discuss mailing list</DIV><DIV =
>> style=3D"margin-top: 0px; margin-right: 0px; margin-bottom: 0px; =
>> margin-left: 0px; "><A =
>> href=3D"mailto:mvapich-discuss at cse.ohio-state.edu">mvapich-
>> discuss at cse.ohi=
>> o-state.edu</A></DIV><DIV style=3D"margin-top: 0px; margin-right:
>> 0px; =
>> margin-bottom: 0px; margin-left: 0px; "><A =
>> href=3D"http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-
>> discuss">h=
>> ttp://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss</
>> A></DIV> =
>> </BLOCKQUOTE></DIV><BR></DIV></BODY></HTML>=
>>
>> --Apple-Mail-2-709635751--
>>
>> --===============1220798408==
>> Content-Type: text/plain; charset="us-ascii"
>> MIME-Version: 1.0
>> Content-Transfer-Encoding: 7bit
>> Content-Disposition: inline
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>> --===============1220798408==--
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20070510/ab289007/attachment-0001.html
More information about the mvapich-discuss
mailing list