[mvapich-discuss] mvapich compilation error with psm

[Hari Subramoni]

Hello Chulwoo,

We did some more investigation internally. It looks like the error stems
from the "--enable-thread-cs=per-object" configure option. We are able to
build it fine if that option is removed.

It looks like we inherited the error from MPICH (of which we are a
derivative). Further investigation revealed that the error also exists with
the latest mpich release.

While we follow up with the MPICH team on this issue, may I request that
you try a build without the --enable-thread-cs=per-object option? If
needed, feel free to post the issue directly on the mpich discuss list as
well.

Regards,
Hari.

On Nov 26, 2016 7:40 PM, "Jung" <chulwoo at quark.phy.bnl.gov> wrote:

Dear Dhabaleswar,

Thank you very much for the quick reply. Could you confirm which
--enable-threads and --enable-thread-cs options are available for single
OPA, if you know? We are still trying to set our MPI strategy, and whether
we can expect, in the near future if not at present, to be able to truly
use multiple commnication on different threads on 1 MPI is the biggest
question.

Best,
Chulwoo



On Fri, 25 Nov 2016, Panda, Dhabaleswar wrote:

Hi Jung,
>
>  > The main unknown factor here is whether PSM layer can handle dual
 Omni-path (or QLogic/TrueLogic) adapters or not. We have asked some Intel
 folks about it and have not received any reply. Please check with your
 Intel representatives about it.

>
> Thanks,
>
> DK
> ________________________________________
> From: mvapich-discuss-bounces at cse.ohio-state.edu on behalf of Jung [
> chulwoo at quark.phy.bnl.gov]
> Sent: Friday, November 25, 2016 1:23 AM
> To: mvapich-discuss at cse.ohio-state.edu
> Subject: [mvapich-discuss] mvapich compilation error with psm
>
> Hi,
>
> I am getting an error when I'm trying to compile on a KNL machine with
> dual omni-path. --enable-thread-cs=lock-free also fails to compile.
>
> Is there something wrong with the configure options, or the
> enable-thread-cs is limited to global on OPA?
>
> Just to remind you, our ultimate goal is to saturate bandwidth of dual
> rail OPA.
>
> Best,
> Chulwoo Jung
>
>   $ ../mvapich2-2.2/configure --with-device=ch3:psm
> --with-psm2-include=/usr/include --with-psm2-lib=/usr/lib64 --enable-mcast
> --enable-threads=multiple --enable-thread-cs=per-object
> --prefix=/share/test/chulwoo/mvapich2_test
>
>
>   CC       src/mpid/ch3/src/lib_libmpi_la-ch3u_rndv.lo
> In file included from ../mvapich2-2.2/src/include/mpiimpl.h(3878),
>                  from ../mvapich2-2.2/src/mpid/ch3/include/mpidimpl.h(36),
>                  from ../mvapich2-2.2/src/mpid/ch3/src/ch3u_rndv.c(18):
> ../mvapich2-2.2/src/include/mpiimplthreadpost.h(27): warning #159:
> declaration is incompatible with previous
> "MPIU_Thread_CS_enter_lockname_impl_" (declared at line 1343 of
> "../mvapich2-2.2/src/include/mpiimpl.h")
>   MPIU_Thread_CS_enter_lockname_impl_(enum MPIU_Nest_mutexes kind,
>   ^
>
> In file included from ../mvapich2-2.2/src/include/mpiimpl.h(3878),
>                  from ../mvapich2-2.2/src/mpid/ch3/include/mpidimpl.h(36),
>                  from ../mvapich2-2.2/src/mpid/ch3/src/ch3u_rndv.c(18):
> ../mvapich2-2.2/src/include/mpiimplthreadpost.h(46): warning #159:
> declaration is incompatible with previous
> "MPIU_Thread_CS_exit_lockname_impl_" (declared at line 1343 of
> "../mvapich2-2.2/src/include/mpiimpl.h")
>   MPIU_Thread_CS_exit_lockname_impl_(enum MPIU_Nest_mutexes kind,
>   ^
>
> ../mvapich2-2.2/src/mpid/ch3/src/ch3u_rndv.c(210): error: expression must
> have arithmetic or pointer type
>       if (!found && rreq->cc == 0) {
>                     ^
>
> compilation aborted for ../mvapich2-2.2/src/mpid/ch3/src/ch3u_rndv.c (code
> 2)
> make[2]: *** [src/mpid/ch3/src/lib_libmpi_la-ch3u_rndv.lo] Error 1
> make[2]: Leaving directory `/root/chulwoo/mvapich2/testbuild'
> make[1]: *** [all-recursive] Error 1
>
>
> Chulwoo Jung
> Physics Department
> Brookhaven National Laboratory
> U.S.A.
> chulwoo at bnl.gov
> 1-631-344-5254
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
Chulwoo Jung
Physics Department
Brookhaven National Laboratory
U.S.A.
chulwoo at bnl.gov
1-631-344-5254
_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20161126/9af00a16/attachment.html>