[mvapich-discuss] MVAPICH 2.1 check commications device
Hari Subramoni
subramoni.1 at osu.edu
Wed Dec 9 13:30:53 EST 2015
Hi Ryan,
Yes - it is okay. The output indicates that it is using a Mellanox FDR HCA.
Please let us know if you face any other issues. We'll be happy to help.
Thx,
Hari.
On Wed, Dec 9, 2015 at 1:22 PM, Novosielski, Ryan <novosirj at ca.rutgers.edu>
wrote:
> I have the following output from MV2_SHOW_ENV_INFO=1. Is it OK?
>
> MVAPICH2-2.1 Parameters
> ---------------------------------------------------------------------
> PROCESSOR ARCH NAME :
> MV2_ARCH_INTEL_XEON_E5_2680_V3_2S_24
> PROCESSOR FAMILY NAME : MV2_CPU_FAMILY_INTEL
> PROCESSOR MODEL NUMBER : 63
> HCA NAME : MV2_HCA_MLX_CX_FDR
> HETEROGENEOUS HCA : NO
> MV2_VBUF_TOTAL_SIZE : 17408
> MV2_IBA_EAGER_THRESHOLD : 17408
> MV2_RDMA_FAST_PATH_BUF_SIZE : 5120
> MV2_PUT_FALLBACK_THRESHOLD : 8192
> MV2_GET_FALLBACK_THRESHOLD : 0
> MV2_EAGERSIZE_1SC : 8192
> MV2_SMP_EAGERSIZE : 32769
> MV2_SMPI_LENGTH_QUEUE : 131072
> MV2_SMP_NUM_SEND_BUFFER : 32
> MV2_SMP_BATCH_SIZE : 8
> ---------------------------------------------------------------------
> ---------------------------------------------------------------------
>
> --
> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
> || \\UTGERS |---------------------*O*---------------------
> ||_// Biomedical | Ryan Novosielski - Senior Technologist
> || \\ and Health | novosirj at rutgers.edu - 973/972.0922 (2x0922)
> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> `'
> ________________________________________
> From: hari.subramoni at gmail.com [hari.subramoni at gmail.com] On Behalf Of
> Hari Subramoni [subramoni.1 at osu.edu]
> Sent: Wednesday, December 09, 2015 12:20 PM
> To: Novosielski, Ryan
> Subject: Re:
>
> Hi Ryan,
>
> No problems. I don't think thats your problem. The OSU mail server is
> Outlook based. Sometimes, it doesn't like e-mails sent from gmail and other
> mail clients and it decides to mangle things up pretty badly. We've raised
> the issue with OSU IT support. But a large organization has its lag :-(.
> Please bear with us while we resolve this issue.
>
> With the config line you gave, MV2 should never use Ethernet ports. It
> should always use IB HCAs.
>
> Regards,
> Hari.
>
> On Wed, Dec 9, 2015 at 12:02 PM, Novosielski, Ryan <
> novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>> wrote:
> BTW, sorry about the subject line -- something chewed it up. Maybe barfs
> on a question mark somewhere along the line?
>
> OK, thanks, I thought that might be the one, but it doesn't say for sure
> in the documentation.
>
> # mpiname -a
> MVAPICH2 2.1 Fri Apr 03 20:00:00 EDT 2015 ch3:mrail
>
> Compilation
> CC: icc -DNDEBUG -DNVALGRIND -O2
> CXX: icpc -DNDEBUG -DNVALGRIND -O2
> F77: ifort -O2
> FC: ifort -O2
>
> Configuration
> --prefix=/opt/sw/packages/intel-16.0.1/mvapich2/2.1 --with-pmi=pmi2
> --with-pm=slurm
>
> --
> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
> || \\UTGERS |---------------------*O*---------------------
> ||_// Biomedical | Ryan Novosielski - Senior Technologist
> || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu> -
> 973/972.0922<tel:973%2F972.0922> (2x0922)
> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> `'
> ________________________________________
> From: hari.subramoni at gmail.com<mailto:hari.subramoni at gmail.com> [
> hari.subramoni at gmail.com<mailto:hari.subramoni at gmail.com>] On Behalf Of
> Hari Subramoni [subramoni.1 at osu.edu<mailto:subramoni.1 at osu.edu>]
> Sent: Wednesday, December 09, 2015 12:02 PM
> To: Novosielski, Ryan
> Cc: mvapich-discuss at cse.ohio-state.edu<mailto:
> mvapich-discuss at cse.ohio-state.edu>
> Subject: Re:
>
> Hi Ryan,
>
> Please run after setting "MV2_SHOW_ENV_INFO=1". MVAPICH2 will print out
> the HCA being used.
>
> Could you also share the output of <install_path>/bin/mpiname -a? If
> MVAPICH2 were configured in the default way, it should always use the IB
> HCA instead of the Ethernet one. Please look at the following section of
> the userguide for more info about how to configure and build MVAPICH2.
>
>
> http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.2b-userguide.html#x1-120004.4
>
> Regards,
> Hari.
>
> On Wed, Dec 9, 2015 at 11:57 AM, Novosielski, Ryan <
> novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu><mailto:
> novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>>> wrote:
> X-MS-Exchange-CrossTenan
> Is there an environment variable or something I can set that will confirm
> which network device is being used for communication? It appears as if my
> hosts may be using Ethernet instead of Infiniband, which is very strange.
> I'm using MPAPICH2 2.1 via PMI2 (using SLURM).
>
> --
> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
> || \\UTGERS |---------------------*O*---------------------
> ||_// Biomedical | Ryan Novosielski - Senior Technologist
> || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu
> ><mailto:novosirj at rutgers.edu<mailto:novosirj at rutgers.edu>> - 973/972.0922<tel:973%2F972.0922><tel:973%2F972.0922>
> (2x0922)
> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> `'
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu<mailto:
> mvapich-discuss at cse.ohio-state.edu><mailto:
> mvapich-discuss at cse.ohio-state.edu<mailto:
> mvapich-discuss at cse.ohio-state.edu>>
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu<mailto:
> mvapich-discuss at cse.ohio-state.edu>
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20151209/8aedfb1a/attachment-0001.html>
More information about the mvapich-discuss
mailing list