[mvapich-discuss] MVAPICH 2.1 check commications device

[Novosielski, Ryan]

I have the following output from MV2_SHOW_ENV_INFO=1. Is it OK?

 MVAPICH2-2.1 Parameters
---------------------------------------------------------------------
        PROCESSOR ARCH NAME            : MV2_ARCH_INTEL_XEON_E5_2680_V3_2S_24
        PROCESSOR FAMILY NAME          : MV2_CPU_FAMILY_INTEL
        PROCESSOR MODEL NUMBER         : 63
        HCA NAME                       : MV2_HCA_MLX_CX_FDR
        HETEROGENEOUS HCA              : NO
        MV2_VBUF_TOTAL_SIZE            : 17408
        MV2_IBA_EAGER_THRESHOLD        : 17408
        MV2_RDMA_FAST_PATH_BUF_SIZE    : 5120
        MV2_PUT_FALLBACK_THRESHOLD     : 8192
        MV2_GET_FALLBACK_THRESHOLD     : 0
        MV2_EAGERSIZE_1SC              : 8192
        MV2_SMP_EAGERSIZE              : 32769
        MV2_SMPI_LENGTH_QUEUE          : 131072
        MV2_SMP_NUM_SEND_BUFFER        : 32
        MV2_SMP_BATCH_SIZE             : 8
---------------------------------------------------------------------
---------------------------------------------------------------------

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
 || \\UTGERS      |---------------------*O*---------------------
 ||_// Biomedical | Ryan Novosielski - Senior Technologist
 || \\ and Health | novosirj at rutgers.edu - 973/972.0922 (2x0922)
 ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
      `'
________________________________________
From: hari.subramoni at gmail.com [hari.subramoni at gmail.com] On Behalf Of Hari Subramoni [subramoni.1 at osu.edu]
Sent: Wednesday, December 09, 2015 12:20 PM
To: Novosielski, Ryan
Subject: Re:

Hi Ryan,

No problems. I don't think thats your problem. The OSU mail server is Outlook based. Sometimes, it doesn't like e-mails sent from gmail and other mail clients and it decides to mangle things up pretty badly. We've raised the issue with OSU IT support. But a large organization has its lag :-(. Please bear with us while we resolve this issue.

With the config line you gave, MV2 should never use Ethernet ports. It should always use IB HCAs.

Regards,
Hari.

On Wed, Dec 9, 2015 at 12:02 PM, Novosielski, Ryan <novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>> wrote:
BTW, sorry about the subject line -- something chewed it up. Maybe barfs on a question mark somewhere along the line?

OK, thanks, I thought that might be the one, but it doesn't say for sure in the documentation.

# mpiname -a
MVAPICH2 2.1 Fri Apr 03 20:00:00 EDT 2015 ch3:mrail

Compilation
CC: icc    -DNDEBUG -DNVALGRIND -O2
CXX: icpc   -DNDEBUG -DNVALGRIND -O2
F77: ifort   -O2
FC: ifort   -O2

Configuration
--prefix=/opt/sw/packages/intel-16.0.1/mvapich2/2.1 --with-pmi=pmi2 --with-pm=slurm

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
 || \\UTGERS      |---------------------*O*---------------------
 ||_// Biomedical | Ryan Novosielski - Senior Technologist
 || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)
 ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
      `'
________________________________________
From: hari.subramoni at gmail.com<mailto:hari.subramoni at gmail.com> [hari.subramoni at gmail.com<mailto:hari.subramoni at gmail.com>] On Behalf Of Hari Subramoni [subramoni.1 at osu.edu<mailto:subramoni.1 at osu.edu>]
Sent: Wednesday, December 09, 2015 12:02 PM
To: Novosielski, Ryan
Cc: mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>
Subject: Re:

Hi Ryan,

Please run after setting "MV2_SHOW_ENV_INFO=1". MVAPICH2 will print out the HCA being used.

Could you also share the output of <install_path>/bin/mpiname -a? If MVAPICH2 were configured in the default way, it should always use the IB HCA instead of the Ethernet one. Please look at the following section of the userguide for more info about how to configure and build MVAPICH2.

http://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.2b-userguide.html#x1-120004.4

Regards,
Hari.

On Wed, Dec 9, 2015 at 11:57 AM, Novosielski, Ryan <novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu><mailto:novosirj at ca.rutgers.edu<mailto:novosirj at ca.rutgers.edu>>> wrote:
X-MS-Exchange-CrossTenan
Is there an environment variable or something I can set that will confirm which network device is being used for communication? It appears as if my hosts may be using Ethernet instead of Infiniband, which is very strange. I'm using MPAPICH2 2.1 via PMI2 (using SLURM).

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
 || \\UTGERS      |---------------------*O*---------------------
 ||_// Biomedical | Ryan Novosielski - Senior Technologist
 || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu><mailto:novosirj at rutgers.edu<mailto:novosirj at rutgers.edu>> - 973/972.0922<tel:973%2F972.0922><tel:973%2F972.0922> (2x0922)
 ||  \\  Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
      `'

_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu><mailto:mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>>
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss



_______________________________________________
mvapich-discuss mailing list
mvapich-discuss at cse.ohio-state.edu<mailto:mvapich-discuss at cse.ohio-state.edu>
http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss