[mvapich-discuss] Shell Environment Variables and mpirun_rsh

[Jonathan Perkins]

On Fri, Apr 19, 2013 at 09:09:05AM -0400, Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] wrote:
> On 04/19/2013 08:55 AM, Jonathan Perkins wrote:
> >Since you're using mvapich2-1.8.1, I suggest you put your environment
> >settings for mvapich2 in a ~/.mvapich2.conf.
> >http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.9rc1.html#x1-490006.3
> 
> Nice. I hadn't seen that, but that's a useful hint. Is this
> equivalent to using the '-config' option for mpirun_rsh? I.e., I
> could put those variables in a file next to my script and do
> "-config mvapich2envs.conf'?

Not equivalent.  The -config option was unfortunately named.  That
option is used for MPMD jobs and is used for giving different command
options to different executables within one job.  In the following link
scroll down to Job Launch using MPMD for more information on this.

http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.8.1.html#x1-270005.2.1

In order to dynamically set which mvapich2.conf file to use mentioned in
my previous reply you can use the MV2_USER_CONFIG environment variable
which tells mpirun_rsh (and the mpi processes that are later launched)
where to find it.

> >If you move to mvapich2-1.9rc1 you can take advantage of the new -export
> >option of mpirun_rsh where the environment is forwarded for you.
> >
> >setenv MV2_ENABLE_AFFINITY    0
> >setenv MV2_CPU_BINDING_POLICY scatter
> >setenv MV2_CPU_BINDING_LEVEL  socket
> >mpirun_rsh -export -hostfile $PBS_NODEFILE -np $PROCS ./GEOSgcm.x
> >
> 
> Looks like a good reason to upgrade! We have been holding off
> because of a weird string broadcast issue MVAPICH2 1.9a had with our
> code (sometimes it'd broadcast Fortran NULL characters at the end as
> padding, sometimes /377 chars...). I'll have it installed and give
> it a try.

Interesting, please let us know if you still face this issue with
1.9rc1.  We'll be happy to debug it further to see if there is something
that we need to fix on our side before the final release.

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo