[mvapich-discuss] No route to host in MVAPICH-Virt

[Xiaoyi Lu]

Good to know this helped. Please feel free to let us know if you have any more questions.

Thanks,
Xiaoyi

> On Jul 5, 2017, at 5:00 AM, Maksym Planeta <mplaneta at os.inf.tu-dresden.de> wrote:
> 
> 
> 
> On 07/04/2017 07:31 PM, Xiaoyi Lu wrote:
>> Hi,
>> 
>> In addition to Jie's comments, please try to stop your firewall and
>> rerun the commands.
> 
> This actually helped. I ran systemctl stop firewalld. And now it seems to work. Thank you.
> 
>> 
>> Thanks,
>> Xiaoyi
>> 
>> Sent from my iPhone
>> 
>> On Jul 4, 2017, at 12:05 PM, Jie Zhang <zhanjie at cse.ohio-state.edu
>> <mailto:zhanjie at cse.ohio-state.edu>> wrote:
>> 
>>> Hi,  Maksym,
>>> 
>>> This should be a network setup issue.
>>> Can you please try a few things:
>>> 
>>> 1. Do you have correct IP/hostname mapping for both hosts in the file,
>>> such as /etc/hosts
>>> 2. What is the output if you run
>>> /opt/mvapich2/virt/2.2/bin/mpirun_rsh  -np 2 host11 host12
>>> MV2_VIRT_USE_IVSHMEM=1 hostname
>>> 
>>> 
>>> On Tue, Jul 4, 2017 at 11:01 AM Maksym Planeta
>>> <mplaneta at os.inf.tu-dresden.de <mailto:mplaneta at os.inf.tu-dresden.de>>
>>> wrote:
>>> 
>>>    Hello,
>>> 
>>>    I have setup several virtual machines and I'm trying to run simple
>>>    MPI programs in there.
>>> 
>>>    My current test is following:
>>> 
>>>    /opt/mvapich2/virt/2.2/bin/mpirun_rsh  -export-all -n 2 host11
>>>    host12 MV2_VIRT_USE_IVSHMEM=1 bash -c 'netstat -l | grep
>>>    $(printenv MPISPAWN_CHECKIN_PORT)'
>>>    tcp        0      0 0.0.0.0:49379 <http://0.0.0.0:49379>
>>>     0.0.0.0:*               LISTEN
>>>    connect [mt_checkin]: No route to host
>>>    [host11:mpirun_rsh][child_handler] Error in init phase, aborting!
>>>    (1/2 mpispawn connections)
>>> 
>>>    So I'm trying to look if the checkin port is open. But the remote
>>>    host connection fails with a message "no route to host".
>>> 
>>>    Showing the command which is about to be executed gives me this:
>>> 
>>>    $ /opt/mvapich2/virt/2.2/bin/mpirun_rsh -show  -export-all -n 2
>>>    host11 host12 MV2_VIRT_USE_IVSHMEM=1 bash -c 'netstat -l | grep
>>>    $(printenv MPISPAWN_CHECKIN_PORT)'
>>>    /usr/bin/ssh -q host12 cd /home/user; /usr/bin/env
>>>    MPISPAWN_MPIRUN_MPD=0 USE_LINEAR_SSH=1 MPISPAWN_MPIRUN_HOST=host11
>>>    MPISPAWN_MPIRUN_HOSTIP=141.76.84.15 MPIRUN_RSH_LAUNCH=1
>>>    MPISPAWN_CHECKIN_PORT=52056 MPISPAWN_MPIRUN_PORT=52056
>>>    MPISPAWN_NNODES=2 MPISPAWN_GLOBAL_NPROCS=2 MPISPAWN_MPIRUN_ID=5190
>>>    MPISPAWN_ARGC=3 MPDMAN_KVS_TEMPLATE=kvs_506_host11_5190
>>>    MPISPAWN_LOCAL_NPROCS=1 MPISPAWN_ARGV_0='bash'
>>>    MPISPAWN_ARGV_1='-c' MPISPAWN_ARGV_2='netstat -l | grep $(printenv
>>>    MPISPAWN_CHECKIN_PORT)' MPISPAWN_ARGC=3
>>>    MPISPAWN_GENERIC_ENV_COUNT=1
>>>    MPISPAWN_GENERIC_NAME_0=MV2_VIRT_USE_IVSHMEM
>>>    MPISPAWN_GENERIC_VALUE_0=1 MPISPAWN_ID=1
>>>    MPISPAWN_WORKING_DIR=/home/user MPISPAWN_MPIRUN_RANK_0=1
>>>    /opt/mvapich2/virt/2.2/bin/mpispawn 0
>>> 
>>>    /usr/bin/bash -c cd /home/user; /usr/bin/env
>>>    MPISPAWN_MPIRUN_MPD=0 USE_LINEAR_SSH=1 MPISPAWN_MPIRUN_HOST=host11
>>>    MPISPAWN_MPIRUN_HOSTIP=141.76.84.15 MPIRUN_RSH_LAUNCH=1
>>>    MPISPAWN_CHECKIN_PORT=52056 MPISPAWN_MPIRUN_PORT=52056
>>>    MPISPAWN_NNODES=2 MPISPAWN_GLOBAL_NPROCS=2 MPISPAWN_MPIRUN_ID=5190
>>>    MPISPAWN_ARGC=3 MPDMAN_KVS_TEMPLATE=kvs_506_host11_5190
>>>    MPISPAWN_LOCAL_NPROCS=1 MPISPAWN_ARGV_0='bash'
>>>    MPISPAWN_ARGV_1='-c' MPISPAWN_ARGV_2='netstat -l | grep $(printenv
>>>    MPISPAWN_CHECKIN_PORT)' MPISPAWN_ARGC=3
>>>    MPISPAWN_GENERIC_ENV_COUNT=1
>>>    MPISPAWN_GENERIC_NAME_0=MV2_VIRT_USE_IVSHMEM
>>>    MPISPAWN_GENERIC_VALUE_0=1 MPISPAWN_ID=0
>>>    MPISPAWN_WORKING_DIR=/home/user MPISPAWN_MPIRUN_RANK_0=0
>>>    /opt/mvapich2/virt/2.2/bin/mpispawn 0
>>> 
>>> 
>>>    I can reach both hosts from each other and they see the same IP
>>>    addresses, and I can ssh to each of the two hosts from each of the
>>>    two hosts.
>>> 
>>> 
>>>    Could you help me to identify the issue?
>>> 
>>>    The MVAPICH-Virt version is the last accessible on the mvapich
>>>    download page
>>> 
>>>    P. S.
>>> 
>>>    You say to allow reading for the user in section 5 of the Virt
>>>    user manual, but actually the library asks for write access as
>>>    well, so I set up rights to 766.
>>> 
>>>    --
>>>    Regards,
>>>    Maksym Planeta
>>> 
>>>    _______________________________________________
>>>    mvapich-discuss mailing list
>>>    mvapich-discuss at cse.ohio-state.edu
>>>    <mailto:mvapich-discuss at cse.ohio-state.edu>
>>>    http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>> 
>>> --
>>> Thanks,
>>> -Jie
>>> 
>>> --------------------------------------------
>>> Department of Computer Science & Engineering
>>> Erwin E. Dreese Laboratories, 2015 Neil Avenue
>>> Ohio State University
>>> _______________________________________________
>>> mvapich-discuss mailing list
>>> mvapich-discuss at cse.ohio-state.edu
>>> <mailto:mvapich-discuss at cse.ohio-state.edu>
>>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
> 
> -- 
> Regards,
> Maksym Planeta
>