[mvapich-discuss] MVAPICH IB /tmp files

[LEI CHAI]

Dr. Potts,

The files you have seen are not a problem :-) 

MVAPICH uses shared memory for intra-node communication. All the processes on the same node mmap some temporary files, and use the mapped memory regions for intra-node point-to-point and collective communication. As long as you compile MVAPICH with the _SMP_ flag, you will see the files during startup. The files will be cleaned up after the program finishes.

The files are initially created as empty, and then processes touch the files to bring them into the memory. So depending on when you take a look at the files, right after the empty files are created or after they are touched by the processes, the files may be "empty" or "filled".

Lei



----- Original Message -----
From: Mark Potts <potts at hpcapplications.com>
Date: Sunday, June 3, 2007 10:04 am
Subject: [mvapich-discuss] MVAPICH  IB /tmp files

> Hi,
>   I'm observing files created during MVAPICH app startup and need
>   to understand their purpose.  Their existence seems related to a
>   configuration problem on an IB x86_64 cluster system.  I'm
>   using MVAPICH-0.9.9 ch_gen2 singlerail and the app is quite simple.
> 
>    The 3 files are named:     /tmp/ib_pool-NNN-nodename.tmp
>                               /tmp/ib_shmem-NNN-nodename.tmp
>                               /tmp/ib_shmem_coll-NNN-nodename.tmp
>   where "NNN" is the MPIRUN_ID of the app and "nodename" is the name
>   of the compute node on which the files are found.  On some nodes
>   the files are empty and on others they have significant binary 
> (non-
>   text) content.
> 
>   Since these files are not created for all run-time selections
>   of compute nodes, I'm trying to understand two things:
>   (1) Under what circumstances are these files generated or not, and
>   (2) what is the intended content of these files that leaves some
>       empty and others "filled"?
> 
>   Thanks.
>        regards,
> -- 
> ***********************************
> >> Mark J. Potts, PhD
> >>
> >> HPC Applications Inc.
> >> phone: 410-992-8360 Bus
> >>        410-313-9318 Home
> >>        443-418-4375 Cell
> >> email: potts at hpcapplications.com
> >>        potts at excray.com
> ***********************************
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>