[mvapich-discuss] Keep some MPI tasks from connecting to GPU

[Oliver Fuhrer (MeteoSwiss)]

Hi,

I'm running a hybrid application with 4 MPI tasks restricted to pure CPU operation and 8 MPI tasks running compute on GPUs .(amounting to a total of 12 MPI tasks per node) I'm running this code on a fat node architecture with 8 GPUs connected to two Sandybridge sockets on each node.

Is there a way to keep the pure CPU tasks _completely_ from accessing the devices? It seems that upon MPI_Init() all MPI tasks are initializing and setting up a context on the device.

+-----------------------------------------------------------------------------+
| Compute processes:                                               GPU Memory |
|  GPU       PID  Process name                                     Usage      |
|=============================================================================|
|    0      7484  ./lm_f90                                            3917MB  |
|    0      7492  ./lm_f90                                              77MB  |
|    1      7485  ./lm_f90                                            3922MB  |
|    1      7493  ./lm_f90                                              77MB  |
|    2      7486  ./lm_f90                                            3922MB  |
|    2      7494  ./lm_f90                                              77MB  |
|    3      7487  ./lm_f90                                            3917MB  |
|    3      7495  ./lm_f90                                              77MB  |
|    4      7488  ./lm_f90                                            3909MB  |
|    5      7489  ./lm_f90                                            3914MB  |
|    6      7490  ./lm_f90                                            3914MB  |
|    7      7491  ./lm_f90                                            3909MB  |
+-----------------------------------------------------------------------------+

Cheers,
Oli

_________

Oliver Fuhrer
Numerical Models

Federal Departement of Home Affairs FDHA
Federal Office of Meteorology and Climatology MeteoSwiss

Kraehbuehlstrasse 58, P.O. Box 514, CH-8044 Zurich, Switzerland

Tel. +41 44 256 93 59
Fax +41 44 256 92 78
oliver.fuhrer at ginko.ch
www.meteoswiss.ch - First-hand information