[mvapich-discuss] Output large files using MPI-IO

[Madhusudan Pai]

Hello,

This is probably a novice MPI question but I can't seem to figure out the
reason behind an error I get when I try to output a large array on Ranger. I
have created a snippet of my code that can reproduce this error, although I
have pasted only portions here. I can post the entire code (about 117 lines
if needed).

Essentially, I use MPI_TYPE_INDEXED to create a view, then I use
MPI_FILE_SET_VIEW and MPI_FILE_WRITE_ALL to output my file.

 call MPI_TYPE_INDEXED(ncells,blocklength,map,MPI_INTEGER,fileview,ierr)
 call MPI_TYPE_COMMIT(fileview,ierr)

blocklength and map are 1-d arrays of size ncells, and ncells, blocklength
and map are declared as integer (kind=4).

Then I set the view and output an array called hexa as

 disp = 0
 call
MPI_FILE_SET_VIEW(iunit,disp,MPI_INTEGER,fileview,"native",mpi_info,ierr)
 call MPI_FILE_WRITE_ALL(iunit,hexa,ncells,MPI_INTEGER,status,ierr)

where hexa is a 1-d array of size ncells. The array hexa contains the global
node numbering of my mesh (so the last cell contains a value of order
(ncells*nproc)).

For small problem sizes the code works just fine. But the problem arises
when ncells is close to the integer*4 limit. And since the array map is a
function of hexa (specifically, map = hexa * 8), the entries of map also
cross the integer*4 limit. The routine stalls at MPI_FILE_WRITE_ALL with the
error "*io Invalid argument**io Invalid argument**io Invalid argument**io...
" on several processes.

1) I can't seem to figure out which "argument" is causing this error.

2) I also changed the type declaration of map and ncells to integer
(kind=8), but this did not seem to correct the problem. I have also tried
with MPI_INTEGER8 in the WRITE_ALL routine.

I am using mvapich 1.0.1 and intel 10.1 fortran for compilation.

Any help greatly appreciated!

Thanks,
Madhu Pai
Stanford University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20100303/d8a54e7b/attachment.html