[mvapich-discuss] gather receive order

[Krishna Kandalla]

Hello Andrew,
          Thank you for reporting this issue.  We have a fix for this in our
current development version. Can you please let us know which version of
mvapich2 you are using? I can generate a patch for your version and send it
over. Meanwhile, you can also try setting the following run-time flags to 0:

MV2_USE_DIRECT_GATHER<http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.7_alpha.html#x1-20100011.75>and
MV2_USE_TWO_LEVEL_GATHER<http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.7_alpha.html#x1-22300011.97>.


Regards,
Krishna


On Tue, May 24, 2011 at 6:02 AM, Andrew Senin <andrew.senin at itseez.com>wrote:

> Hello list,
>
>
>
> I have an application which uses MPI_Gather. It worked perfectly with
> openmpi-1.4.3 and mpich2-1.3.2p1. However when I switched to mvapich2 it
> starts failing. After investigation I found that MPI_Gather does receives
> array in rank order while the other 2 implementations do. Is this an issue
> in mvapich2? The standard says that “The root process receives the messages
> and stores them in rank order”.
>
>
>
> Here is the output I receive from 4 ranks:
>
> rank 2 sent (4,5,)
>
> rank 3 sent (6,7,)
>
> rank 0 sent (0,1,)
>
> rank 0 received 0 1 6 7 2 3 4 5 , expected 0 1 2 3 4 5 6 7
>
> rank 1 sent (2,3,)
>
>
>
> And the test which generates this output:
>
> #include "mpi.h"
>
> #include <string.h>
>
> #include "stdio.h"
>
>
>
> #define num_el 2
>
> #define num_ranks 4
>
>
>
> int main(int argc, char *argv[])
>
> {
>
>     int i = 0;
>
>     int rank;
>
>     int size;
>
>     int gather_send[num_el] = {0};
>
>     int gather_recv[num_el * num_ranks] = {0};
>
>
>
>     MPI_Init(&argc, &argv);
>
>
>
>     MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>
>     MPI_Comm_size(MPI_COMM_WORLD, &size);
>
>
>
>     if (size != num_ranks)
>
>     {
>
>         printf("This sample works with only %i ranks\n", num_ranks);
>
>         goto FreeMemory;
>
>     }
>
>
>
>     for (i = 0; i < num_el; i++)
>
>     {
>
>         gather_send[i] = num_el * rank + i;
>
>     }
>
>
>
>     MPI_Gather(gather_send, num_el, MPI_INT,
>
>                gather_recv, num_el, MPI_INT,
>
>                0, MPI_COMM_WORLD);
>
>
>
>     printf("rank %i sent (",rank);
>
>     for (i = 0; i < num_el; i++)
>
>     {
>
>         printf("%i,", gather_send[i]);
>
>     }
>
>     printf(")\n");
>
>
>
>     if (rank == 0)
>
>     {
>
>         int i = 0;
>
>
>
>         char buffer_received[10000] = {0};
>
>         char buffer_expected[10000] = {0};
>
>
>
>         for(i = 0; i < num_el * num_ranks; i++)
>
>         {
>
>             char temp_received[128] = {0};
>
>             char temp_expected[128] = {0};
>
>
>
>             sprintf(temp_received, "%i ", gather_recv[i]);
>
>             sprintf(temp_expected, "%i ", i);
>
>
>
>             strcat(buffer_expected, temp_expected);
>
>             strcat(buffer_received, temp_received);
>
>         }
>
>
>
>         printf("rank %i received %s, expected %s\n",
>
>                 rank,
>
>                 buffer_received,
>
>                 buffer_expected);
>
>     }
>
>
>
>
>
> FreeMemory:
>
>     MPI_Finalize();
>
>
>
>     return 0;
>
> }
>
>
>
> Thanks,
>
> Andrew.
>
>
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20110524/f685d55d/attachment-0001.html