[mvapich-discuss] OMB collective memory limit

[Hari Subramoni]

Dear Dr. Kempf,

Thanks for reporting the issue to us. We will make appropriate changes. It
should be available in an upcoming release of OMB.

Best Regards,
Hari.

On Sep 5, 2016 11:31 PM, "Andreas Kempf" <Andreas.Kempf at emea.nec.com> wrote:

> Dear MVAPICH team,
>
> while running large OSU benchmarks (5.3.1 and 5.3.0) using the
> "collective" test cases, I noticed that the -M parameter for setting the
> "per process maximum memory consumption" does not accept values larger than
> 2 GiB:
>
> $ mpirun -np 2 ./osu_alltoall -M 33221225472
> Requested memory limit too low, using [1048576] instead.Requested memory
> limit too low, using [1048576] instead.
>
> Apparently, while most of the code treats large values properly, the
> routine set_max_memlimit in osu_coll.c accepts its parameter as a regular
> integer:
>
> osu_coll.c line 348
> set_max_memlimit(atoll(optarg));
>
> osu_coll.c line 227-239
> static int
> set_max_memlimit (int value)
> {
>     options.max_mem_limit = value;
>
>     if (value < MAX_MEM_LOWER_LIMIT) {
>         options.max_mem_limit = MAX_MEM_LOWER_LIMIT;
>         fprintf(stderr,"Requested memory limit too low, using [%d]
> instead.",
>                 MAX_MEM_LOWER_LIMIT);
>     }
>
>     return 0;
> }
>
> So the optarg is properly treated as 64 bit value but passed to
> set_max_memlimit as a regular integer. Consequently, after the cast the
> check in set_max_memlimit fails and the default is used as a result. Using
> "set_max_memlimit (uint64_t value)" seems to allow for larger memory
> allocations.
>
> Please fix the collective test cases to allow for larger memory
> allocations. On several thousand processors, 2 GiB can become a real
> limitation.
>
> Best regards,
>
> Andreas Kempf
>
> ----------------
> Dr. Andreas Kempf
> Benchmarking Analyst at NEC Deutschland GmbH
> Tel: +49 211 5369 310
> ----
> NEC Deutschland GmbH, Prinzenallee 11, 40549 Düsseldorf
> Geschäftsführer Yuichi Kojima
> Handelsregister Düsseldorf HRB 57941; VAT ID DE129424743
>
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20160905/e3b7c093/attachment-0001.html>