[mvapich-discuss] mvapich2 2.3a on ppc64le

[Hari Subramoni]

Thanks for getting back quickly. It is surprising that you are facing the
registration cache issue with osu microbenchmarks.

We will take a look at it and get back to you shortly.

Thx,
Hari.

On Thu, Jun 29, 2017 at 8:03 AM, Jamil Appa <jamil.appa at zenotech.com> wrote:

> Hi Hari
>
>  Thanks for the quick reply. I get this error running any of the osu
> benchmarks that are distributed with mvapich2 as well as my own application.
>
>   The applications work if I have 4 threads per core so a total 80 threads
> per node.  It looks like there is a limit to the maximum number of threads
> fixed by the size of the mask.
>
>  Let me know if you want me to run with different switches to generate
> more output.
>
>  Jamil
>
>
> On Thu, 29 Jun 2017 at 12:49 Hari Subramoni <subramoni.1 at osu.edu> wrote:
>
>> Hello,
>>
>> Sorry to hear that you are facing issues. These are two separate issues
>> actually. Could you please let us know what program you are running? That
>> will help us narrow the issue down further.
>>
>> Thx,
>> Hari.
>>
>>
>> On Jun 29, 2017 6:49 AM, "Jamil Appa" <jamil.appa at zenotech.com> wrote:
>>
>> Hi
>>
>>     I am trying to use mvapich2 2.3a on a 2 node ppc64le system with 8
>> threads per core (total threads per node of 160)
>>
>>     There appears to be a bug in ptmalloc that prevents correct startup
>> related to setting the affinity.
>>
>>    WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing
>> without InfiniBand registration cache support.
>> Warning! : Core id -1 does not exist on this architecture!
>> CPU Affinity is undefined
>> Error parsing CPU mapping string
>> INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in
>> MPIDI_CH3I_set_affinity:2673
>> Warning! : Core id -1 does not exist on this architecture!
>> CPU Affinity is undefined
>> Error parsing CPU mapping string
>> INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in
>> MPIDI_CH3I_set_affinity:2673
>> [cli_2]: aborting job:
>> Fatal error in PMPI_Init_thread:
>> Other MPI error, error stack:
>> MPIR_Init_thread(490):
>> MPID_Init(386).......:
>>
>> [cli_0]: aborting job:
>> Fatal error in PMPI_Init_thread:
>> Other MPI error, error stack:
>> MPIR_Init_thread(490):
>> MPID_Init(386).......:
>>
>>  mpiexec -env MV2_NUM_HCAS 1 -env MV2_NUM_PORTS 1 -env
>> MV2_USE_THREAD_WARNING 0 -env MV2_SHOW_HCA_BINDING 0 -env
>> MV2_CPU_BINDING_LEVEL numanode -env MV2_CPU_BINDING_POLICY scatter
>>
>>   cat /etc/redhat-release
>> Red Hat Enterprise Linux Server release 7.2 (Maipo)
>>
>>  uname -a
>>
>>  Linux nux gpu02.cluster 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29
>> 17:31:13 EDT 2015 ppc64le ppc64le ppc64le GNU/Linux
>>
>>
>>
>>
>> *Jamil Appa* | Co-Founder and Director | Zenotech
>> [image: Papercut]
>> [image: Tel:] +44 (0)7747 606 788 <+44%207747%20606788> [image: Zenotech
>> LTD - Simulation Unlimited] <http://www.zenotech.com/>
>> [image: Email:] jamil.appa at zenotech.com
>> [image: Web:] www.zenotech.com
>> [image: Papercut]
>> [image: linkedin:] <http://uk.linkedin.com/pub/jamil-appa/1/165/120>[image:
>> Twitter:] <https://twitter.com/zenotech>[image: Location:]
>> <https://www.google.co.uk/maps/place/Bristol+%26+Bath+Science+Park/@51.500921,-2.478567,17z/data=!3m1!4b1!4m2!3m1!1s0x48719ab86a5a9f7d:0xd17394f3400abb0a>
>>
>> Company Registration No : 07926926 | VAT No : 128198591
>>
>> Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16
>> 5UF, UK
>>
>> Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green,
>> Bristol BS16 7FR
>>
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20170629/d9023295/attachment-0001.html>