[mvapich-discuss] mvapich2 2.3a on ppc64le

[Jamil Appa]

Hi Hari

 Thanks for the quick reply. I get this error running any of the osu
benchmarks that are distributed with mvapich2 as well as my own application.

  The applications work if I have 4 threads per core so a total 80 threads
per node.  It looks like there is a limit to the maximum number of threads
fixed by the size of the mask.

 Let me know if you want me to run with different switches to generate more
output.

 Jamil

On Thu, 29 Jun 2017 at 12:49 Hari Subramoni <subramoni.1 at osu.edu> wrote:

> Hello,
>
> Sorry to hear that you are facing issues. These are two separate issues
> actually. Could you please let us know what program you are running? That
> will help us narrow the issue down further.
>
> Thx,
> Hari.
>
>
> On Jun 29, 2017 6:49 AM, "Jamil Appa" <jamil.appa at zenotech.com> wrote:
>
> Hi
>
>     I am trying to use mvapich2 2.3a on a 2 node ppc64le system with 8
> threads per core (total threads per node of 160)
>
>     There appears to be a bug in ptmalloc that prevents correct startup
> related to setting the affinity.
>
>    WARNING: Error in initializing MVAPICH2 ptmalloc library.Continuing
> without InfiniBand registration cache support.
> Warning! : Core id -1 does not exist on this architecture!
> CPU Affinity is undefined
> Error parsing CPU mapping string
> INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in
> MPIDI_CH3I_set_affinity:2673
> Warning! : Core id -1 does not exist on this architecture!
> CPU Affinity is undefined
> Error parsing CPU mapping string
> INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in
> MPIDI_CH3I_set_affinity:2673
> [cli_2]: aborting job:
> Fatal error in PMPI_Init_thread:
> Other MPI error, error stack:
> MPIR_Init_thread(490):
> MPID_Init(386).......:
>
> [cli_0]: aborting job:
> Fatal error in PMPI_Init_thread:
> Other MPI error, error stack:
> MPIR_Init_thread(490):
> MPID_Init(386).......:
>
>  mpiexec -env MV2_NUM_HCAS 1 -env MV2_NUM_PORTS 1 -env
> MV2_USE_THREAD_WARNING 0 -env MV2_SHOW_HCA_BINDING 0 -env
> MV2_CPU_BINDING_LEVEL numanode -env MV2_CPU_BINDING_POLICY scatter
>
>   cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.2 (Maipo)
>
>  uname -a
>
>  Linux nux gpu02.cluster 3.10.0-327.el7.ppc64le #1 SMP Thu Oct 29 17:31:13
> EDT 2015 ppc64le ppc64le ppc64le GNU/Linux
>
>
>
>
> *Jamil Appa* | Co-Founder and Director | Zenotech
> [image: Papercut]
> [image: Tel:] +44 (0)7747 606 788 <+44%207747%20606788> [image: Zenotech
> LTD - Simulation Unlimited] <http://www.zenotech.com/>
> [image: Email:] jamil.appa at zenotech.com
> [image: Web:] www.zenotech.com
> [image: Papercut]
> [image: linkedin:] <http://uk.linkedin.com/pub/jamil-appa/1/165/120>[image:
> Twitter:] <https://twitter.com/zenotech>[image: Location:]
> <https://www.google.co.uk/maps/place/Bristol+%26+Bath+Science+Park/@51.500921,-2.478567,17z/data=!3m1!4b1!4m2!3m1!1s0x48719ab86a5a9f7d:0xd17394f3400abb0a>
>
> Company Registration No : 07926926 | VAT No : 128198591
>
> Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF,
> UK
>
> Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol
> BS16 7FR
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20170629/eac4bb17/attachment-0001.html>