[mvapich-discuss] mvapich 0.9.7 vapi driver + SilverStorm drivers

[Dhabaleswar Panda]

Hi Troy, 

Thanks for chiming in and indicating that you are also seeing similar
problems with 0.9.7 and SilverStorm drivers.

It appears that SilverStorm's VAPI driver has different functions or
data structures than Mellanox IBGD and this is leading to this
incompatibility problem.

Since SilverStorm people are running a version of MVAPICH (not sure
which older version of MVAPICH is being used), they will be able to
clearly indicate the required changes needed to make the latest
MVAPICH 0.9.7 run with their proprietary latest driver. This will help
SilverStorm's customers to take advantage of the latest features of
MVAPICH.

As I had indicated earlier, we do not have access to SilverStorm's
stack. Thus, it is very hard for us to figure out what is going on
here and what changes are needed.

I am cc'ing this note to Mr. Todd Rimmer, Chief Systems Architect, of
SilverStorm. 

Mr. Rimmer: Could you please let all of us know about this
incompatibility problem and how Silverstorm's users can solve it.

Thanks, 

DK




> On Sun, 02 Apr 2006 21:11:23 -0600, Dhabaleswar Panda  
> <panda at cse.ohio-state.edu> wrote:
> 
> >> > Are you trying the gen2 device or vapi device? If you are using the
> >> > vapi device, I hope you are using the ADAPTIVE_RDMA_FAST_PATH support
> >> > (not the SRQ support which is available with gen2 device only). We are
> >> > working on the SRQ support for vapi device and will be available soon.
> >> I use vapi device with ADAPTIVE_RDMA_FAST_PATH.
> >
> > Thanks for this information.
> >
> >>  > This discussion list has some members from SilverStrom. May I request
> >> > them to extend help on this and let us know why this error is coming
> >> > with the latest SilverStrom driver.
> >> >
> >> > If some other users are using the same driver (3.2.0.0.25) driver from
> >> > Silverstorm with mvapich 0.9.7, may I request them to indicate whether
> >> > they are seeing this error or not.
> 
> >> > [0] Abort: Could not modify qp to RTR (Invalid Parameter) at line 935  
> >> in
> >> > file viainit.c
> 
> I just wanted to chime in and report that I too am seeing this error with  
> MVAPICH 0.9.7 and the SilverStorm 3.2.0.0.25 drivers.
> 
> I'm heartened with the progress towards the 2nd generation OpenIB stack,  
> but sometimes it's not an option.
> 
> Interestingly enough, SilverStorm seems to distribute their own branch of  
> MVAPICH (albeit an older one).  Binaries compiled with the semi-functional  
> MVAPICH (ie using SilverStorm drivers & libs) will run fine on the  
> SilverStorm branch of MVAPICH.  (Running SS MVAPICH-combiled binaries on  
> MVAPICH 0.9.7 results in the same error as reported above.)
> 
> I've also verified that I'm not running into a case of mixed libraries  
>  from MVAPICH and SilverStorm's MPI.  (Start with a fresh OS install, then  
> install the OS & MPI, test, wipe the system, start over.)
> -- 
> Troy Telford
>