[Rdma-spark-discuss] Hi! Ihave some error in rdma spark.

[zino lin]

HI :
  I deploy yours RDMA-spark. we have a error which is no rdmaipcserver in
java.library.path.  Do I need to add any path?  I'm sure  my server can run
RDMA.



error log:

 ✘ paslab at fastnet  ~/zino/rdma-spark-0.9.1-bin 
MASTER=spark://fastnet:7077 ./bin/run-example
org.apache.spark.examples.GroupByTest 96 65536 4096 96
Using Spark's default log4j profile:
org/apache/spark/log4j-defaults.properties
16/04/26 16:49:59 INFO SparkContext: Running Spark version 1.5.1
16/04/26 16:49:59 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
16/04/26 16:49:59 INFO SecurityManager: Changing view acls to: paslab
16/04/26 16:49:59 INFO SecurityManager: Changing modify acls to: paslab
16/04/26 16:49:59 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(paslab); users
with modify permissions: Set(paslab)
16/04/26 16:49:59 INFO Slf4jLogger: Slf4jLogger started
16/04/26 16:49:59 INFO Remoting: Starting remoting
16/04/26 16:50:00 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://sparkDriver@192.168.1.2:45536]
16/04/26 16:50:00 INFO Utils: Successfully started service 'sparkDriver' on
port 45536.
16/04/26 16:50:00 INFO SparkEnv: Registering MapOutputTracker
16/04/26 16:50:00 INFO SparkEnv: create RdmaBlockTransferService with 0
16/04/26 16:50:00 INFO SparkEnv: Registering BlockManagerMaster
16/04/26 16:50:00 INFO DiskBlockManager: Created local directory at
/tmp/blockmgr-b31827b1-3026-4a08-ba16-4cd0d0048197
16/04/26 16:50:00 INFO MemoryStore: MemoryStore started with capacity 530.3
MB
16/04/26 16:50:00 INFO HttpFileServer: HTTP File server directory is
/tmp/spark-ed579380-aedb-4798-aff0-9b021592b93e/httpd-22553087-c264-488e-bafe-4ab25b89d138
16/04/26 16:50:00 INFO HttpServer: Starting HTTP Server
16/04/26 16:50:00 INFO Utils: Successfully started service 'HTTP file
server' on port 38954.
16/04/26 16:50:00 INFO SparkEnv: Registering OutputCommitCoordinator
16/04/26 16:50:00 INFO Utils: Successfully started service 'SparkUI' on
port 4040.
16/04/26 16:50:00 INFO SparkUI: Started SparkUI at http://192.168.1.2:4040
16/04/26 16:50:00 INFO SparkContext: Added JAR
file:/home/paslab/zino/rdma-spark-0.9.1-bin/lib/spark-examples-1.5.1-hadoop2.6.0.jar
at http://192.168.1.2:38954/jars/spark-examples-1.5.1-hadoop2.6.0.jar with
timestamp 1461660600396
16/04/26 16:50:00 WARN MetricsSystem: Using default name DAGScheduler for
source because spark.app.id is not set.
16/04/26 16:50:00 INFO AppClient$ClientEndpoint: Connecting to master
spark://fastnet:7077...
16/04/26 16:50:00 INFO SparkDeploySchedulerBackend: Connected to Spark
cluster with app ID app-20160426165000-0002
Exception in thread "main" java.lang.UnsatisfiedLinkError: no rdmaipcserver
in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1889)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at org.apache.spark.ipc.rdma.RdmaServer.<clinit>(SourceFile:92)
        at
org.apache.spark.network.server.RdmaShuffleServer.<init>(SourceFile:125)
        at
org.apache.spark.network.rdma.RdmaBlockTransferService.init(SourceFile:82)
        at
org.apache.spark.storage.BlockManager.initialize(BlockManager.scala:199)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:528)
        at org.apache.spark.examples.GroupByTest$.main(GroupByTest.scala:37)
        at org.apache.spark.examples.GroupByTest.main(GroupByTest.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:672)
        at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16/04/26 16:50:00 INFO DiskBlockManager: Shutdown hook called
16/04/26 16:50:00 INFO ShutdownHookManager: Shutdown hook called
16/04/26 16:50:00 INFO ShutdownHookManager: Deleting directory
/tmp/spark-ed579380-aedb-4798-aff0-9b021592b93e


by   zino

from National Taiwan University  Department of Computer Science &
Imformation System
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/rdma-spark-discuss/attachments/20160426/f04c18ef/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: qperf RDMA.PNG
Type: image/png
Size: 37542 bytes
Desc: not available
URL: <http://mailman.cse.ohio-state.edu/pipermail/rdma-spark-discuss/attachments/20160426/f04c18ef/attachment-0001.png>