[HiBD] Announcing the release of RDMA-Apache-Spark 0.9.1
Panda, Dhabaleswar
panda at cse.ohio-state.edu
Tue Jan 5 17:54:08 EST 2016
The High-Performance Big Data (HiBD) team is pleased to announce the
release of RDMA-Apache-Spark 0.9.1 with the following features.
* RDMA-Apache-Spark 0.9.1 Features
- Based on Apache Spark 1.5.1
- High performance design with native InfiniBand and RoCE
support at the verbs-level for Spark
- RDMA-based data shuffle
- SEDA-based shuffle architecture
- Efficient connection management and sharing
- Non-blocking and chunk-based data transfer
- Off-JVM-heap buffer management
- Compliant with Apache Spark 1.5.1 APIs and applications
- Easily configurable for native InfiniBand, RoCE, and the
traditional sockets based support (Ethernet and InfiniBand
with IPoIB)
- Tested with
- Mellanox InfiniBand adapters (DDR, QDR, and FDR)
- RoCE support with Mellanox adapters
- Various multi-core platforms
- RAM Disks, SSDs, and HDDs
For downloading RDMA-Apache-Spark 0.9.1 package, the associated user
guides, please visit the following URL:
http://hibd.cse.ohio-state.edu
Sample performance numbers for RDMA-Apache-Spark using benchmarks can
be viewed by visiting the `Performance' tab of the above website.
All questions, feedback and bug reports are welcome. Please post to
the rdma-spark-discuss mailing list (rdma-spark-discuss at
cse.ohio-state.edu).
Thanks,
The High-Performance Big Data (HiBD) Team
http://hibd.cse.ohio-state.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/hibd-announce/attachments/20160105/b5792c62/attachment.html>
More information about the hibd-announce
mailing list