[HiBD] Announcing the release of RDMA-Apache-Spark 0.9.1

Panda, Dhabaleswar panda at cse.ohio-state.edu
Tue Jan 5 17:54:08 EST 2016


The High-Performance Big Data (HiBD) team is pleased to announce the
release of RDMA-Apache-Spark 0.9.1 with the following features.

* RDMA-Apache-Spark 0.9.1 Features

    - Based on Apache Spark 1.5.1
    - High performance design with native InfiniBand and RoCE
      support at the verbs-level for Spark
        - RDMA-based data shuffle
        - SEDA-based shuffle architecture
        - Efficient connection management and sharing
        - Non-blocking and chunk-based data transfer
        - Off-JVM-heap buffer management
    - Compliant with Apache Spark 1.5.1 APIs and applications
    - Easily configurable for native InfiniBand, RoCE, and the
      traditional sockets based support (Ethernet and InfiniBand
      with IPoIB)
    - Tested with
        - Mellanox InfiniBand adapters (DDR, QDR, and FDR)
        - RoCE support with Mellanox adapters
        - Various multi-core platforms
        - RAM Disks, SSDs, and HDDs

For downloading RDMA-Apache-Spark 0.9.1 package, the associated user
guides, please visit the following URL:

http://hibd.cse.ohio-state.edu

Sample performance numbers for RDMA-Apache-Spark using benchmarks can
be viewed by visiting the `Performance' tab of the above website.

All questions, feedback and bug reports are welcome. Please post to
the rdma-spark-discuss mailing list (rdma-spark-discuss at
cse.ohio-state.edu).

Thanks,

The High-Performance Big Data (HiBD) Team
http://hibd.cse.ohio-state.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/hibd-announce/attachments/20160105/b5792c62/attachment.html>


More information about the hibd-announce mailing list