[mvapich-discuss] Announcing the release of Hadoop-RDMA 0.9 package

Dhabaleswar Panda panda at cse.ohio-state.edu
Fri Jun 14 01:54:14 EDT 2013


This announcement is being sent `only once' to this list for information 
related to this new software package.

Interested and perspective users of Hadoop-RDMA package should subscribe 
to hadoop-rdma-discuss mailing list to carry out further discussions 
related to this new software package.  More details are available at: 
http://hadoop-rdma.cse.ohio-state.edu

Thanks,

DK

==================


The Hadoop-RDMA team is pleased to announce the release of Hadoop-RDMA
0.9 package with the following features.

* Hadoop-RDMA 0.9 Features

   - Based on Apache Hadoop 0.20.2
   - High performance design with native InfiniBand
     support at the verbs-level for HDFS, MapReduce
     and RPC components
   - Compliant with Apache Hadoop 0.20.2 APIs and applications
   - Easily configurable for both native InfiniBand and the traditional
     sockets-based support (Ethernet and InfiniBand with IPoIB)
   - HDFS over native InfiniBand
      - RDMA-based write
      - RDMA-based replication
      - Tuned packet size
   - MapReduce over native InfiniBand
      - RDMA-based shuffle
      - Prefetching  and caching of map output
      - In-memory merge
      - Overlapping of merge and reduce
   - RPC over native InfiniBand
      - JVM-bypassed buffer management
      - On-demand InfiniBand connection setup
      - RDMA or send/recv based adaptive communication
   - Tested with
      - Mellanox InfiniBand adapters (DDR, QDR and FDR)
      - Various multi-core platforms
      - Different file systems with disks and SSDs

For downloading Hadoop-RDMA 0.9 package and the associated user guide,
please visit the following URL:

http://hadoop-rdma.cse.ohio-state.edu

Sample performance numbers for benchmarks using Hadoop-RDMA 0.9
version can be viewed by visiting the `Performance' tab of the above
website.

All questions, feedbacks and bug reports are welcome. Please post it
to the hadoop-rdma-discuss mailing list
(hadoop-rdma-discuss at cse.ohio-state.edu).

Thanks,

The Hadoop-RDMA Team


More information about the mvapich-discuss mailing list