[mvapich-discuss] Announcing the release of Hadoop-RDMA 0.9 package
Dhabaleswar Panda
panda at cse.ohio-state.edu
Fri Jun 14 01:54:14 EDT 2013
This announcement is being sent `only once' to this list for information
related to this new software package.
Interested and perspective users of Hadoop-RDMA package should subscribe
to hadoop-rdma-discuss mailing list to carry out further discussions
related to this new software package. More details are available at:
http://hadoop-rdma.cse.ohio-state.edu
Thanks,
DK
==================
The Hadoop-RDMA team is pleased to announce the release of Hadoop-RDMA
0.9 package with the following features.
* Hadoop-RDMA 0.9 Features
- Based on Apache Hadoop 0.20.2
- High performance design with native InfiniBand
support at the verbs-level for HDFS, MapReduce
and RPC components
- Compliant with Apache Hadoop 0.20.2 APIs and applications
- Easily configurable for both native InfiniBand and the traditional
sockets-based support (Ethernet and InfiniBand with IPoIB)
- HDFS over native InfiniBand
- RDMA-based write
- RDMA-based replication
- Tuned packet size
- MapReduce over native InfiniBand
- RDMA-based shuffle
- Prefetching and caching of map output
- In-memory merge
- Overlapping of merge and reduce
- RPC over native InfiniBand
- JVM-bypassed buffer management
- On-demand InfiniBand connection setup
- RDMA or send/recv based adaptive communication
- Tested with
- Mellanox InfiniBand adapters (DDR, QDR and FDR)
- Various multi-core platforms
- Different file systems with disks and SSDs
For downloading Hadoop-RDMA 0.9 package and the associated user guide,
please visit the following URL:
http://hadoop-rdma.cse.ohio-state.edu
Sample performance numbers for benchmarks using Hadoop-RDMA 0.9
version can be viewed by visiting the `Performance' tab of the above
website.
All questions, feedbacks and bug reports are welcome. Please post it
to the hadoop-rdma-discuss mailing list
(hadoop-rdma-discuss at cse.ohio-state.edu).
Thanks,
The Hadoop-RDMA Team
More information about the mvapich-discuss
mailing list