Experiences applying data staging technology in unconventional ways

J Lofstead, R Oldfield… - 2013 13th IEEE/ACM …, 2013 - ieeexplore.ieee.org
J Lofstead, R Oldfield, T Kordenbrock
2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and …, 2013ieeexplore.ieee.org
Several efforts have shown the potential of using additional compute-area resources to
enhance the IO path to storage. Efforts like data staging, IO forwarding, and similar
techniques can accelerate IO performance and reduce the impact of IO time to a compute
application. Hybrid staging enhanced this path by adding processing functionality to
locations along the data path to storage. While these efforts have been effective, they have
taken a somewhat limited view of the potential benefits using some additional compute …
Several efforts have shown the potential of using additional compute-area resources to enhance the IO path to storage. Efforts like data staging, IO forwarding, and similar techniques can accelerate IO performance and reduce the impact of IO time to a compute application. Hybrid staging enhanced this path by adding processing functionality to locations along the data path to storage. While these efforts have been effective, they have taken a somewhat limited view of the potential benefits using some additional compute resources can offer both to enhance a compute application as well as to offering a way to exploit HPC-style resources for non-traditional tasks. Over the last few years, we have been experimenting with the potential for other sorts of activities using a staging style approach to add or enable new functionality. The efforts in this area have yielded a collection of small projects that yield some insights into both the potential and limitations of this approach for both achieving exascale computing and for enabling alternative uses for HPC resources.
ieeexplore.ieee.org