Tag Archives: hipc

Modeling Complex Storage Systems

hipclogo.gif Thirumale Niranjan, Sai Susarla, Chiranjib Bhattacharyya, and K. Gopinath.

In this tutorial, the presenters explained the details of modern storage systems and how to construct models for intelligent storage management.

A Tutorial Presented at the International Conference on High Performance Computing 2009 (HiPC ’09)

Abstract

Applications are becoming more and more data intensive, leading to petascale and exascale storage systems. Consolidation in this space is occurring rapidly in the form of Cloud Storage. As more applications share a common storage infrastructure that itself gets more complex, the need for automated management increases. Reasoning about such systems requires sophisticated models. In this session, the presenters explained the details of modern storage systems, and how to construct models for intelligent storage management. The presenters surveyed different modeling approaches such as white-box, black-box and grey-box modeling, relative fitness modeling, and hierarchical/compositional modeling that others had employed in the past, and discussed successes and failures. The mathematical basis for much of model-based autonomics is in the area of statistical machine learning. The presenters elaborated on the range of machine learning tools, the mathematics behind them, and how they can be used for modeling. The session was concluded with a discussion on the many interesting open problems in the area.

 

A Load Balancing Framework for Clustered Storage Systems

hpic08.pngDaniel Kunkle and Jiri Schindler.

This paper presents a load-balancing framework for high-performance clustered storage systems that offers a general method for reconfiguring a system facing dynamic workload changes.

The load balancing framework for high-performance clustered storage systems presented in this paper provides a general method for reconfiguring a system facing dynamic workload changes. It simultaneously balances load and minimizes the cost of reconfiguration. It can be used for automatic reconfiguration or to present an administrator with a range of (near) optimal reconfiguration options, allowing a tradeoff between load distribution and reconfiguration cost. The framework supports a wide range of measures for load imbalance and reconfiguration cost, as well as several optimization techniques. The effectiveness of this framework is demonstrated by balancing the workload on a NetApp Data ONTAP GX system, a commercial scale-out clustered NFS server implementation. The evaluation scenario considers consolidating two real world systems, with hundreds of users each: a six-node clustered storage system supporting engineering workloads and a legacy system supporting three email severs.

In Proceedings of the International Conference on High Performance Computing 2008 (HiPC ’08)

Resources

  • A copy of the paper is attached to this posting.

load-balancing-hipc08.pdf