Improving Profile-Based Optimization
Our research improves the understanding of profile-based optimization. The research will create tools to extract information and conduct studies using these tools to understand how source and trace data are interacting and affecting optimizations. This research classifies program workloads in order to develop benchmarking workloads that yield better overall performance. This research enables one to identify workloads that have similar profiles and performance improvements. As a result, it will shed light into the black box that is profile-based optimization in order to more effectively use it.
This research is an extension of research begun during an independent study in the Spring Semester of 2014. Our previous results, although unfinished, indicate that the performance improvements to the optimized program are highly dependent on the benchmarking workload and vary significantly.
Workload Aware Database Storage
This research seeks to provide sufficient hints from the database storage engine to the underlying storage system to allow it more efficient data management and I/O operations when accessing data when executing queries. With OLTP workloads, the idea is to automatically partition the data according to continuous monitoring and observations what tuples are accessed (read or updated) together. Similarly, for OLAP workloads, the DB storage engine will monitor what columns of data are accessed together and provide hints to the storage system so that it can collocate, distribute and/or partition the data as necessary.
Automated Workload Mapping and SLO Creation
The use of consolidated storage systems requires deep understanding of performance requirements as well as suitable tools to manage such requirements for configuring such systems. As advanced features such as Quality-of-Service (QoS) are introduced, most customers will have difficulties to configure these features. This is mainly due to the fact, that state-of-the-art storage products integrate various kinds of devices and offer customers several interfaces to seamlessly access file system space in their infrastructure. Common features include high availability, overbooking, de-duplication, and even support for configuring priorities for achieving desired levels of Quality-of-Service for individual workloads sharing the same system. This results in multiple QoS criteria to be taken in account and to be optimized in order to guarantee proper operation of the consolidated storage environment.
However, the business interest does not lie in mere storage management, but in fulfilling the requirements of applications and higher-level services, from which it is in general difficult to deduce specific storage QoS parameters. Here, this fellowship proposal is aiming at creating one essential pillar linking business objectives and storage-level QoS parameters through an automated mapping approach. Instead of setting priorities on certain parameters, we propose research and development tasks for describing the desired Quality-of-Service at a higher level (called Service Level Objectives) and automatically map those to storage requirements including the configuration parameters necessary to configure the storage system. This approach simplifies the configuration of the system and also allows administrators more easily to verify whether the target desired QoS level is has been achieved.
This research work will create suitable application models based on workload analyses of typical business applications. As a result, tools will be provided which allow workload modeling on this application level and to automatically create suitable storage QoS/SLO configurations for this workload. The fellowship will, in addition, cover the verification of the models through simulation and testing on real systems. Based on these results, adaption mechanism will be evaluated to dynamically verify whether the workload for the forecasted application setting fits the original modeling and to propose or initiate changes to the QoS/SLO configuration of storage systems.
Workload Modeling for Storage Systems and Networks with Markovian Arrival Processes
This project proposes to use Markovian Arrival Process (MAPs and MMAPs) to classify workload traces and to use MMAPs to generate synthetic workload traces. In order to do this, we propose to enhance current MAP fitting techniques in at least two ways, namely
- Achieve an automated MAP fitting technique that obtains a MMAP by observing workload traces such that we can distinguish different facet of the request arrival process
- Develop a variant of MAPs that take explicitly takes phases into account since interarrival times of file IO requests often contain phases of user request where in the statistical properties of the interarrival times differ significantly between adjacent phases (compile phase versus install phase).
As a result of the MAP fitting techniques, we obtain many important statistical properties of workload traces (moments, joint moments, autocorrelation) that provide a base for characterizing and comparing workload traces. The resulting MAPs or MMAPs can be simulated to generate synthetic workload traces. With these steps, we seek to obtain modeling, techniques and tools that help to generate more accurate workload models than what is possible with current state-of-the-art techniques.