On-demand multimedia server clustering using dynamic content replication

Abstract

This thesis examines the provision of on-demand multimedia streaming services using clusters of commodity PCs. In the proposed HammerHead multimedia server cluster architecture, a dynamic content replication policy is used to assign non-disjoint subsets of the presentations in a multimedia archive to cluster nodes. Replicas of selected presentations can be created on more than one node to achieve load-balancing, increase performance or increase service availability, while avoiding complete replication of a multimedia archive on every node. Since the relative demand for the presentations in a multimedia archive will change over time, the assignment of presentations to nodes must be periodically reevaluated. A group communication system is used by the HammerHead architecture to implement a cluster-aware layer, which maintains the aggregated state of the commodity stand-alone multimedia server on each cluster node. The aggregated cluster state is used to redirect client requests to specific nodes and to implement the dynamic content replication policy. By replicating the aggregated cluster state on each node, the client redirection task can be shared and the implementation of the dynamic replication policy can tolerate multiple node failures. The HammerHead architecture proposed in this thesis is the first multimedia server cluster architecture to combine the use of group communication with the implementation of a dynamic replication policy. The Dynamic RePacking content replication policy, which has been used in the prototype HammerHead server cluster, is a significant improvement of the existing MMPacking replication policy. Dynamic RePacking separates replication to achieve v load-balancing from replication to increase the availability of selected multimedia presentations, allowing increased service availability and performance to be traded against storage cost. In addition, replicas are assigned to nodes in a manner that allows loadbalancing to be maintained when nodes fail. Performance results obtained from a prototype HammerHead cluster and from an event-driven simulation show that Dynamic RePacking achieves a level of performance close to that achieved by replicating an entire multimedia archive on every cluster node, while significantly reducing the required storage capacity. It is believed that this is the first study to evaluate the performance and behaviour of a dynamic replication policy, outside a simulation environment.