Multimedia Gateway Architecture for Adaptive Content Distribution

Introduction

Distribution of multimedia streams across communication networks consumes high bandwidths. High requirements resulting from this often cannot be met due to resource limitations (CPU, memory, network, display), especially when dealing with mobile devices. To overcome this problem, the multimedia streams have to be adapted to the capabilities and requirements of mobile devices. One promising approach to meet the requirements of such devices is to transcode each stream either into another or into the same coding format with other coding parameters while it is transmitted to the client.

In this project investigations on mechanisms and protocols to support mobile devices accessing high quality multimedia streams should be done. For this purpose a multimedia system consisting of several multimedia gateway, which is capable of converting multimedia streams into a requested format or reducing the quality and therefore the required bandwidth of the stream, will be developed. During a multimedia transmission a migration from one gateway to another needs to be supported by the system in order to deal with the mobility of the clients.

Objectives

The comprehensive objective of this project is the development of an RTSP/RTP based system which provides a flexible and adaptive access to multimedia content for users of mobile devices. The implementation of this system will be based on multimedia gateways which act as a proxy between media server and client. The requirements of the system can be summarised as follows:

1. The system should support different transcoding techniques which can be loaded at runtime.
2. From the users point of view a high level of transparency should be archived. In other words: the existence of the gateways should be completely hidden for the user of a mobile device in the ideal case.
3. The choice of a transcoding technique to be used is done on the gateway according to the capabilities and requirements of the client as well as other information about the requested media stream and the current state of the system.
4. The mobility of the users has to be considered and needs to be supported by suitable methods.
5. To support more than one clients a load-balancing mechanism needs to be implemented between the gateways.
6. The network load should be reduced by using suitable caching strategies within the gateways.
7. The multimedia transmission should be based on Internet standard protocols RTP and RTSP.
8. The requirements on the client application should be rather low. To use the multimedia gateway system a standard RTP/RTSP-client should be suitable but optimal usability can only be archived by the use of an optimised client.

Through the chosen gateway architecture a high level of flexibility and independency of different media server implementations is archived. An alternative approach would be the integration of the transcoding techniques into the media servers. But then the transcoding would only be possible for streams from servers which are based on such implementations. Additionally, the adaptation would be a bottleneck of the multimedia transmission when there are many requests on this server. Whereas by the use of gateways the transcoding techniques are available for multimedia streams from every reachable server. Moreover, by the use of several gateways in one network a load balancing between the gateways could be archived.

For the implementation of this system the architecture of the proxy system as well as mechanisms and protocols of the following subareas hast to be investigated:

1. Service Location
   Before any multimedia transmission could take place the client application on a mobile device needs to locate the service provided by the system. Because of the system consisting of several gateways an appropriate one needs to be located. If there are more than one gateway available an optimal choice has to be made. Therefore, suitable methods needs to be investigated to make such an optimal choice.
2. Capability Exchange
   From the users point of view the usability of the multimedia gateways should be as easy and transparent as possible. Therefore, the configuration complexity of the client application should be minimized or ideally completely omitted. However, the requirements of resources on the client's device should as low as possible. The choice of the resulting stream format as well as the transcoding technique to be used has to be made on the gateway to reduce the requirements on the client. Moreover, this is also reasonable and necessary because only the gateway itself has information about its supported stream formats as well as its resource capacities and utilisation. To make such a choice the gateway needs information about the capabilities of the mobile device. Especially display resolution respectively the desired resolution of the stream, the processing power, the supported media formats as well as the available bandwidth are relevant for the choice of the transcoding method to be used. The exchange of such information should be automated but also a manually configuration of the client application should be possible.
3. Mobile Clients
   For usability of the system an optimised client application which can profit from the capabilities of the gateway system needs to be developed. This application should be ported to different platforms to be able to investigate and evaluate the characteristics of different mobile devices such as PDAs.
4. Load Balancing and Gateway Handover
   To be able to support multiple users as well as the mobility of their devices it is necessary to develop mechanisms to migrate an active transmission from one gateway to another. If a user is moving towards another gateway and the connection to this gateway becomes more favourable the migration of the active session should be taken into account. But this migration must not disturb the active presentation on the clients device and it should not even be noticeable by the user.
5. Caching Strategies
   To reduce the network load between media server and the gateway it is sensible to cache the stream on the gateway to server future requests directly from the gateway. In this case it is especially necessary to develop caching strategies beyond traditional caching problems and replacement strategies which are usable in conjunction with transcoding techniques. For example it needs to be clarified which media format and which quality level of the stream should be cached - the original version or a transcoded version of the stream.
6. Transcoding Modules
   To reach a high level of flexibility of the proposed system as well as resulting from the fact that no optimal transcoding technique for all situations exists it is necessary to implement different transcoding modules. Furthermore, those different modules are needed to be able to evaluate the system later on. The transcoding modules can be loaded by the gateways at runtime according to the information which they got by the capability exchange.

Project members

If you have any questions, please feel free to contact Dr. Jens Brandt.

Publications and presentations

• Jens Brandt and Lars Wolf: Impact of Adaptation Dimensions on Video Quality, in Proceedings of the 14th IEEE International Symposium on Consumer Electronics (ISCE2010), Braunschweig, Germany, 2010 (BrWo10, BibTeX, Slides)
• Jens Brandt, Martin Gutbrod, Oliver Wellnitz and Lars Wolf: Plagiarism Detection in Open Access Publications, in Proceedings of the 4th International Plagiarism Conference, Newcastle Upon Tyne, UK, 2010 (BGWW10, BibTeX, Slides)
• Jens Brandt and Lars Wolf: Adaptive Video Streaming for Mobile Clients, in Proceedings of the 18th International workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV'08), Braunschweig, Germany, 2008 (BW08, BibTeX, Slides, Poster)
• Jens Brandt, Jens Trotzky and Lars Wolf: Fast Frame-Based Scene Change Detection in the Compressed Domain for MPEG-4 Video, in 1st International Workshop on Future Multimedia Networking (IEEE FMN'08), Cardiff, Wales, GB, 2008 (BTW08, DOI, BibTeX, Slides)
• Jens Brandt and Lars Wolf: Multidimensional Transcoding for Adaptive Video Streaming, in Proceedings of the 17th International workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV'07), Urbana, Illinois, 2007 (BW07, BibTeX, Slides)
• Verena Kahmann, Jens Brandt and Lars Wolf: Collaborative Streaming in Heterogeneous and Dynamic Scenarios, in Communications of the ACM, Special Section on Entertainment Networking - Recreational Use of IP Networks, 2006 (KBW06, BibTeX)
• Jens Brandt and Lars Wolf: A Gateway Architecture for Mobile Multimedia Streaming, in European Symposium on Mobile Media Delivery (EuMob06), Alghero, Italy, 2006 (BW06, BibTeX, Poster)
• V. Kahmann, J. Brandt and L. Wolf: Flexible Media Reflection for Collaborative Streaming Scenarios, in Proceedings of the IEEE Tenth International Workshop on Web Content Caching and Distribution, Sophia Antipolis, France, pages 71-76, 2005 (KBW05, BibTeX)
• M. Dick, J. Brandt, V. Kahmann and L. Wolf: Adaptive Transcoding Proxy Architecture for Video Streaming in Mobile Networks, in Proceedings of the IEEE International Conference on Image Processing (ICIP 2005), Genova, Italy, pages 700-703, 2005 (DBKW05, BibTeX)
• Jens Brandt, Verena Kahmann and Lars Wolf: A Flexible Reflector for Media Streams, in Kommunikation in Verteilten Systemen, Kurzbeiträge und Workshop der 14. GI/ITG-Fachtagung, Lecture Notes in Informatics, pages 41-48, Gesellschaft für Informatik, Bonn, 2005 (BKW05, BibTeX)

• Jens Brandt, Presentation about ongoing PhD work at Dagstuhl-Seminar "Content Distribution Infrastructures", May 2004; Slides

Theses for this project