ABSTRACT

In a world where all devices will be interconnected, the boundaries between the different devices will start to disappear. Devices will be able to access each other's applications; sessions can be suspended on one device and resumed on another device; devices can serve as each other's input and output device, and all devices will be able to connect to the Internet. This will give true mobility to the user as he/she will not be restricted to the time and location where he/she accesses an application.

Of course, we need a variety of different mechanisms and technologies to enable this, such as:

• Remote rendering of UIs on other devices in the network.
• Infrastructure for discovering client and servers in a network.
• Mechanisms to exchange capability information between devices, and to adapt the UI based on these capabilities.
• Mechanisms to deal with session migration.
• Support for a wide range of consumer devices, ranging from mobile phones to high-end TVs.

This requires technologies that cross different domains, i.e. the PC domain, mobile domain, and TV domain. Several major companies within these different domains have decided to work together on these issues. One of the results is a framework for remote user interfaces for both UPnP networks and the Internet. This framework is called Web4CE (a.k.a. CEA-2014) [1], and has been accepted as the baseline remote user interface technology within the Digital Living Network Alliance (DLNA) [2], which is a large industry-wide effort for creating true interoperability between network-enabled devices.

This paper provides a short overview of the Web4CE framework, and some of the use cases that it enables.

INTRODUCTION

Despite the fact that the web and other networking technologies have become increasingly pervasive, we still have a long way to go before all devices will truly interoperate and cross the boundaries of different domains, be it the PC, mobile, and TV domain and/or the in-home networking domain and the Internet.

Since most technologies are focused on a single domain, current networking technologies are insufficient to provide interesting user scenarios such as the one depicted in Figure 1, whereby a mobile phone is used to connect to Google Maps to plan a route, but since there is a big screen TV in the vicinity, the TV is used as output (with the user input provided on the mobile phone):

The Web4CE framework [1] is a first step towards a unified framework that enables such use cases. The next section describes the main features of Web4CE in more detail.

FEATURES OF WEB4CE

The Web4CE framework (as depicted in figure 2) adopts a number of technologies for both in-home and the Internet that span multiple domains. These technologies are: XHTML, DOM level 2, CSS TV Profile [3] (which is a superset of the CSS Mobile Profile) and UPnP [4]. It also supports AJAX (i.e. XMLHttpRequest) [5] to support the latest web applications, such as GoogleMaps, and to enhance interactive applications such as Electronic Program Guides (EPG). The collection of technologies to describe the user interface (i.e. XHTML, CSS TV profile, AJAX, etc) is called CE-HTML.

Several extensions are needed on top of these existing standards to fully support use cases such as the one depicted in Figure 1. The following subsections describe some of these extensions.

Capability exchange

In order to deal with a wide variety of CE-devices, i.e. ranging from TVs to mobile phones, a mechanism is needed for exchanging UI and A/V capabilities. This mechanism allows a server to adapt the UI to the capabilities of a client device. In case of Web4CE, also resource-constrained devices (e.g. a mobile phone) may act as a server. Existing technologies such as CC/PP are too demanding for resource-constrained server devices. Therefore, Web4CE defines a lightweight capability exchange mechanism, which is based around a few base abstract profiles:

• "SD_UIPROF" (i.e. default profile), which describes clients with a 640x480 screen, a base set of keys and fonts that are readable for different screen sizes and viewing distances.
• "HD_UIPROF", which is used for 1280x720 displays.
• "MD_UIPROF", which is used for lower-end devices with at least a 208x208 display, with 4096 colors and fewer keys.

Web4CE defines an XML language to describe extensions on top of these profiles. Servers may take these extensions into account for adapting the UI. However, since these are strictly extensions to a basic profile, capability matching can in principle be done by simple string matching on the profile name, since it is the profile that determines if the client is able to render the UI correctly. Web4CE also defines a mechanism for a server to list the capabilities that it requires to correctly render the UI. This allows clients to filter out compatible UIs before a connection is made.

Remote control / TV-based interaction

An important part of the Web4CE framework is to provide support for showing applications on a TV, and to interact with these applications by using a remote control (incl. text-entry and focus navigation). Since current standards are lacking in this area, Web4CE defines the necessary extensions to handle remote control keys, and introduces e.g. a tag (called the <op>-tag) that allows key codes to be mapped on the physical labels used on the remote control (or device, such as mobile phone). This allows authors to give feed-forward information, e.g. "Press <op code="VK_ENTER"> to continue", without having to worry which physical label the device actually uses (e.g. "RETURN", "OK", "SELECT", or another representation of the VK_ENTER-key).

Two-way communication

Two-way communication between a client and a server is an important requirement for interactive consumer services. The currently de-facto XMLHttpRequest mechanism [5] can be used in most cases. However, use cases, such as the one denoted in Figure 1, require key events and/or page updates to be forwarded to the client in a timely manner, without introducing significant delays (which may be introduced when using 'polling-based' methods, such as XMLHttpRequest). Therefore, we introduce the so-called "NotifSocket" scripting object, which can be used for setting up a TCP/IP connection between a client and a server through Javascript and to send/receive data over this connection.

Another mechanism we introduce is so-called third-party notifications. These allow a server to send notification messages to interested clients at any moment in time, outside of an active session between the client and the server. This is useful for example to send a caller-ID notification to a TV when a phone-call or a text message has been received on your mobile phone.

Session Migration

In order for a user to move a session from one device to another (e.g. when starting a session on a mobile phone, and moving the session to the TV), a session migration mechanism is needed. Web4CE supports this using a mechanism called save-restore that uses a Javascript API for storing the current client-side state of the UI (e.g. cookies, important Javascript variables) on a server in the network, which can then be resumed on another client (or the same client) at a later time.

A/V integration

Access to A/V content is a very important use case for consumer electronics devices. Since current standards are lacking in this area, Web4CE defines several mechanisms to fully integrate A/V to be part of a web-page. It allows for example alpha-blending and overlaying parts of the UI on top of video, controlling the playback of an A/V stream, and controlling the size of the video window. It also allows the A/V to persist across different pages.

CONCLUSIONS

Accessing web-based applications served by devices in the home and on the Internet on a variety of different CE-devices has a lot of interesting opportunities. The Web4CE framework (which is supported by important industry consortia such as DLNA and CEA) has all the mechanisms and ingredients in place to achieve this, and provides a sound technical foundation to build upon.