Portolano/Workscape:
Charting the new territory of invisible computing for knowledge work
A proposal to DARPA in response to BAA 99-07
from the Workscape group at the Xerox Palo Alto Research Center
and the Portolano group at the University of Washington.
COMPARISION WITH ONGOING RESEARCH
INNOVATIVE CLAIMS
Computing and telecommunications are maturing, and the next century promises a shift away from technology-driven general-purpose devices. Instead, the focus will be on the needs of consumers: easy-to-use, low-maintenance, portable, ubiquitous, and ultra-reliable task-specific devices. Such devices, although not as limited by computational speed or communication bandwidth as their predecessors, will instead be constrained by new limits on size, form-factor, and power consumption. Data generated by these devices will need to be injected into the Internet over a variety of wireless media. Data collected at the services will also have to find its way back to the devices, wherever they may be and whenever they may be again connected to the network.
These are not simply the problems of ad-hoc networking and palm-sized PDAs. For computing to reach the broadest spectrum of our population, the entire infrastructure and the devices themselves must be as invisible as possible, requiring little or no configuration and performing reliably and predictably. To accomplish this requires a review of our basic assumptions regarding user interfaces and network transactions. It challenges us to develop entirely new models for distributed services. User interfaces will have to be based on multiple types of input ranging from the user’s physical movement, location, data available from the network, and a variety of sensor data we can only begin to imagine today. Network topologies will be intermittent and services will have to be discovered independently of user guidance. The network fabric will have to provide computing and storage cycles to the data bundles. Data will need to find its own way from the user to the services and back, possibly replicating itself along the way to prevent data loss. We will require an open services architecture that not only permits users to have their data directed to the appropriate services but also allows the services to interconnect with each other.
Our proposal is to work in each of these areas to create a prototype of the future consumer computing landscape. We will deploy and study the elements of this vision.
User Interfaces: New modes of interaction such as user movement, proximity of devices, and embodied information presentation will augment the keyboard, pen, audio, and video interfaces we see today. Data fusion (combining the data gathered from location sensors, identification tags, and on-line databases) will be crucial in determining user intent rather than relying on user commands.
Network Infrastructure: The networking fabric must provide robust data transfer with replication and discovery as well as the ability to marshal computing resources at internal network nodes. The network must be data-centric in that transmission, routing, authentication, and resource reservation should be handled independently of the location and the media from which the user injected or will receive the data.
Distributed Services: Rather than abstract capabilities, emphasis must be placed on applications to which users can easily relate. These services will have to be more openly organized into horizontal layers rather than the vertically integrated monolithic services of today to better facilitate consumer choice. New development environments are needed to make services (with a palette of user interfaces) easier to design and deploy into the network and partition across our myriad computing devices.
1. Introduction
As computing and communication technology advances and matures, we will transition into a consumer-driven mass-market. This market will consist of a wide range of interconnected task-specific devices and, unlike most of the past history of computing, the motivation will come from function and convenience rather than technological prowess. Our challenge is to build applications:
A simple analogy inspired by Donald Norman [Norman98] illustrates our vision. Today's desktops and palmtops are multi-purpose tools — electronic Swiss Army knives. But how many of us would use a Swiss Army knife for preparing a dinner at home? It may be fine on a camping trip, but not for more routine activities where efficiency and quality are more highly valued. Instead, we use specialized tools such as paring knives, tongs, stirring implements, and a variety of food processing appliances. Moreover, we have a very easy time borrowing a carving knife from a neighbor or using one in a kitchen other than our own i.e., the affordances or visible properties of these task-specific tools makes their usage predictable even in unfamiliar settings.
This is not the case for computers today. They are too complex by virtue of trying to be all things to all people. Although we may encounter them everywhere, we cannot easily predict how to use them for our immediate purposes; they often operate in an unfamiliar way, attempt to do too much, or use services we may not (or should not) be able to access. Consumer computing devices should be just as easy and flexible to use as kitchen utensils. The focus needs to shift from the innards of devices to the uses and services that make sense in our daily lives.
Such a conceptual shift becomes possible as technology ceases to be a limiting factor. We can expect in the very near future to pay less than $10 for 1GB of RAM, a GFLOP/sec of computation, a megapixel of display area, 1Mb/sec of wireless communication, or 100GB of disk space. New computing capabilities will also be common-place: speech and handwriting recognition are rapidly improving and disseminating, vision-based gesture recognition is coming out of the laboratory, and wireless technologies are proliferating and with ever-increasing bandwidth. Finally, the decreasing size of individual components means that progressively smaller and lighter computational devices may be realized.
But before these changes radically alter the consumer landscape, crucial research must be carried out in at least three proposed research areas: user interfaces, network infrastructure, and distributed services.
We will first describe a scenario that will highlight our vision and the major features of the new landscape we will explore. Then we will dissect that vision to illuminate the areas we believe require further research to bring to realization. This will serve as a preamble to the discussion of the three areas that will follow.
2. A view of the new territory and its main features
For too long, our computing tasks have been driven primarily by technological innovation. The fact that, for most users, the tasks they care about should be accomplished easily, ubiquitously, and worry-free, seems to have gotten lost in the drive for higher-performance and ever-expanding feature lists. This situation must change if we are to truly embed computing and communication into our daily lives and involve the great majority of our population and not just those of us that are technologically savvy. If computing and communication are to become productivity-enhancing tools beyond their current instantiation, we must move from a tool-centric perspective to a work-centric perspective driven by task goals.
The landscape of the 21st century is one where computing devices are highly specialized to particular tasks, are ubiquitous consumer items, and their user interfaces are invisible to all but the most sophisticated users. Users are not concerned with such esoteric issues as file formats, configurations, and connectivity. They can expect tools to match tasks and not be dependent on ownership or access rights. They can exploit a vibrant, robust, and highly differentiated market of information services to get their objectives accomplished. Moreover, they can trust the communication infrastructure to safeguard their data and deliver it to the services to which they have subscribed or requested in a timely and cost-effective manner.
The following scenario illustrates the terrain to be explored:
Alice begins the day with a cup of coffee and her personalized newspaper. When her carpool arrives, she switches to reading the news on her handheld display, where she notices an advertisement for a new 3-D digital camera. It looks like something that would interest her shutterbug friend Bob, so Alice asks her address book to place the call. The speech recognizer mishears "Bob" as "Babs". Alice quickly interrupts the mistaken call to "Babs" by shaking her address book - this signals it to re-analyze her speech. Its second guess, "Bob", is the correct one.
Bob's home entertainment system softens the volume of his custom music file as his phone rings. Alice begins telling Bob about the camera, and forwards him a copy of the advertisement that pops up on his home display. Bob is sold on the product, and after hanging up with her, he asks his electronic shopping agent to check his favorite photography stores for the lowest price and make the purchase.
When the camera arrives, Bob snaps some photos of his neighbor's collection of antique Portuguese navigation instruments. After reviewing the photo album generated automatically by a web-based service, Bob directs a copy of his favorite image to the art display in his foyer. He also sends a pointer to the photo album to Alice and instructs his scheduling agent to set up a lunch date so that he can thank her for the suggestion. Later that afternoon, Bob will receive a notification on his PDA that arrangements have been made.
In the following section, we will decompose this scenario into its constituent parts and highlight the major issues that it raises, but first, let us briefly speculate about what sort of environment would support this sequence of events.
Alice has a news service subscription that has customized the day’s news garnered from the sources that Alice consults most often. It also includes customized advertising based on past buying patterns and community announcements. Alice loses her news connection as she exits her apartment, but it is automatically reestablished via a wide area wireless connection when she turns the next page in her (now PDA-based) personal newspaper. Alice's address book uses speech recognition software. A difficult, and largely unaddressed, issue in such software is how a user can easily and rapidly signal errors in speech detection. By augmenting the address book with small and cheap accelerometers, a shaking gesture can be used to signal an error in the speech detection. People often think of shaking things to reset them or to signal error, Alice can use her existing familar gesture seamlessly in this new augmented environment.
Bob’s decision to buy the camera involves another set of services. One acts as a shopping agent to verify that the price is the best possible and another verifies that the purchase is within budget norms. When the purchase of the camera is made, Bob’s voice-print and his bio-metric identification earring (which is only one of multiple personal items that can serve that function) authenticate the transaction. His digital camera comes equipped with a short-range radio transceiver as well as a removable cartridge. The data watch he wears can communicate with the camera as well. The two devices informed each other of their protocols and requirements when they were first brought within range of each other. When he snaps a picture, the photo data is communicated to the watch where it is held until a connection can be made to the rest of the world. Bob may have alternatively borrowed a camera which should behave the same way but observing that Bob is a new user and that previously stored photos should not be available to him without explicit permission. Bob’s watch acts as a personal storage device and as a gateway to other connections. Unlike current technologies the storage is personalized to the individual rather than the device.
Eventually, Bob will enter his home or bring his watch close to his mobile network terminal. At this point, the photo data from the wrist-watch can be injected into the wider-area network for the next step of its journey. The pictures find their way through various proxies and active network nodes to the photo album service (As a shutterbug, Bob decided to purchase the album service instead of paying on a per use basis.) The costs for all the communications listed above are borne by Bob. When the camera takes the snapshot it included his encrypted personal ID (from the earring with bio-metric safeguards) in the data packets. Lower-cost data transfer options are always considered automatically based on Bob’s past usage patterns and likely future locations (extracted from his schedule). For example, if Bob is near his home (and likely headed there), his cellular phone will not place a call to transfer the data, but rather defer the transfer to happen at home where a cheaper Internet connection is available through the home’s wireless network and network portal.
The photos exercise another set of services. The photo album service assembles the photos it receives from Bob’s camera as well as e-mail and video cameras into multiply indexed albums. They can be arranged by data, medium, and subjects (identified automatically by image processing as well as known locations of other individuals both from Bob’s sensors as well as other public information). Sending the photo to the foyer display may also involve another financial transaction to purchase rights to a copyrighted subject and, again, interaction with the budget advising service.
Finally, to make the arrangements for Bob and Alice’s lunch date, yet another set of services are contacted that use Bob’s eating preferences, past restaurant experiences, critic’s reviews, budget and scheduling constraints, and travel convenience to determine the most useful time and place. They are both notified that the date has been made with a message appearing the next time their PDA links to the network as well as having automatic entries made in their calendars as well as the restaurant’s reservation agent.
|
Device |
Function |
Wireless Links |
Wired Link |
||
|
Personal |
Home/Car |
Wide Area |
|||
|
Browser tablet |
Web browsing |
ü |
ü |
|
|
|
PDA |
Web browsing |
ü |
ü |
|
|
|
Cellular phone |
Internet/voice |
ü |
|
ü |
|
|
Home server |
Appliance controls |
|
ü |
|
ü |
|
Network portal |
Gateway |
ü |
ü |
|
ü |
|
Home displays |
Large visuals |
ü |
|
|
ü |
|
3-D camera |
Photos |
ü |
ü |
|
|
|
Earring |
Authentication |
ü |
|
|
|
|
Data watch |
Data storage |
ü |
|
|
|
Table 1. A list of devices used in the scenario, their functions, and likely links to the network.
|
Service |
Description |
|
Newspaper |
personally customized version of the day’s news and events |
|
Purchase |
buying agent that finds best values |
|
Budget |
financial assistant to check spending habits and budget constraints |
|
Rolodex |
address and contacts list |
|
Location |
keeps track of a user or device |
|
Calendar |
calendar and scheduling secretary |
|
Dining |
food and restaurant preferences |
|
Travel |
determines expected travel times for preferred modes of travel |
Table 2. A list of the services used in the scenario with a short description of each.
The important properties of the scenario are that the protagonists are minimally aware of all activity going on behind the scenes. Their user interfaces, services, and agents use the infrastructure to get the tasks done. The network fabric is designed to make this possible and a wide variety of task-specific information appliances, each specialized to perform its small set of tasks efficiently, are available to the users. The tables below provide a list of the devices and services used in the scenario and some of their properties.
3. Realizing the vision
Why is such a story line not possible now? We have cellular data networks, the beginnings of home automation, and personal digital assistants. Why can’t these elements be brought together to solve this problem? It seems as if it should be a simple matter of having the will to realize the scenario of the previous section. However, its not that simple. Each of the areas identified in the introduction (user interfaces, network infrastructure, and distributed services) has numerous obstacles that prevent the vision from being realized without further fundamental research - research that is essential if such a vision is to become a reality in the next 10 years, rather than merely more proliferation of disjoint and unintegrated technologies. We will now consider each of these areas in some detail.
3.1. User Interfaces
User interface (UI) issues are of central importance in our vision. Alice and Bob are surrounded by information appliances, but both the interactions between the users and the devices as well as among the devices themselves occur smoothly and easily. While there are certainly many issues in UI evolution raised by our vision, in this proposal we consider two that we feel are especially important.
3.1.1. Invisible Interfaces
Moving away from explicit commands and toward interfaces that implicitly take their direction from people's behavior (and from the current context) is a fundamental issue in realizing invisible computing. The challenge is to make the UI an inherent part of a well-designed tool, such that this tool fits so well into the user's task environment that it becomes invisible. Invisible in that the user is largely unaware of interacting with a computing or communication device (one traditional example is the pen as it relates to the user’s task of writing a memo – the user is focused upon the memo, not upon the operation of the pen itself). This idea is the thrust of much recent work in the HCI community including that of Harrison et al. [Harrison98], Fishkin et al. [Fishkin98], Want et al. [Want99], and Weiser [Weiser96].
Fundamental to this view of interfaces is the embedding of the new generations of small sensors into everyday objects, clothing, and the computing devices themselves. A simple example is presented by accelerometers. They are now cheap commodity electronic devices that provide an electronic signal proportional to the forces acting upon the device. Including such a sensor in a shoe could provide information on whether a user is walking, running, or climbing stairs. Multiple accelerometers could go much further. If embedded in clothing (with wireless links to a pocket data collector or a data belt) they could give us the capability of developing a user interface that can use the user’s movements and gestures as input. Accelerometers are straightforward but have yet to explored in all their potential applications. Other sensors may be able to detect a human hand passing over them and making a particular gesture, or detect muscle contractions from clothing, jewelry, or eyewear.
Accelerometers and sensors are not the only input devices and output does not only belong on LCD screens. It is likely that we will want a variety of physical objects to represent the data of importance to us. These span the range from objects that may alter their color based on weather predictions to ones that alter their position, shape, or sound if we have notifications pending. In fact, recent advances in consumer robotics are spurring exactly this type of thinking. This represents a significant move away from the world of beeping watches and telephones to a world that uses a person’s full array of senses in order to convey information. The implications and design of such a world is not well understood. The ability to create such a world and make it non-threatening, productive, and operate as "calm technology" is a daunting task.
To achieve truly seamless invisible design, we must conduct careful studies of how users accomplish particular tasks and prefer to have information made available. We will need to create prototype interfaces integrating a variety of novel sensing technologies with network connectivity, and we will want to deploy applications and services that a large community of users will find useful and therefore will exercise. This process is apt to be a multidisciplinary effort involving talent from fields traditionally less closely related to computing such as anthropology, human factors, and ethnography. This is an approach that has been successfully used in other projects at Xerox PARC. We wish to draw on these experiences to enable the artful yet grounded design of these new interfaces and embedded technologies.
3.1.2. Integrating Multiple Interfaces
The integration of data from multiple interfaces has also received rudimentary attention in the world of computing devices. We use mice and keyboards together to direct input to a personal computer. Personal digital assistants have merged those functions into a single stylus that is used for pointing and writing on a touch-sensitive screen. Our physical presence at a computer station is now being used to automatically log users in and out, thus guaranteeing the terminal is not used by unauthorized personnel. We can go much further.
Our physical presence can not only be used to customize our displays but also to get information to our current location. How we determine location raises many important privacy issues. We favor the approach where locations broadcast and user’s devices receive thus being able to use the location information while keeping the option of whether to divulge it or not to other parties.
The sensors described above can be used to bring many pieces of information about the physical world together. This data fusion can provide much more powerful and robust capabilities over relying on one interface mode in isolation. For example, presence can be integrated with accelerometer data, scheduling data, and location to determine what the user is likely to be doing and with whom. This integrated data can then be used to customize the information that may be conveyed by devices so that, for example, notifications may be turned off so as not to disturb a conversation or company confidential data may be hidden from visual displays when appropriate. Context-aware computing (CAC) is a nascent area of human-computer interaction that seeks to make this integration. Though in its infancy, there are some isolated groups exploring aspects of this such as Georgia Tech 's CyberDesk [Abowd98]. The likely result is that the fusion of data from a variety of sensors and databases will be crucial to inferring intention and/or current context.
Multiple output devices can also be used in concert. Some information might be displayed on high-resolution displays while other data is conveyed via audio feedback, speech synthesis or even changes in the patterns of light on a wall. The idea is to use the many sensory channels without having to attract users’ immediate and undivided attention, as is the case with today’s computer screens and telephones. This potentially leverages both the active "foreground" attention and the seldom-used peripheral or "background" attention of users.
A fundamental question is how all these pieces of information get to the devices that need them and what capabilities these devices require to best handle this multi-modal input and output. This will lead us into the subject of the remaining subsections on network infrastructure and distributed applications and services. Interface devices will need a way to register their capabilities with the surrounding networked environment. In turn, the network must also be able to discern when a device is no longer active or is moved out of range and whether it is likely to return. When conveying information to the user (i.e., "computer output"), the major issue will be to determine dynamically what information should be displayed for which users in which format and where to go to get that information. All this has to be done as timely, efficiently, and cost effectively as possible.
3.2. Network Infrastructure
New user interface modalities for a set of highly task-specific devices, applications, and services is one of the central elements of the invisible computing vision. However, single well-designed devices operating in isolation will not fundamentally change the way we perceive and use computing. What must accompany these devices to make them truly useful is the capability to communicate with each other and with the greater network. The devices will necessarily use a variety of methods chosen for their low power consumption, range, and confinement. They will have to discover each other’s presence as well as any network services that can use their capabilities, properly authenticate users and their proxies, and deal with intermittent connectivity.
3.2.1. Wireless Technologies
A dominant constraint for many information appliances will be their power consumption. If we assume a future where many task-specific devices exist instead of few general purpose machines, clearly users will not be interested in charging or replacing dozens of batteries, nor in having parts of their "working tools" appear broken or unusable as a result of dead batteries. It is imperative that devices therefore optimize their power usage based on what is minimally necessary to accomplish their tasks. Ideally, users should be ready to purchase new devices (because of features, fashion, or wear) before the original batteries are exhausted.
Thus, for the foreseeable future, it is likely that we will see several wireless technologies employed in concert. They will be chosen for their power and cost characteristics (assuming adequate bandwidth). For example, only a small subset of mobile devices will need to communicate with wide-area networks. Conceivably, a user would only carry one of these types of devices (e.g., a cellular phone) but carry several smaller more power thrifty devices that communicate over a link that only covers a person’s immediate space. Examples of these may include a viewing/writing tablet, an identification ring or earring, or accelerometers in shoes that track movements. Operating in concern will almost certainly require seamless interconnectivity between cellular networks, radio frequency LANs, and IrDA devices.
|
Name |
Speed |
Type |
Range |
Use |
Market |
Cost |
|
CDPD |
19.2 kbps |
packet radio |
1km |
wide area |
moderate |
medium |
|
Serial IR |
115 kbps |
asynchronous PPM |
1m |
point to point |
widely deployed |
low |
|
Bluetooth |
1 Mbps |
spread spectrum |
10m |
point to point |
still in development |
low |
|
Proxim |
1.6 Mbps |
spread spectrum |
100m |
LAN |
de facto standard |
high |
|
Fast IR |
4 Mbps |
PPM |
1m |
point to point |
widely deployed |
low |
|
Altair (Motorola) |
10 Mbps |
CSMA/CA packet radio |
30m |
LAN |
very small |
high |
Table 3. Wireless technologies and their corresponding transfer rates, ranges, uses, cost, market penetration, and costs.
Wireless technologies that have limits on their range imposed by physical obstacles have their own advantages: low power, location sensitivity, and physical security. Furthermore, there is no need to provide every device with a high power Internet connection when they only need to communicate around the immediate vicinity of an individual. For example, wireless communication at very low power can be accomplished through the person’s skin permitting two devices being held or worn simultaneously to communicate [Zimmer96]. Eavesdropping is very difficult and the two devices also know that they are on the same individual. If we use IrDa, we can leverage the fact that walls block infrared radiation. This makes it easy to create location beacons [Want95] that do not interfere with each other. Line of sight properties of IR are also useful to indicate intention by pointing (as in today’s remote control devices).
This collection of wireless technologies can be overlaid to provide multi-hop connections between a wide range of devices. Even with RF technologies, we see the development of cellular wide-area networks (e.g., CDPD), home or building sized networks (e.g., Bluetooth [Bluetooth98]), and person-sized networks (e.g., HP’s Footbridge [Neaves98]). Table 1 gives an idea of how these may be used for a collection of devices. Some devices, with links of two or more types, can serve as routers and gateways for the others that are limited to only one type.
3.2.2. Resource Discovery
In our scenario, Alice and Bob’s devices were able to discover the network services available to their owners and communicate with them and with each other without the end-user's assistance. When Alice asked to call Bob, the voice-recognition subsystem in her car called up her rolodex service (somewhere in the network), downloaded an interface to that phone, and invoked a remote calling method using the interface. Similarly, Bob’s phone was able to discover the stereo system, retrieve its interface, and soften the volume – given the current task context (i.e., incoming voice call).
Resource discovery is the subject of a number of research efforts including the RDP protocol [Perkins96] and Sun Microsystem's Jini lookup service [Waldo98]. Both require local service databases with which services can register themselves. When a client asks its environment for the available services, RDP returns a Uniform Resource Name while the Jini lookup service provides clients with a proxy implemented as a Java object. Regardless of the format, it is important for any discovery system to be self-managing since both clients and resources (including the database server) are likely to change as devices are disconnected and reconnected.
Due to intermittent connections and ad hoc networks, data may have to find services on their own without the assistance of the application that injected them into the network. This necessitates the ability of the networks to execute code in the data packet that can call on discovery functions provided on major nodes or, if it finds itself on minor or legacy nodes that only route, select the best route to follow to get to those services. To guarantee data safety, this will also require controlled replication of data packets (with finite lifetimes) and a re-direction of the copies towards the different services. For example, a photo may find itself replicated and sent to a photo album service, a diary service, and a printing service in parallel. Acknowledgements from the services that receive the data packets back to the original generators of the data are also problematic as the source may be disconnected or may have moved to a new location. Thus, replies also need to be able to call upon distributed location services (that may in turn call upon schedule and profile services) to help them find their routes to where their requestor’s interfaces are likely to be.
The infrastructure and technology of distributed computing also plays an important role in building the services illustrated in the scenario. When Bob sent his photos to the photo album service, they needed to be in a specific compression format. Rather than waste bandwidth and process the photos when they arrive at the service, the camera downloads software to run locally. The camera must discover the methods applicable to this service and be able to cache them locally. Several emerging solutions to this problem, including Jini and the Liquid Software infrastructure [Hartman96], are based on the Java language. In these models, byte-code is downloaded and executed on the client. Using the Java RMI, clients can then use the services of other devices. The CORBA [OMG98] specification and Microsoft's COM [MS98] also allow clients to execute code located elsewhere. Mechanisms such as CORBA's Dynamic Invocation Interface and COM's IUnknown interface allow clients to discover an object's interface at runtime. Unlike Jini and Liquid Software, CORBA and COM place restrictions on the interface rather than the implementation language. The challenge will be in coming to a consensus and developing an open standard that incorporates the positive aspects and strengths of CORBA/COM and Jini and provides and environment for naming, locating, controlling, composing, and debugging these types of networks and services.
3.2.3. Data-Centric Networking
Data packets acting autonomously in the network (discovering services, replicating, adapting to traffic, and performing distributed storage management) is another distinguishing feature of this proposal. Current networking protocols are based on the concept of pushing all application-specific functions to the periphery of the network and keeping the center simple and homogeneous for maximum performance. The ad hoc networking we are proposing in this proposal changes these assumptions.
First, the networks are no longer uniform in media or protocols. Devices will use specialized protocols to limit their power consumption and properly deal with the data items that they transact. A device may be designed to stay connected only for as long as it takes to transfer crucial data and no longer. There may not be time to wait for acknowledgements or carry on a dialog with a server.
Second, data bundles traveling on the network will need to marshal resources along the way for determining the location of services, determining the locations of users, and taking advantage of temporary storage. Given that all these functions are likely to be highly task specific, the best strategy might be to send the code along with the data. Network nodes will have the ability to run the code that came with the data to determine what should be done with it. In this manner, we minimize the requirements of continuous connectivity on the devices. They only need to remain connected long enough to transfer the data bundle to the next node. It is now up the data to determine where to go next. It may move on to another overlay network that gets it closer to the Internet backbone where it can finally run its code to find the services it seeks. Bob's camera, for example, is capable of transferring photos to several locations without the aid of cables or a base station. Its RF transmitter can communicate with nearby devices including Bob's wristwatch, cellular phone, and printing and display appliances. Data moves from device to device until it reaches the service it is intended for.
Third, acknowledgements need to find their way back to the devices that injected the data in the first place. For example, once the photos reach a user’s persistent storage, the camera needs to be informed that it no longer needs to cache the photo. The persistent storage now becomes the new destination for acknowledgments from the various other services that use the photo (diary, album, and printing). The camera must still be informed that its photo has been safely stored away. But the camera is mobile and its new, or most likely new, location needs to be discovered by monitoring or by inference from the calendar and location services. When the acknowledgment finally reaches it, the camera can reclaim the cache space it allocated to the photo. If no acknowledgment is received for some application or device-specific time interval, a device may want to re-inject the data into the network just in case the first copy lost its way or was trapped in a node that did not reconnect. The services must guard against these potential duplicates. In addition, an indicator on the camera may alert the user if the camera’s memory is getting dangerously full so that the user has the time to get it to a network link and have it transfer its data.
Though the ideas of ad-hoc networking are valuable, we need to re-think our basic assumptions about network operations and construct a data-centric network architecture as a means for distilling, naming, and locating the data objects that travel within the network. We expect active networking techniques [Wether98] will be valuable as we construct our infrastructure because there is a good fit with the environments in which active networking is most beneficial.
Our infrastructure is comprised of many thin-client devices that are connected intermittently by wireless channels. These connections also link thin-client devices to stationary and potentially more powerful nodes that are embedded in the local environment, as well as servers that are reached across the wide-area. In this environment, wireless bandwidth is the scarce resource whose use should be optimized, while the computation needed to support active network techniques and network-embedded services is readily available in the local environment. This means that we expect the benefits of active networks can be realized with no appreciable penalty in terms of raw network throughput for running mobile code within the network infrastructure.
The heterogeneity of our environment then offers opportunities to improve performance by tailoring network processing to the available resources, rather than relying on a default one-size-fits-all solution such as TCP/IP. Active networks provide one basis for tailoring network processing. A number of services that take advantage of such opportunities have already been identified. The TCP-Snoop protocol [Bala95], for example, improves TCP performance across wireless links by augmenting the IP processing at wireless base stations to mask losses that are not related to congestion. Other proxy agents can be used to trans-code media at points within the infrastructure depending on the network and client characteristics, for example, by decreasing the resolution of images if a small display must be reached over a wireless link [Fox96]. We expect to identify further opportunities that depend on selecting the most appropriate network paths. By expressing network-embedded services in terms of mobile code running in an active network, code injected with the data, we will be able to rapidly evolve our infrastructure as we gain experience with users and introduce new and improved versions of our services.
Another relevant advantage of the active network approach is the ability to run small programs in the network infrastructure. This provides an effective means of discovering the characteristics of network paths. Traditional protocols treat the network as a black box and infer its properties. A number of Internet measurement studies, for example Paxson's [Paxson97] rely on sophisticated inferences of traffic patterns. On the other hand, an active network allows characteristics of the network to be discovered by formulating a query program that is executed as it travels through the network. This ability will be important for our research because it will allow us to design intelligent services that adapt to operate in diverse environments. A device might determine whether or not there is a high-bandwidth path between its current location and the information services for which it serves as interfaces and then act accordingly. For example, a viewing tablet that finds itself brought back into the home and thus near an Internet portal may go and retrieve any new electronic mail that may have arrived (pre-fetching it for possible display). If it doesn’t detect the high-speed connection, it may be limited to only indicating that new mail has arrived without being able to display it quickly.
Finally, by allowing the routing of packets to be controlled, active networks will help us to utilize network-embedded services in a topology-independent fashion. Most existing systems that embed value-added processing within the network rely on knowledge of the topology that is not easily be learned by applications and is instead configured by users, for example, the location of a nearby proxy. In mobile ad-hoc networks such as ours, it is important not to rely on static network configurations, and instead discover nearby embedded services and adapt as the topology evolves. Active networks provide a flexible means of rendezvousing with network embedded services because it is possible to change the route of a packet within the network as it passes close to a useful proxy.
When Alice left home, her personalized newspaper was still available to her. It is certain that the network characteristics of her home and car are very different, so the quality of the newspaper adapts. A high-bandwidth service may be free at home, but costly to use elsewhere. Network infrastructure must be able to inform devices about the network they are using. For example, it should also be able to offer guarantees about the quality of service (QoS) to those users willing to pay a premium. Protocols such as RSVP [Zhang93] and QEX [Davies96] have laid the groundwork for effective QoS management in mobile applications. RSVP allows users to request a guaranteed bandwidth or delay bound. Those services which do not have specific QoS requirements may still benefit from knowledge about the current network state in preparing their future actions and providing feedback to the user about expected connectivity bandwidth. For example, we could design and make links in a browser easier or harder to select (based on a force-feedback device or time delay).
3.2.4. Authentication
While it is important for devices to be able to discover services, they should only be able to use those services for which they have permission. Bob's camera, for example, did not publish the photos in his neighbor's album. Likewise, Alice needs to be assured that the person who made the lunch date was, in fact, Bob and not an impostor. The need to authenticate financial transactions is also obvious.
The IPsec protocol [RFC2401] is a newer scheme built on public-key infrastructure, which may be better suited to the needs of mobile computing than distributed security systems such as Kerberos [RSA98]. However, significant challenges remain. We would like to shift the authentication burden to the services and to the data bundles injected into the network rather than placing it on the user’s interaction with the device. For example, one possible scenario is that Bob’s ID earring is used by his camera to label the photo with his identity (similarly, we could design the system to use fingerprint identification, retinal scans, personal signatures, or voiceprints – emerging technologies which all show substantial recent improvement). The photo can then move into the network and discover the services to which Bob has subscribed by contacting known directory servers. Bob can also be billed for the bandwidth and storage the data uses in its traversal to the services.
3.3. Distributed Services
The third thrust of our research is in the area of distributed services, with greater emphasis placed on applications to which users can relate. When Bob creates his new photo album, he may subscribe to several services to operate on his photos. One of these stores the photos on a reliable server in his home or at a rental site. Web pages are then prepared by the service and stored as part of the subscription arrangement. The request to forward his favorite photo to the foyer display is handled by the photo album service. Because satisfying the request may involve the purchase of rights to images, the photo album service checks with Bob's financial management service to see if he is on budget and the purchase meets his expected usage profile. Once the purchase is made, it may then negotiate with a home decorator service to determine in which situations it should be displayed in the foyer (for example, especially when the neighbor visits).
The most important message to remember about how services are used in our scenario is that they are highly interactive. The difference to existing approaches is that services here are interactive with each other and with the data users have destined for them (or implied based on the type of the data, e.g., photo). Users only interact directly with services on the rare occasion that an output interface has failed to infer which data the user is likely to want to see next. Users remain focused on their work-centric goals and not the means by which these goals are incrementally achieved by the system. Additionally, it must be possible to add new services incrementally so that users can have more or less of their computing activities (and the coordination of those activities) automated.
3.3.1. Horizontal Integration
The current focus on infrastructure has led to network and distributed systems architectures that are vertically integrated, not horizontally layered, making it difficult to quickly deploy new services. By vertical we mean those services that attempt to provide entire solutions to a specific problem. Traditionally, these suffer from high-cost and inflexibility (e.g., getting information from users that don't subscribe to the same service). An example of a vertically integrated service is an office calendar service. Multiple users can share their calendars making it easier to see time conflicts and plan collaborative activities. However, users of that service also have private lives but may hesitate to place personal information into their work environment. In addition, family members and friends may also have personal schedules that are of interest and may or may not use the same calendar service (and almost certainly are not on the same network). Today, a user can not integrate all this data easily.
We argue that horizontal integration is more appropriate for the mobile networks of the future. Each person could subscribe to a personal calendar service and instruct it as to which other services it can forward data to. For example, a scheduling service to which Bob’s employer subscribes may be able to only gather information about the time that Bob is unavailable during a typical workday and see the detail of only work related appointments. Horizontal integration also allows for more competition in services. Bob may wish to change the photo album service to one with a more exciting graphic design or get a more reliable and cheaper service for mass persistent storage. It should be possible to make these changes easily and modularly. Clearly, interactions between services will stretch the needs for directory and discovery procedures.
3.3.2. Agent Technology
Many of the tasks that Alice and Bob performed required the users’ unintentional use of a variety of distributed services. When Bob scheduled lunch with Alice, he did not explicitly (intentionally) access Alice's calendar and find an available date. A scheduling agent performed this task on his behalf - taking into account both Alice and Bob's schedules, eating habits and preferences, and convenient travel and meeting times. Yet another agent assisted Bob in publishing a photo album by making layout and presentation decisions based on his preferences.
The term agent is difficult to define. Nwana [Nwana96] describes an agent as a "component of software and/or hardware which is capable of acting exactingly in order to accomplish tasks on behalf of its user." The technology and protocols used to implement agents {umbc98} are becoming better understood, but how they can be applied to mobile applications in environments with intermittent connectivity and widely distributed data sources is research that must be further explored. A possible approach is to encapsulate agents into the data bundles or allow the data to call upon appropriate agents stored elsewhere. The distinction we make between agents and services is that agents operate on the data and make decisions while services store the data in the form of a relational database. Part of this data may, in turn, include other agents that will call upon other services.
3.3.3. Deployment of New Devices, Agents, and Services
Implementing new functionality will be a daunting task if this vision comes to fruition. Applications will not only need to be partitioned across multiple computing engines (devices and servers), but the code to accompany data in its travels also needs to be written. It is clear that we will need a new generation of development environments for design, simulation, and debugging.
API libraries will not be up to the task. Some attempts in this direction demonstrate the drawbacks. For example, the Rover toolkit [Joseph97] seeks to allow developers to partition an application into two pieces, one running on the user interface device and the other on the server. The emphasis is on mapping controllable elements to appropriate UI widgets. In addition, communication mechanisms such as queued RPC are provided to deal with intermittent connectivity between the two halves. There are two principal drawbacks. First, the developer is still responsible for the bulk of the communication code and they must deal with all the details of connecting to other sources of information that may be needed on the server end. Furthermore, any changes in state must be communicated explicitly. This places an extra burden on the developer to be constantly aware of the partition.
Another approach, based on the Chinook project at the University of Washington [Chou98], seeks to apply synthesis techniques developed for the design automation of heterogeneous multi-processor embedded systems to this new arena. Developers write an application in a single environment without any concern given to intra-application communication. After partitioning the functionality onto a set of devices (e.g., the camera, cell phone, and services), the developer then simulates the new application and collects data about communication patterns and required bandwidth. This can be used to refine the partition or alter the functionality until the design converges on an acceptable solution. At that point, the environment automatically generate all the communication code that will be needed on each device by consulting a library of available protocols that includes code to bundle with the data. The generated code is then optimized for the operating system of the device to which it is mapped as well as to the specific application requirements. Thus, the details of discovery, data migration, data safety and storage, and authentication are not at the forefront of the development effort nor do they dominate development time.
The development of such an environment is not a trivial task and involves creating a simulation environment for the devices outlined above as well as a library of communication and data bundle protocols. Furthermore, APIs to discovery and directory services as well as the relational databases that make up part of the service will also be required. Mitigating this development burden - by synthesizing communication and coordination code and deploying it automatically onto a distributed fabric - will be crucial to ensuring that applications are written by as large a segment of the population as possible.
4. Summary
We have presented a vision that we believe will require fundamental new work in several disciplines if it is to become a working reality. There are three basic assumptions behind our work:
Our expectation is that we will achieve important results in understanding the issues in the following areas:
We strongly believe that any exploration should begin with user studies and analysis of current practices. It is imperative that we shift from a technology or tool-centric approach to an application or work-centric focus. We must determine how our prototypes will meet the challenges of user-specified tasks, and how these motivate the new devices and services. We should begin with applications motivated by needs of the office and home environments. For example, data fusion should make it possible for a calendar service to adjust the morning wake-up alarm based on traffic conditions or the knowledge that the car is almost out of gas. Building these devices will expose interesting issues in all of the research thrust areas outlined above.
COMPARISION WITH ONGOING RESEARCH
We believe the convergence of hardware trends and software technologies makes now the right time to reach for making invisible computing a reality. Moore's Law has pushed the price of computing, storage and display technologies to the point where they can be feasibly incorporated into everyday consumer devices. At the same time, advances in ubiquitous computing environments, active networks, ad hoc wireless networks, speech and gesture recognition, and distributed systems based on mobile code have laid many of the foundations for our work, so that we will not need to start from scratch. In fact, many of proposers have been involved in building the foundation we will be building upon.
A unique aspect of our approach is a vertically integrated attack on all of the challenges of invisible computing -- from user interfaces to networking to distributed systems. We believe the synergy between traditionally distinct areas will be crucial to our success. For example, making distributed systems invisible requires providing seamless user interfaces so users do not notice machine boundaries. The flip side is also clear. Without an early push in the right direction from DARPA, the future is likely to offer a bleak hodgepodge of non-interoperable devices, as difficult to program as a VCR but making the situation even worse as there will be hundreds of them. If we do not move in this direction rapidly, instead of having computers serving people, we will create a bipolar society of system adminstrators servicing our own technology and Luddites resisting more complexity no matter what the benefits.
The aspects of our work that provide the greatest discontinuities with past efforts are:
The rest of this section lists some of the past and ongoing work we intend to leverage.
1. User interfaces
The most relevant work for us comes from the Xerox PARC Ubiquitous Computing project and embodied user interfaces. We can reuse and adapt several of the hardware elements derived from the ubiquitous computing work [Want95, Want99]. In addition, we can leverage the work on adding new motion and pressure sensing user interfaces to PDAs [Fishkin98, Harrison98].
Other seminal work that will be important to our mission is that of Hodes [Hodes98] on interface description languages for dynamically generating user interfaces to controllable objects that may be encountered. We will seek to generalize that work to gestural and embodied interfaces rather than limiting it to graphical interfaces.
The work of Zimmerman and Gershenfeld [Zimmer96, Gershen98] on personal area network and tagging technology will provide some of the elements of our vision in the areas of short range wireless technology and in location sensing of objects and people.
Currently there are isolated pockets of research in wirelessly tagging and tracking objects. Some of the earliest work was done at Olivetti Research with the creation of the Active Badge for tracking people in a building and using this information to create context-aware applications [Want et al, 1992]. Close to this time, work was also underway at Xerox PARC's sister research lab, EuroPARC in Cambridge, England in 1993 (Wellner, 1993). However, this prototype system, called the Digital Desk, used computer vision, video projection and much "smoke and mirrors" to simulate a truly integrated physical-virtual workplace. The prototype system was more conceptual than a working model and is now defunct (though Xerox holds several patents in the area).
Subsequent to this early work, Fitzmaurice (Univ. of Toronto, Fitzmaurice et. al., 1996) and Ishii (MIT Media Lab, Ishii and Ullmer, 1997, Underkoffler and Ishii, 1998)) have built several simple prototypes of singular tagged objects operating within confined laboratory settings. (Both are former colleagues and collaborators of one of the authors of this proposal). At PARC, we have also experimented with prototype objects and tags in simplistic settings (Harrison et al., 1997, Fishkin et al., 1998,Want et al., 1999). To date, elements of this approach remain technically challenging (e.g., range of readers, how to register actions-to-objects, parallel processing of multiple tags). No widespread or more general deployments have been attempted across a multitude of objects, nor across a multitude of settings. And only limited efforts have been made to integrate this type of technology into existing or real work settings. All current efforts have been aimed at building systems that operate in isolation of a larger work context.
2. Network Infrastructure
There has been much attention paid recently to Active Networks. The work of Wetherall and Tennehouse [Tenn96, Tenn97, Wether98] provides an ideal foundation for our data-centric networking ideas. This has already been discussed in the body of the proposal.
Jini [Waldo98] is another fundamental technology for distributed system deployment and service discovery. Although we are unlikely to place a Java Virtual Machine on every one of our devices, we are likely to borrow many ideas from that approach. Liquid Software’s code mobility will also serve as a guide for the work on code encapsulation into data bundles [Hartman96].
Mobile ad hoc networking is another crucial area for us. The IETF’s MANET group[Macker98, Perkins98] has already looked at the problems of evolving network topologies and multi-hop requirements for data transfers. In addition, they also provide guidelines for bandwidth-constrained variable capacity links which will be the norm if we are to have hundred of devices within a home and dozens around a person. Bandwidth reservation, prioritization, and congestion control will undoubtedly be issues we will also have to deal with [Choi98]. Our work will exist in a hybrid world where we will greatly leverage a wired infrastructure in addition to wireless links. This is likely to alter many of the assumptions of the MANET work and may lead to different strategies and optimizations.
3. Distributed Services
Several groups are beginning to look at organizing network services horizontally. Most notable among these is the work at UC Berkeley on the Ninja and Proactive Networks projects. Both are looking at creating composable distributed systems easily, however, they focus on simple pipeline composability [Hodes97, Fox96]. Our work is likely to borrow elements from relational databases as it seeks to combine data stored at different servers whose original services and data repositories were designed independently. In that sense, it is closer to the Bayou work at Xerox PARC [Edwards97].
The Rover toolkit is a first step toward a new development environment [Joseph97]. However, it doesn’t address some of the thorniest problems for application designers, namely, the maintenance of coherent soft and hard state. This will be especially critical in a world of highly intermittent connections.
Furthermore, unlikely the approaches above that look at static design of a system, we will be striving for self-tuning capabilities. This will be the case not only for performance but also for efficiency in handling replicating data and the multiple acknowledgments they will generate.
4. Workscape-like Infrastructures
Architectures for systems integration are prevalent in many areas of endeavor. The ones most relevant to the Workscape are from the areas of human computer interaction, real-time control, and artificial intelligence systems.
In [Walker 98] an architecture for ‘groupware’ (shared text editing, etc.) is presented that supports the sharing of information from a variety of sources through the propagation of events. Interest in events is declared through a registration process (much like in the Distributed Events in [Wald098], and other numerous publish and subscribe systems). Here an event is a change to the value of some attribute (an intermediary could construct events that are based on the state of multiple attribute values via aggregation). This architecture addresses the concerns of component heterogeneity through modularization -- having communication occur only through event propagation.
An architecture for intelligent rooms described in [Coen, et.al.] utilizes a modular system of software agents that collect and move sensor and derived information. Here, the parallel operations of the agents are used to address multiple events simultaneously and allow for quick changes in context. The agent separation also supports the need for separating computation about activities that deal with different conceptual concerns, and also increases robustness by allowing agents to come and go without reinitializing the entire architecture. (The discovery and Join mechanisms in [Waldo98] also support the dynamic appearance and disappearance of components.)
In industrial style real-time control systems, [Glass83], time constraints are paramount, and the scheduling of activities to meet these constraints is a primary focus. Deterministic, bounded algorithms are often the norm so that schedules can be accurately constructed. Due to the criticality of the service these systems must provide, sensors and actuators are often operated in a polled manner to increase robustness. Elements of real-time control systems are relevant to the Workscape since there can be critical time and resource dependencies between actions. In [Larner 90], a blackboard based architecture is described that addresses real-time control through modularity of conditions and actions, and provides support for both polled and event driven computation.
Uses of both declarative and procedural approaches are common in artificial intelligence systems, and heterogeneous modularity is particularly prevalent when these systems are distributed [Bond 88]. The declarative style is especially useful to dynamically infer (and subsequently retract) consequences, allowing these systems to adjust to changes in state, and context. Typically, procedural approaches are used to coordinate declarative activity, and to capture functionality that is either difficult to express declaratively, or has other specific purposes (e.g. interaction with other systems, special efficiency concerns, etc.). Explicit representations of control strategy allow introspection and subsequent shifts in focus due to changing conditions.
STATEMENT OF WORK
Initial Scouting Task
We propose to "time travel" to the future world of invisible computing by deploying enough technology (both hardware and software) to create a realistic prototype of its terrain. Our goal is to enable the principled study of the issues this paradigm presents. Invisible computing promises to break the complexity barrier created by the dominance of the personal computer in managing our information. We will bring together embodied user interfaces, wireless technology, active networking, and network-based services to create a seamless information web around individual users and their work and living spaces. Information will flow mostly invisibly as devices will work from inferred user intent and not just the explicit commands we use today. Information will be safe and indexed without the user focusing on file systems or data transfers and storage. Information will be managed by a new economy of distributed services. Applications will be developed with an advanced development environment that lets designers work on function and added-value rather than low-level distributed computing.
We will break up the description of our work into the same three research thrusts as described above. Our rationale for the base request and optional funding is to permit a second iteration of further refinement of our testbeds and permit the careful study of real users of the new territory (the option starts one year later and also runs for three years).
1. User Interfaces
We refer to all the means by which users enter and retrieve/view data from embedded network services as user interfaces. We will we will deploy a rich collection of devices, both wireless and wired, upon which we have created innovative user interfaces that are appropriate for home and office environments.
Embedded web servers both commercial and of our (smaller and more invisible) design will be used to connect stationary appliances to the network. In addition, they can serve as gateways between the wireless and wired domains. We will interface wall-displays, control panels, digital cameras, fingerprint scanners, and similar devices to the wired/wireless network infrastructure. We will develop several embodied user interfaces focusing mostly on the use of small accelerometers and pressure sensors to provide input from the handling of an object or movement of a person.
We will create a new IrDA compatible location beaconing system based on components left over from the Parctab project, part of the previously DARPA-funded Ubiquitous Computing project at PARC. In addition, we can leverage this proposal with convenient key-chain sized personal computers, designed at PARC in 1996, called Minders, which are small enough to be carried on a key-chain and store user preferences along with short messages. They have IrDA transceivers that allow casual communications with the surrounding computing environment. We will augment personal digital assistants with wireless links that work at different ranges and with embodied user interfaces.
2. Network Infrastructure
Data-centric networking is crucial if we are going to have hundreds of wireless devices per individual. These devices will be heavily power constrained and will use connection time sparingly. Data bundles transferred by the devices will include within them instructions about their importance, urgency, replication, life-times, and destination services. The data will have to be able to marshal computational and storage resources of network nodes. Billing for these services will be another issue for our investigation.
Our data-centric network can be conveniently constructed on top of an active network fabric. Active networks can naturally be extended to accomplish our tasks by deploying our own code to handle the data bundles injected by our devices. This will include general utility functions that do not need to be included with the data but can simply be "called" when the data arrives at a node. In addition, we will need a safe language for writing data/application-specific code to be executed by the active network.
Devices that we prototype, and especially those that will serve as gateways between interconnection media, will need complementary networking code to create the data bundles and/or move them toward their destination if they are acting as only a stepping stone. In addition, a subset of the devices will implement longer-term storage functions as well as managing returning acknowledgements while they await a reconnection from the device. One of the challenges will be to ensure that data is not lost, not over-replicated nor over-written, and properly acknowledged in ad hoc mobile networking environment.
In addition to these functions, we will also develop a service directory and discovery service that supports our data-centric model. Data bundles will be able to query for the services available to their user for their corresponding data type and then issue commands to the network node on which they currently reside to initiate a replication (or multi-casting) of the data. In addition, it will also have to take care of setting up for multiple acknowledgements to make their way back to the originating user interface or service.
3. Distributed Services
Horizontally layering of services is central to our vision. Users should be free to mix and match not only a variety of data repositories but also select a set of autonomous agents that operate on that data. The agents will be of a form that can be deployed into the network from any of the Internet-connected user interface devices. It will be possible to construct agents of varying granularity, e.g., specializing in finding family scheduling conflicts to inferring the location and time commitments of the members of co-workers.
Of particular interest is determining services dynamically so users are free to subscribe and unsubscribe at any time without concern for data losses. Transfer of data from the previous to the next service will be handled automatically. Users will not need to reconfigure their user interfaces nor will they be concerned with the precise location of the service. Consumer agents may also be designed to monitor service performance on behalf of a user and detect situations that may not meet standards set by consumer protection organizations.
For the office environment, we will construct services to handle scheduling, location tracking, diaries, and shared workspaces. We will also adapt these for the home environment and add a set of photo services with the intent of further enriching the interconnected nature of the services and the agents that operate upon them.
Other Proposed Work
Portolano infrastructure and its distribution. Extend the Workscape architecture to effectively cross many environmental locations/machines. This effort would focus on the concerns of distribution and connectivity. Concerns include: the representation of remote information in a local space; the ability to accommodate partitioned (disconnected) situations; and the cost/benefit curve associated with accessing distributed (non-local) information, where the cost is determined primarily by the amount of time it takes to obtain the information. This task would investigate approaches toward remote data representation, disconnected operations and the making of reasonable decisions in light of the cost/benefit tradeoffs. It would also include further development of special-purpose wireless devices, portals to the wired infrastructure, location/position/movement sensing devices, and actuators for various modes of display of remote information. Technical support is required distribution and maintenance of the infrastructure and interaction with other sites.
User and usage studies of wide deployment. Workscape promises to be a dramatically different kind of workplace when deployed in a large-scale setting. We would like to investigate the kinds of work practices that evolve in this environment when it is used by more than a small group of the original builders, and use that information to in turn affect future design developments. Ideally, this task would not just inform us about the design of Workscape, but also yield insights into how work can happen in richly interactive environments.
We anticipate that if Workscape is reasonably deployed (i.e., has more than 10 daily users in the non-builder group), over a reasonably large area (two working group pods), then a user study would require 1 research assistant, and 2 summer interns for data analysis. The output would be a series of detailed reports on what actually happens in the Workscape use setting, and assessments about which aspects of Workscape (rough documents, sensing, tagging, etc.) appear to have the largest benefits.
Ubiquitous large displays. Our goal is to publish information into a working groups' peripheral awareness. One method we wish to test is how well large, public displays (mounted in hallways at key locations, and in public venues such as the lobby or public auditorium) can communicate large amounts of collected and integrated information.
Large wall displays have been used before in many settings, but none has studied how to actually use group displays effectively with a constantly changing set of viewers and dynamic underlying set of data to display. Viewers will be able to notice something of interest on the public display and then "drill-down" easily for additional depth.
The public displays will have personal tag identifiers, and can personalize their content to each person (or set of persons) walking by. In this investigation we seek to discover what the design principles and underlying technologies are needed to place something of this kind into daily public use over a large set of people. That to end, we are seeking 12 50-inch plasma displays to be located across multiple sites, attempting to integrate information about physically separate work groups.
Application development environment. A new world rich in information services, a wide range of user intefaces, many different types of user devices (both mobile and fixed), and a rich communication infrastructure will require a new type of development environment. Application developers will be much concerned with the limitations of the hardware platform(s) on which their code will run. This task will create a new development environment that permits application developers to simulate their creations from a desktop and then deploy them on a wide range of architectures. The emphasis will be on increasing productivity through re-use with customization performed by automatic tools.
Creation of a set of office application/device suites. Using our application development environment, new Workscape application suites (along with their accompanying user interface devices) will be developed as part of this task. The objective is not only the development of useful applications for using the contents of rough documents and the integration of remote information but also the exploration of new user interface paradigms for group and tele-collboration.
Modular sensor system. The Portolano proposal describes many scenarios that assume sensors have been placed in the work environment and are providing data that services can use to some advantage. In order to deploy a variety of sensors in large numbers, it is desirable to standardize on a modular system that provides sensor components, their digital interface and a inter-connect network.
This part of the work sets out to investigate design options for such a system, leveraging existing standards wherever possible. It is intended that we build a working demonstration of a system providing a suite of sensors that include: location, occupancy, thermal, identification (via electronic tags), microphones and cameras. In some cases, such as in-building location, the sensor technology itself will be a subject of research (see next option).
Networking components may include RF and Infrared transceivers and wired interconnect such as Ethernet or RS485. The system should be able to combine all flavors of these network technologies in order to suit the situation in which they are being deployed.
This work is intended to provide a pragmatic solution for deploying large numbers of sensors in an environment that does not have a viable infrastructure for that purpose e.g. an impromptu occupation of a building.
Location sensor technologies. When discussing scenarios that utilize smart environments, one of the sensing 8requirements that repeatedly appears is the ability to accurately locate and identify objects or people. At this point in time there are no solutions that can provide accurate in-building location (certainly GPS is not a solution as the satellites are rarely visible to a GPS receiver within the walls of a building). Some commercial solutions such as 3DId by Pinpoint are beginning to appear in the market. However these have there own set of problems and early adopters report accuracy is a problem due to the interaction between the systems and various materials used for building construction.
This part of our work will examine various options for locating objects in a building with the aim of providing location to better than 1 foot (30cm) of accuracy. We may use a combination of RF signal strength and the propagation time from tag to an array of sensors . In order to avoid some of the building structural problems that disrupt existing location systems, the sensors could be placed in each room but leverage the sensor network infrastructure (described later) in order to fuse the data into a unified system.
Novel automatic identification technologies. Low-cost automatic identification technologies are either optical, such as barcodes (1D and 3D) or Vericodes, or passive RF transponders, RFID. The RFID technologies have many advantages over the optical technologies because they do not need a line of sight for their use and can be read at arbitrary orientations. The main reason they have not replaced barcodes is their cost. The cost is high because the tag consists of a silicon core, a wound copper coil and a glass or plastic encapsulation. The latest RFID technologies (e.g. Motorola’s Bistatix) are less expensive because the coil is printed from conductive inks on a label and the chip is bonded directly onto the label. These techniques are still more costly than printing barcodes and less expensive technologies that can be mass-produced are still needed. Another shortcoming of the existing commercial tags is the relative short reading range. There are some situations in which low-cost identification at a distance (say greater than 4m) would be desirable. Tags that have a long read-range and that are suitable for in-building use would enable many new applications for invisible computing.
The work proposed here would use some novel optical techniques to design tags with long read ranges. We would also investigate the use of some printable highly paramagnetic materials that may have application in the production of in-expensive tags for unique identification.
Flexible active network infrastructures. In order to test the algorithms suggested in other sections of this proposal we wish to build enough infrastructure to demonstrate the ideas are viable beyond a simple example or simulation. An active network contains entry nodes, intermediate routing nodes and end nodes. We will design and build components that support the range of algorithms we wish to test. At this stage of the proposal we predict that simple processor components may not be sufficient to support all aspects of the node design. In fact, programmable logic may be an integral part of a node design to implement the fast algorithms that are required when processing real-time data. As a result our node design will contain processor components, memory and FPGA devices which can be configured by services available on the network. Although this is a convenient infrastructure for our experimental work, it is also possible that field programmable logic will be a key part of the design of future active networks. These systems may take advantage of the notion of "just in time hardware" to tune the functionality of a network node to a particular data type that , unannounced, begins to flow through it. By taking this approach we will have the opportunity to both test active network algorithms and design approaches to "just in time" hardware.
Applied demonstrations of sensor fusion. There are many situations in which a sensor can only sense a small part of a target parameter. In order for the complete picture to form, sensor data must be combined in a systematic way. Traditional systems have used sensor arrays in which each sensor has a well defined and usually predetermined relationship with every other sensor. When an array of sensors has an impromptu deployment, it may be necessary for the system to derive the relative arrangement of its components on the fly. Such a system may learn about the location of sensor components, or the sensors themselves may learn about the presence of their neighbors as the result of data flow. There are various strategies for combining sensor data, its ID and the location from which it was derived, all with the aim of fusing data into a single consistent picture – sensor fusion. This work will experiment with various novel techniques to achieve sensor fusion and demonstrate the relative merits through the construction of a real systems.
Extended information-space management infrastructure. To effectively support a future of invisible computing, the described user interaction, sensor and networking infrastructure, and distributed service model needs to integrate with the ever-more distributed and heterogeneous sources of information that people access and more importantly need to re-access over time. The work proposed by this optional task is to exercise and further develop an infrastructure that enables such customizable, context-aware, integrated information-space management. In particular, the infrastructure:
The work to be pursued is the deployment of a prototype of the information-space manager integrating actual instances of: contextual data on information use, different information sources (as listed above), functional extensions to information, availability of information spaces from different locations and devices, automatic inference of information usage behavior, and security in support of individual and shared access to an information space .
A new user experience for distributed information flows. With many new sources of information coming online, and with invisible computing architectures in place comes a need to be able to do something with these new capabilities.
For many years, most user interface work has centered around fairly simple conceptual models of UI wedded to a single processor and a fairly small number of points of interaction. We do not yet know of reasonably effective ways to portray the information that flows through a large-scale complex network.
In this option, we will develop new ways of portraying, controlling and understanding the content of a ubiquitously located set of information sources. We will investigate the user conceptual models, ways of depicting large complex information sets and methods for expressing the goals of the users.
Making sense of complex information from distributed information sources. A common use of sensor networks and integrated information sources will be trying to identify larger patterns in the data. While sensor fusion techniques will be an enormous help, we anticipate that the problem will not be solved completely by them alone.
"Making sense" of a situation describes what a person (or team) goes through when trying to construct the best, most reasonable interpretation of a large, complex data set. Previous work at PARC has identified a number of required aspects of this process.
This investigation will provide a key driving application and use of the Portolano systems when in real day-to-day use. Information can be overwhelming, contradictory or just unmanagable -- but with sensemaking methods that exist atop the base system, we hope to create an environment where real work can be accomplished very quickly and effectively.
[Abowd98] G. Abowd, C. Atkeson, Future Computing Environments: CyberDesk, Georgia Institute of Technology, 1998. (http://www.cc.gatech.edu/fce/cyberdesk/)
[Bala95] H. Balakrishnan, S. Seshan, E. Amir, R. Katz, Improving TCP/IP Performance over Wireless. Proceedings 1st ACM Conference on Mobile Computing and Networking, Berkeley, CA, November 1995.
[Bluetooth98] Bluetooth SIG, Bluetooth Technology, 1998. (http://www.bluetooth.com/technology/)
[Bond88] Bond, A., Gasser, L. (eds.), Readings in Distributed Artificial Intelligence, Morgan Kaufmann, San Mateo, CA, 1988.
[Choi98] S. Choi, K. Shin, Predictive and Adaptive Bandwidth Reservation for Hand-Offs in QoS-Sensitive Cellular Networks, SIGCOMM '98, 1998.
[Chou98] P. Chou, R. Ortega, K. Hines, K. Partridge, G. Borriello, ipChinook: An Integrated IP-based Design Framework for Distributed Embedded Systems, working paper, submitted to DAC-99, 1998.
[Coen] Coen, et.al., see http://www.ai.mit.edu/projects/hal/.
[Davies96] N. Davies, A. Friday, G. S. Blair, K. Cheverst, Distributed systems support for adaptive mobile applications, Journal of Mobile Networks and Applications, Vol 1, No 4, 1996.
[Edwards97] W. Edwards, et al., Designing and Implementing Asynchronous Collaborative Applications with Bayou, Proceedings of the 10th ACM Symposium on User Interface Software and Technology (UIST), Oct. 1997.
[Fishkin98] K. Fishkin, T. Moran, B. Harrison. Embodied User Interfaces: Towards Invisible User Interfaces. Proceedings of EHCI '98 (Heraklion, Crete, Sept. 13-18), in press, 1998.
[Fox96] A. Fox, S. Gribble, E. Brewer, E. Amir, Adapting to Network and Client Variability via On-Demand Dynamic Distillation, Proceedings Seventh International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-VII), Oct. 1996.
[Gernshen98] N. Gershenfeld, MIT Media Lab, Penny Tags Consortium Workshop, Boston, Mass., Personal communication, Aug. 1998. (http://www.media.mit.edu/physics/tags/tags.html)
[Glass83] Glass, R., Real-Time Software, Prentice-Hall Inc., Englewood Cliffs, NJ. 1983.
[Hartman96] J. Hartman, U. Manber, L. Peterson, T. Proebsting, Liquid Software: A New Paradigm for Networked Systems, University of Arizona, Department of Computer Science, 1996.
[Harrison98] B. Harrison, K. Fishkin, A. Gujar, C. Mochon, R. Want, Squeeze Me, Hold Me, Tilt Me! An Exploration of Manipulative User Interfaces, ACM SIGCHI '98, 1998.
[Hodes97] T. Hodes, R. Katz, E. Servan-Screiber, L. Rowe, Composable Ad-hoc Mobile Services for Universal Interaction, Mobicom '97, 1997.
[Hodes98] T. Hodes, R. Katz, Enabling "Smart Spaces": Entity Description and User Interface Generation for a Heterogeneous Component-based Distributed, DARPA/NIST Smart Spaces Workshop, 1998.
[Joseph97] A. Joseph, J. Tauber, F. Kaashoek, Mobile Computing with the Rover Toolkit, IEEE Transactions on Computers: Special Issue on Mobile Computing, Vol 46, No 3, 1997.
[Larner90] Larner, D., "A Distributed, Operating System Based, Blackboard Architecture for Real-Time Control", Proceedings of the International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, July 1990.
[Macker98] J. Macker, S. Corson, Mobile Ad hoc Networking (MANET): Routing Protocol Performance Issues and Evaluation Considerations, IETF Draft, Oct. 1998.
[MS98] Microsoft Corp., Microsoft COM Technologies, 1998. (http://www.microsoft.com/com/)
[Neaves98] P. Neaves, J. Bedford-Roberts, Dynamic Connection of Wearable Computers to Companion Devices Using Near-Field Radio, 2nd International Symposium on Wearable Computers, Oct. 1998.
[Norman98] D. Norman, The Invisible Computer, MIT Press, 1998.
[Nwana96] H. Nwana, Software Agents: an overview, Knowledge Engineering Review, Vol 1, No 3, 1996.
[OMG98] Object Management Group, CORBA News, 1998. (http://www.omg.org/corba/)
[Paxson97] V. Paxson, End-to-End Internet Packet Dynamics, ACM SIGCOMM '97, Sep. 1997.
[Perkins96] C. Perkins, H. Harjono, Resource discovery protocol for mobile computing, Journal of Mobile Networks and Applications, Vol 1, No 4, 1996.
[Perkins98] C. Perkins, Mobile Ad Hoc Networking Terminology, IETF Draft, Nov. 1998.
[RFC2401] S. Kent, R. Atkinson, Security Architecture for the Internet Protocol, RFC 2401, 1998.
[RSA98] RSA Data Security, Inc., Frequently Asked Questions About Today's Cryptography, 1998.
[Tenn96] D. Tennenhouse, D. Wetherall, Towards an Active Network Architecture, Multimedia Computing and Networking 96, Jan. 1996.
[Tenn97] D. Tennenhouse, et al., A Survey of Active Network Research, IEEE Communications Magazine, Jan. 1997.
[Waldo98] J. Waldo, Jini Architecture Overview, Sun Microsystems, Inc., 1998.
[Walker98] Walker, W, "Rapid Prototyping of Awareness Services using a Shared Information Server," SigCHI, Vol. 30, #2, April, 1998.
[Want95] R.Want, B.Schilit, N. Adams, R. Gold, D. Goldberg, K. Petersen, J.Ellis, M. Weiser, An Overview of the Parctab Ubiquitous Computing Experiment, IEEE Personal Communications, Vol 2, No 6, Dec. 1995.
[Want99] R. Want, K. Fishkin, B. Harrison, A. Gujar, Bridging Real and Virtual Worlds with Electronic Tags, accepted at CHI 99 and PARC technical report, 1998.
[Weiser96] M. Weiser, Open House, Review, Vol 2, 1996.
[Wether98] D. Wetherall, J. Guttag, D. Tennenhouse, ANTS: A Toolkit for Building and Dynamically Deploying Network Protocols, OPENARCH'98, 1998.
[Zhang93] L. Zhang, S. Deering, D. Estrin, S. Shenker, D. Zappala, RSVP: A New Resource ReSerVation Protocol, IEEE-Network, Vol 7, No 5, 1993.
[Zimmer96] T. Zimmerman, Personal Area Networks (PAN): Near-Field Intra-Body Communication, IBM Systems Journal, Vol 35, 1996.