In Design We Trust

By Rusty Rockets

Do you feel that your freedom to explore the wide-open road is curtailed by your car's navigation system? Do you rant and rave at your bank's automated telephone "help" service? Sure you do - you become incensed, exasperated, and furious. You want blood. You see, humanity is currently involved in a war for the future with the latest wave of smart machines; a war that we humans seem to be losing quite badly. Sure, it's not the kind of glamorous, dystopian future war where you might hear the words "I'll be back..." uttered. But according to author Donald A. Norman, automated machines are driving us all crazy for the very same reasons the Terminator was considered the perfect killing machine. They can't be bargained with, and they can't be reasoned with. They don't feel pity or remorse or fear and they absolutely will not stop, ever, until you fall into a pathetic sobbing heap on the floor.

In his new book The Design Of Future Things, Norman, Professor of Computer Science at Northwestern University, argues that advanced technology is getting the better of us because design-wise, its too simple and too unsophisticated. Sound counter-intuitive? Well, yes it is, admits Norman. But if you've ever tried to negotiate with an automated answering service, or car navigation system you'll know that Norman is spot on. There is simply no communication between human and machine; the dialogue is all one-way. "Designers of advanced technology are proud of the 'communication capabilities' they have built into their systems," writes Norman. "But closer analysis shows this to be a misnomer: there is no communication, none of the back-and-forth discussion that characterizes true dialogue."

This isn't to say that navigation systems aren't useful, as they have in fact saved many lives, says Norman, a former Apple Fellow. But because these dictatorial machines cannot be reasoned with or, in some cases, even switched off - "many modern cars are so authoritarian that they do not even allow this choice," says Norman - many users simply don't use inbuilt car navigation systems, or the cars that have them.

As Norman points out, your choice of an automated system can often be a matter of life or death. Just imagine for a moment what would happen if you decided to turn off your anti-skid brakes or stability control. "Many drivers believe they can control the car better than automatic controls," says Norman. "But antiskid and stability systems actually perform far better than all but the most expert professional drivers. They have saved many lives."

But this brings us back to a question of trust in technology. The lack of trust in intelligent devices is a crucial reason why most automation is actually only semi-automation. But semi-automation, the interface between human and machine, has equally serious problems. "Whenever a task is only partially automated, it is essential that each party, human and machine, know what the other is doing and what is intended." Come the day when cars combine both navigation and semi-automated driving systems, a failure in effective human-machine communication may ultimately prove fatal.

But as a wide variety of intelligent products and services continue to infiltrate our lives, we put aside the fears we may harbor regarding semi-automated or fully automated systems. "Automatic banking systems already exist that determine whether you are eligible for a loan. Automated medical systems determine whether you should receive a particular treatment or medication," writes Norman. But why do we put so much stock in automated devices that have a hand in running our everyday affairs? Convenience? But convenience for who? Consumers or providers? Perhaps it's because we have such an unshakeable faith in the meticulousness of machines, and their infallible calculations. If it's the latter, then we really need to snap out of this mindset before we end up in Huxley's brave new world.

If we again consider the intelligent systems in your car, Norman reveals that their precise calculations cannot always be trusted to get you out of a tight spot. "The arithmetic may be correct, but before doing the computation, it must make assumptions about the road, the other traffic, and the capabilities of the driver," explains Norman. "Professional drivers will sometimes turn off automatic equipment because they know the automation will not allow them to deploy their skills." If this is true of cars, there seems to be no reason why it's not also true of intelligent banking and medical diagnostic systems. Be it anything from automated reading suggestions, to automated medical prognosis and prescription, "actions are apt to be taken arbitrarily, with the systems making gross assumptions about your intentions from a limited sample of your behavior," argues Norman. True, such systems may provide convenience, or even save your life, but the trick is to know when to invest your trust in automation. At present, it would seem that we just have to accept our technological guinea pig status.

The inherent problem of human-machine miscommunication, and lack of trust, lies in the ambiguous, complex twilight realms of context and common sense. Researchers at institutions like MIT are currently tearing their hair out trying to produce intelligent systems capable of communicating like a human would. And as last week's article showed, there is some progress being made in this area. But as a partner in the Nielsen Norman Group Consulting Firm, which consults with corporations on design, and if current examples of automated design are any indicator, Norman is a little concerned for the future of intelligent devices.

"Emotions play an essential role in human and animal behavior, helping us judge what is good or bad, safe or unsafe, while also providing a powerful communication system for conveying feelings and beliefs, reactions and intentions among people," says Norman. "Machine emotions are simplistic."

At one point in The Design Of Future Things, Norman cannot contain his amusement at innovations such as smart kitchens - packed with sensors, TV cameras, and pressure gauges - which are supposed to pre-empt your every culinary move. While the eating habits and kitchen appliance usage of an individual can be roughly monitored as a single action, the kitchen as a whole doesn't "get" the context of a situation right very often. "If people use the refrigerator and then walk to the microwave oven, they have a 'high probability of reheating food.' This is highfalutin scientific jargon for guessing. Oh, to be sure, it is a sophisticated guess, but a guess it is," says Norman.

Other, less trivial, concerns revolve around privacy issues. Norman predicts that systems capable of monitoring your personal eating, reading, television viewing, and music habits and preferences are on their way, or currently in production. A little further down the track will be systems that will monitor copyright violations, or "watch where you drive, alerting the insurance company, the rental car agency, or even the police if they decide that you have violated their rules," says Norman. The guts of such intelligent systems will likely be what are known as radio-frequency identification (RFID) tags - also used in "intelligent kitchens - which are already a hot topic among tin-foil hatters.

RFID tags are nifty little devices that need little maintenance, and can provide a raft of information about a person or device very quickly. "No batteries are required because these devices cleverly take their power from the very signal sent to them asking them to state their business, their identification number, and any other tidbits about the person or object they feel like sharing," explains Norman. At the user's discretion, owners of certain cars fitted with RFID tags can interact with billboards, in a way similar to those featured in the film Minority Report. "Billboards in multiple cities recognize owners of BMW's Mini Cooper automobile by the RFID tags they carry," writes Norman. "The Mini Cooper advertisements are harmless, and each driver has volunteered and selected the phrases that will be displayed." But only time will tell whether RFID tags will become a mandatory component of all cars. Couple this with television cameras on buildings and streets, facial recognition and gait recognition software, and we'll all become easy prey to commercial interests everywhere.

Whether it's deciding what automated devices to trust, whether we want our kitchens to think for us, or deciding whether a smart technology is actually a breach of our civil liberties, the average consumer is going to have a lot to think about in the near future. According to Norman, how well these technologies function will come down to designs capable of facilitating effective communications between human and machine. But as communications become more sophisticated, and we begin to enjoy the benefits of improved medical and automotive performance, it just might come at the cost of our privacy.

2 November 2007

In Design We Trust

By Rusty Rockets