Designing Systems Where Trust Is the Output
A rice cooker that plays a tune, the Japanese idea of anshin, and what it takes to design agents so that trust becomes their output.
Right after my first trip to Japan I did what is expected of a white man in his forties who loves food and cooking. I bought a rice cooker. I skipped the sensible upgrade from my stovetop pot and bought a Zojirushi, the Rolls-Royce of rice cookers, for reasons I will not defend here.
It plays “Twinkle, Twinkle, Little Star” when it starts and “Amaryllis” when the rice is done. Until 1999 Zojirushi’s cookers beeped like everything else. Then, as its blog puts it: “We realized that so many appliances existed together in our customers’ kitchens that it was difficult to know which appliance was beeping.” The tune tells you from the next room which machine wants you, and that it has finished.
Japanese has a word for what that produces: anshin, literally an “eased heart”. Usually translated as relief, it is to me a state of complete psychological safety, trust and freedom from worry. The rail operator JR East promises to “deliver security to everyone”, its English for anshin, and calls “the expansion of trust through security” the foundation of its business. Anshin differs from anzen, objective safety: a system can be perfectly safe and still leave you anxious. Anshin is what trust feels like when a system has earned it.
I have several agents running at once, and soon many more people will. Most of the conversation asks people to trust them. My friends at Stripe Partners recently argued, drawing on the sociologist Hartmut Rosa, that the agents people choose “are valued for their restraint”. I think restraint is one part of a larger design task: building these systems so that trust is their output. That trust comes from assurance, from a system that is reliable, legible about what it is doing and clear about its limits. Restraint is a limit you can rely on. The system also needs what Mark Weiser, who coined ubiquitous computing at Xerox PARC, called “beautiful seams”. For agents, I read those as places where you can look at how the work was done when something unexpected happens, and understand it more deeply.
Feeling at ease can also make you do less, though. In this year’s Danish security survey by the foundation TrygFonden, people with great trust in the authorities’ ability to handle a crisis were, to a degree, less likely to prepare for one themselves. I read that as assurance standing in for their own preparation. Delegation without a strategy behind it goes the same way: you hand over the decision along with the work, and delegating becomes abdicating.
Restraint has to be decided by someone. Delegating got much easier for me once I was involved in designing how my agents work, including what they leave alone. That is where my intent takes shape.
My rice cooker splits the work for me. With a great rice cooker comes great responsibility, which is washing the rice until the water runs clear. No cooker, however expensive, makes good rice if you don’t know how to wash it.