AUTOMATION

How Voice Assistants Killed Automation And Saved The Smart Home

 Tyler Lizenby/CNET

When the Nest Learning Thermostat burst into the smart home market nearly a decade ago, it seemed poised to change how we saw smart home tech: CNET's editor-in-chief at the time decided to purchase the $250(!) thermostat within a week of trying it out. The sell? Nest would learn over time to set itself according to your comfort and budget. It was the promise of automation -- a home that truly takes care of itself (and you) -- realized.

But a few years later, another device called the Amazon Echo strolled onto the scene. At first the smart speaker couldn't do much, other than house the digital assistant Alexa. Even its developers didn't fully grasp Alexa's potential at the time of its release, Amazon's VP of Software for Devices and Services recently told me. Yet by 2020, voice control has become almost a prerequisite for smart home gadgets, and Alexa boasts integration with over 100,000 devices.

These two approaches to the smart home space are necessarily in conflict: automation aims to cut out your daily around-the-house tasks, while voice assistance changes the apparatus for accomplishing them, instantiates your active role in them. Alexa and Google Assistant are winning this battle, and for good reason: they offer a more inspiring vision of the future.

The promise (and problems) with automation

I like automation in theory: a house that does everything you'd expect, but cuts out many of the small touchpoints that take up your time. It sounds like a no-brainer. I hate running through rooms to switch off all the lights while my 2- and 3-year-olds whine at the front door because they want to get out of the house. I hate coming back to a cold home when I've turned down the thermostat for a weekend out of town. I hate bumbling around a dark house to get my sons their sippy cups in the middle of the night.

Sensors can help automate smart lights, but is that as useful inside as out?

Chris Monroe/CNET

While a handful of devices help mitigate these particular pain-points -- most directly, the Nest smoke detector's motion-sensing night light and the Thermostat's learning features -- true automation that goes beyond those problems seems far off. I've tested motion-sensing smart lights and Bluetooth-enabled smart plugs, but home-living is often too fluid for devices like these. More importantly, someone still needs to do the automating.

When we buy our phones and laptops, we expect them to already be programmed for us; yet if we want automated homes, we have to do much of the "programming" ourselves. Nest's Thermostat stood apart from the crowd by "learning" from how you used it, but the majority of automation on the market today requires you to take the time to set up your own routines: "When I open my garage, turn on the lights and open the blinds," or "At 11 p.M., turn off all lights and lock all doors."

These routines really are helpful, but setting them up and keeping track of them is a pain. Plus, exceptions to the rule always turn up, and getting locked out because you took the garbage to the street late one night isn't fun (yes, I've been locked out by smart locks before, along with half my office mates). And those annoyances don't even take into account the frustrating idiosyncrasies native to many of the platforms for setting such routines. (Works with Google Assistant, for instance, doesn't even let you delete routines.)

Automation is cooler when you have dozens of shades and lights all integrated. But maintaining that setup can also become a full-time job.

Tyler Lizenby/CNET

Most fundamentally, though, home automation faces a problem of philosophy. Automation looks to all the things we've done in our homes in the past -- opening shades, toggling lights, locking the door -- then asks how to make those things effortless. Alexa and Google Assistant have always been less concerned with what we already do than with what we could be doing. It's a more forward-looking approach, and that resonates more with people chasing the future.

A new, vocal contingent

At a private happy hour at CES 2020, while Guy Fieri prepped to come out, tell some groaners and cook with Alexa, I got the chance to chat with Amazon developers. There, VP of Smart Home Daniel Rausch told me they wanted Alexa to become a utility in the home, no different from electricity or water -- available in nearly every room, optimized for convenience and accessibility.

It sounds like a lofty goal, but Amazon has been pursuing it aggressively. Between its growing line of speakers and displays -- plus super-cheap, modular devices like the Echo Flex and Echo Input -- Alexa could feasibly find its way into every room of your house soon. And if it controls your kitchen appliances, your washer and dryer, your TV, your music and your alarms, then it could actually be useful in each of those rooms, too.

Google Nest seems to have a similar approach, particularly by investing in devices like the thermostat, security cameras, doorbells and smoke detectors -- now all integrated with their Works with Google Assistant program, and slightly less focused on the "learning" aspects of their technology than the voice-integrated aspects.

By prioritizing ubiquity above all else for the moment, Amazon and Google have drowned out older automation-centric devices. But in the same move, paradoxically, they've breathed new life into home automation as an idea.

Salvaging the promise of automation

Smart speakers can do more every day, from telling time to monitoring your house for break-ins. What began as a gimmicky gadget is now positioning itself at the center of the smart home.

Tyler Lizenby/CNET

 Alexa Guard was a simple software upgrade, but when it hit every nearly Echo device in 2018, it also provided a crucial insight into the future of the voice assistant in the home: the feature simultaneously listened for break-ins, automated connected lights to simulate an occupied house, and communicated with security systems like ADT. It wasn't a feature any great number of customers were begging for, in part, because few of us had imagined such a thing.

Automation has the potential to be banal (switching on lights when you enter a room) or to be visionary (attempting to diminish the chance of break-ins and improve their outcomes). The same imagination that produced a voice assistant, which felt like science fiction a decade ago, is starting to infect how companies think about automation.

The kitchen space offers plenty of examples: a new breed of smart kitchen appliances, for instance, is working to automate multiple steps of cooking -- comparing the contents of your fridge with online recipes to select ones you can cook, preheating the oven automatically once you start prepping, or even using cameras to tell you when your meat is fully cooked.

These more recent examples of automation give me hope: that as our homes become more intraconnected, and more connected to us, developers in all areas -- not just voice assistance -- keep their eyes forward, less concerned with optimizing how we live than redefining it.

‘We’re Stepping On The Gas:’ Miso Robotics Launches New Restaurant Automation Prototype

The Miso Robot on a Rail (ROAR) prototype features Flippy the robotic kitchen assistant on an ... [+] upside-down rail.

Miso Robotics

You may have heard of Miso Robotics, the company made famous about three years ago with its burger-flipping kitchen robot named Flippy. Flippy exists at multiple CaliBurger locations across the country, as well as Dodger Stadium and Arizona Diamondbacks’ Chase Field.   

The company today announced its latest prototype, Robot on a Rail (ROAR). The design takes Flippy and, well, flips it–hinging the robotic arms on an upside-down rail. The updated technology also features new software that has expanded Flippy’s food categories, from burgers and chicken tenders to over a dozen types of fried food like chicken wings and onion rings.   

“We made product changes based on customer feedback. First, it’s a no-no to take up square footage in the kitchen, so we wanted to make this easier to slide into existing work stations. Second, it was important for us to lower the cost of the product so we could add value,” said CEO Buck Jordan.  

The first goal has been achieved with a new design. ROAR is intended to be installed under a standard kitchen hood and out of the path of kitchen workers, taking up significantly less space than its predecessor, which was cart-based. 

The second goal is a work in progress. Jordan said Miso Robotics plans to have ROAR commercially available by the end of the year and has launched a new crowdfunding round on SeedInvest to bring that goal to fruition. The company has taken this route before, launching an equity crowdfunding campaign last year to raise $30 million. The amount reserved in the SeedInvest fund thus far is about $1.3 million.

The company’s last round was a $10 million Series B in February of 2018. To date, it’s raised about $15 million. 

Jordan’s ultimate vision is to cut the cost of this robotics technology in half (at least), making the case for a quicker return for spend-shy restaurant operators. The original product was about $60,000. The RAIL system is about $30,000.  

“It not only cuts the cost in half, the design also allows the robots to work two stations and double the output, which adds more value,” Jordan said. “By the end of the year, we’re evaluating even cheaper options, to $20,000. And, next year, we’d like to chop that down to $15,000. I think that’s achievable.” 

The company is able to trim those costs because of the smaller footprint (which also uses less sensors) and because it is using a more efficient robotic arm for this iteration, Jordan said.

He wants this technology to eventually become a “robot as a service” model, with instant return and “significantly increased EBITDA.”  

“We’re working hard to get the cost down because we essentially want to give this away,” he said.  

‘The market is calling for it’  

Jordan adds that the company is under pressure to produce this technology quickly.  

“The market is calling for it. Big restaurant companies are clamoring for this technology. Everything they’re dealing with is going in the wrong direction–food costs, labor costs, real estate costs–they’re all going up. And there just aren’t enough commercial chefs even though demand for them is exploding,” he said. “Labor is the number one focus right now in quick-service.”

Indeed, we’re in the middle of a historically low unemployment rate that has particularly affected low-wage jobs like restaurants. And, a new study from Chef's Pencil based on data from the Bureau of Labor Statics shows that employment for chefs and head cooks has dropped for two consecutive years and is down 4.3% from its peak in 2017. There were 128,600 chefs and head cooks in the U.S. In 2018. 

“Robotics solves all of these problems. Our customers are worried about filling shifts. This is a dire need now,” Jordan said.  

No doubt in a rising labor cost/tight labor market environment, the idea of a robot taking over tedious back-of-the-house tasks is certainly appealing. Still, proliferation of this technology was dealt a major blow earlier this month when Zume shut down its robotics operation. Not long ago, Zume was one of the most promising companies to thrust back-of-house automation into the mainstream. 

The California-based company began testing an automated oven-on-wheels concept all the way back in 2015, earning first-mover status. In an August 2018 interview, CEO and chairman Alex Garden said Zume’s automated system was capable of turning out 120 pizzas per hour. For context, Domino’s averages about 132 pizzas per store per day across its global system. 

SoftBank liked what it saw with these efficiencies, to the tune of a $375 million investment. However, Zume struggled to sustain its automation business and laid off 360 employees, or 50% of its workforce, at the start of this year.

A company valued near $4 billion making such a dramatic move could very well imply that the restaurant industry is just not quite ready for this type of automation. Technology. But Jordan is adamant that’s not the case, and Miso Robotics is hardly alone in chasing this solution.  

Consider the annual CES conference earlier this month in Las Vegas, where techies full of promise showed off the viability of their concepts. Restaurant-related automation had quite a presence.  

Samsung’s Bot Chef, for example, features robotic arms powered by sensors and cameras that help with sous chef tasks like chopping and seasoning. Also at the show, as the Dallas News reported, a pizza-making robot created by Seattle-based startup Picnic churned out 300 12-inch pizzas in one hour. Picnic’s robot can also make wraps and salads.

And, as was reported here in October, Ono Food Co. Is ready to expand its Ono Blends concept, a mobile restaurant powered entirely by robotic technology.  

It’s not just shiny, new startups dipping their toes into this technology, either. Pizza Hut is testing a delivery prototype with an automated kitchen, essentially in a truck bed. McDonald’s has also dabbled in some back-of-house automation with robotic fryers.

So, while Zume’s move made plenty of headlines, and may have even caused some investors to rethink the automated space, the technology still remains a viable solution in the restaurant space. In fact, Jordan estimates that the Miso Robotics workforce will grow by 35% this year to meet demand.

Much of that demand is coming from the steep rise in off-premise occasions, mandating higher throughput. The on-demand food delivery market is expected to grow at a rate of 11.4% throughout the next four years, for example.

“We’re seeing interest from the largest QSR and foodservice providers for this. Major brands are cold emailing us through our website,” Jordan said. “The opportunity is big and I think we’ll start to see much more competition in the next 18 to 24 months. Everyone is focused on this today, so we’re stepping on the gas.” 

Countering Automation Without Representation

Tomorrow’s government workers aren’t your neighbors. They’re algorithms. 

Federal and local governments increasingly are turning to data-trained models and artificial intelligence to help make decisions — big decisions, such as the length of a prison sentence, and smaller decisions, such as when to deploy health inspectors to restaurants. But these algorithms need oversight and it is nowhere to be found.

Congress should act on this critical issue, but so, too, should local governments, where some of the most consequential uses of government by algorithm can be found.

ADVERTISEMENT

In Pittsburgh, for example, the city and county governments are using algorithms to perform a variety of functions. Like many localities across the country, algorithms are helping judges determine who gets bail and who does not. They are guiding police about where and when to send patrols. They are helping child welfare hotline screeners determine when to send in a case worker. And they’re only becoming more common.

Each of these applications has a worthy goal: public safety, preventing crime, and saving children from neglect, abuse or worse. It is possible that some of these algorithms are “better” at doing so than the humans who might be making decisions in their place. Algorithms can be programmed to take more evidence into account; they can do more, faster. The promise of big data and predictive analytics for improving our lives and society should not be ignored.

However, it is also possible that these algorithms could perpetuate — or even accelerate — existing discrimination patterns. Ask Amazon, whose now-discarded interview system trained itself to avoid selecting resumes that indicated the applicants were women. If the data that helped train policing patrol models is using years of notoriously biased data — such as information about historical arrests — how have we ensured that the algorithm doesn’t churn out biased outcomes? 

Yet too many of these algorithms operate like black boxes, with little to no means for the public and researchers to scrutinize how decisions are made. As algorithms increasingly are used for determining government services and benefits, or even whether or not someone goes to prison, what remedies do individuals have if the algorithm is biased?

We cannot rely just on good intentions, or even impressive processes and evaluations within agencies. Instead, we should want public oversight to ensure accountability and fairness. No jurisdiction in the United States has successfully grappled with the complexities of how to adapt to this new administrative state by algorithm, though some are trying, including New York City and Washington State. 

ADVERTISEMENT

To be sure, this is no easy task. This is why the University of Pittsburgh Institute for Cyber Law, Policy and Security — Pitt Cyber — has formed the Pittsburgh Task Force on Public Algorithms to study oversight of local public algorithms. With support from The Heinz Endowments, we have gathered experts and community leaders to develop best practices and issue practical guidance for local policymakers wishing to take full advantage of the promise of data, while still ensuring accountability and equity for all residents. This task force will not work in isolation; we also will be working throughout the western Pennsylvania region to ensure that we learn from residents across communities. 

In a country where techno-optimism long has ruled, it is time we also ensure accountability and fairness. 

David Hickton is the founding director of The University of Pittsburgh Institute for Cyber Law, Policy, and Security and the chair of the Pittsburgh Task Force on Public Algorithms. He is a former U.S. Attorney for the Western District of Pennsylvania. Follow on Twitter @PittCyber.