How we tested

What cook doesn’t dream of a kitchen that can clean itself? Until engineers make the self-cleaning house a reality, we’ll have to settle for robots that at least clean the floor. These automated vacuums, which resemble oversized hockey pucks, can clean on demand with the push of a button, and many can even be scheduled to clean automatically when you’re out of the house. Once their cleaning cycle is initiated, they scoot around the room vacuuming up dirt and dust, and most automatically return to their charging base to juice up when they’re done cleaning (or when their power is running low). 

This technology doesn’t come cheap—we’ve seen some with price tags of close to a thousand dollars—but we wondered if we could find a robot vacuum under $500 that was worth buying. We rounded up seven models and set them to work picking up a sloppy mix of flour, salt, wet coffee grounds, mirepoix (a mixture of chopped onions, celery, and carrots), and garlic peels that we spread over a hard floor. We also tasked each robot with cleaning up multicolored sprinkles spilled in an 80-square-foot carpeted office. We quantified how well the robots cleaned by carefully weighing all the debris before scattering it across the floor; after each cleaning, we weighed the contents of the robot’s collection bin and calculated the percentage of captured matter. Finally, we sent the robots home with editors who put them through their paces, navigating pets, stairs, and other obstacles as they cleaned different types of flooring and carpeting.

First, the bad news: None of the robots worked flawlessly, and we don’t recommend them as a replacement for your regular vacuum. They don’t do well with big jobs—through pretesting the vacuums, we determined that 1/2 cup of debris was the most realistic amount to test them with, as most couldn’t handle any more due to their relatively small intakes and collection bins. Every robot eventually got hung up on one obstacle or another, usually around cords or in tight spaces. This meant we had to take care to prep the room before cleaning, blocking off the area under couches or other low furniture where the robots could get wedged and picking up any loose cords or large pieces of debris. Most robots also missed corners or sections along walls—some even pushed dirt into these hard-to-reach spots—making it necessary to still pull out a regular vacuum for occasional deep cleaning. That said, our testers loved the best robot vacuums as maintenance gadgets for day-to-day cleaning and thought they were a great way to deal with accumulated dust, pet hair, and small messes. When editors took them home, we found that most robots were able to clean the bulk of a space pretty thoroughly once we got the hang of robot-proofing the room. Some, however, still missed large areas of the room no matter how much we babysat them.

To figure out why some vacuums performed better than others, we began by turning them over and examining their intake ports (which ranged from about 8 to 24 square inches). We found little correlation between performance and the size of the intake port or the size or placement of the small sweeper brushes. Two other features proved far more important: suction power and ability to navigate the room. While we asked each manufacturer about suction power, none would disclose specifics. One engineer told us that, unlike specs for regular vacuums, suction power isn’t usually advertised for robot vacuums because the dirt only has to travel a few inches off the floor into the robot’s dirt-collection chamber, so only minimal suction is required. The top-performing robot picked up an average of 86 percent of the debris we scattered, while the worst performer picked up just 51 percent. 

A few robots left behind a dusting of food, which clearly showed insufficient suction power. But we were curious as to why some robots were missing big parts of the areas they were supposed to clean.

To get a better idea of how the robots moved while cleaning, we put the robots in a dark room dusted lightly with flour and took a series of long-exposure photographs. By tracking the lights on the top of each robot, the camera captured the path of each robot throughout its entire cleaning cycle. We discovered that the robots fell into three distinct navigation styles: random lines, concentric circles, and methodical grids. 

Robots that moved in random lines, including the iRobot Roomba 770, cleaned unevenly, often focusing heavily on one area and then breezing over the rest of the room.

Robots that moved in concentric circles, like the Infinuvo Hovo 510 and the iTouchless Robotic Intelligent Automatic Vacuum Cleaner PRO, were more effective at covering most of the room after running for their complete 30-minute cleaning cycle. While not a deal breaker, these lengthy cleaning cycles drained these robots’ batteries, which means that vacuuming a whole house would take multiple recharging stops. These spiraling robots also had trouble navigating out of tight corners, and testers complained that they sometimes scuffed up walls and furniture with their constant bouncing around. 

Robots that map the room and navigate the space in a methodical back-and-forth grid make up the third category. The robot first travels the perimeter of the room and then covers the middle area in a grid pattern, quickly navigating every inch of an entire space. These robots were our runaway favorites, taking only about 10 minutes to clean an 80-square-foot room. Testers also thought these robots banged around a lot less, avoiding obstacles and slowing down as they approached walls.

It turns out that the two robots that follow a grid pattern, the Neato Botvac D80 and the Moneual Rydis H68, use advanced mapping technologies to “see” the room and plot their courses. All robot vacuums have a series of sensors to help with cleaning: a dirt sensor, a front bumper sensor that detects when the robot has hit an obstacle, a sensor that detects drops and stairs, and an infrared light sensor to detect walls. Together, these sensors guide the robot away from major obstacles, but otherwise most robots’ movements are random.

The idea is that, with enough time, the robot will eventually cover the room. Our top-performing robots, however, use an additional component: either an onboard camera or a Light Detection and Ranging (LIDAR) distance sensor, which can detect the shape of obstacles and distance between objects. Using this information, the robots build two-dimensional maps of the room and plan their routes. The result is robots that clean faster and more efficiently and require little babysitting.

Testers also found these “smarter” mapping robots easier to use. We preferred robots with a few clearly labeled buttons, minimal setup, and an intuitive, reliable scheduling function (which allowed us to set the robot to automatically clean while we were away). Remotes, virtual gates (barriers that you set down that emit a beam of infrared light to block off no-go zones, such as staircases), or special cleaning modes such as spot cleaning were nice to have but not necessary and often went overlooked by home testers. While four robots came with a wet mopping pad for cleaning hard surfaces, only one of them, the Moneual, worked well, and most testers thought the other robots’ lackluster mopping results weren’t worth the fussy setup. Until the technology catches up, we’ll just stick with the vacuuming (since not all robots have mopping functions, we did not factor mopping into our overall scoring).

All the robots also required a little bit of upkeep: We had to empty the small collection bins after every couple of cleanings and occasionally had to remove trapped hair and debris from the vacuums’ brushes and vents. Our favorite vacuums had roomy collection bins and came apart easily for more thorough maintenance.  

Our favorite vacuum was the Neato Botvac D80, a D-shaped robot that uses a LIDAR sensor to map the house as it cleans. Time after time, the Neato cleaned quickly and efficiently, barely bumping into walls or obstacles. At home, it effortlessly moved from room to room without babysitting, and testers thought its two-button design was intuitive and easy to use. Since crowning it our winner, we’ve put the Neato to work cleaning every day as we perform long-term durability testing. We’ll report back on its progress in 6 months. 

Methodology

We tested seven robot vacuums, priced from about $120.00 to $500.00. Prices shown were paid online. Models appear in order of preference.

SETUP

We rated each model on how easy it was to get up and running out of the box. Models lost points if instructions were unclear or parts were difficult to assemble.

AVERAGE PERCENTAGE OF DEBRIS PICKED UP

We averaged two cleaning tests: vacuuming a mix of wet and dry foods from a hard floor and vacuuming colored sprinkles from an 80-square-foot carpeted room. For both tests, we weighed the debris before scattering it. We then weighed how much the robots collected and calculated the percentage of captured matter.

CLEANING

We evaluated how well the vacuums could pull debris off the floor by weighing the contents of each robot’s collection bin after it either declared itself done or completed an automatic 30-minute cleaning cycle (three out of four models operated the latter way). Top-performing robots retrieved more than 80 percent of the food we dropped and left little to no dusting of debris on the floor after completing their cleaning cycle.

NAVIGATION

We observed how much of a space the robot was able to cover through long-exposure photographs, tester observation, and user feedback. Top-performing models were able to get into tight corners and hard-to-reach spaces near obstacles and walls. Points were deducted from robots that got trapped under or around objects, scuffed up walls and furniture with excessive bumping, or missed large pockets of the room. Our favorite robots have intelligent navigation systems that allow them to move through the space logically and easily.

EASE OF USE

We rated robots on how hands-off they were to operate. The best robots were almost completely autonomous, requiring minimal prep work from testers and reliably returning to their bases after each cleaning. Robots lost points for confusing interfaces, useless scheduling systems, or unreliable cleaning.

UPKEEP

This criterion evaluated how easy the robots were to empty and maintain. Models lost points for overflowing dirt bins or hard-to-remove filters and parts.