Most robots can’t pick up most objects most of the time. This is for a variety of reasons. The robot is turned off. The object is out of reach. The object won’t fit in the gripper. The robot doesn’t know where the object is, because it doesn’t have a sensor at all, or because the sensor isn’t aimed at the object, or because if it is aimed at the object, the object is transparent or reflective. Or if it is aimed at the object, and it is easy to see, it might not have a detector or pose estimator for that object that can localize and predict grasps accurately enough to pick things up.

Existing work by Robb Platt, Ken Goldberg, Sergey Levine, and Chelsea Finn is working towards more robust grasping. But doing things on real robots in the real world is hard, and this won’t change any time soon. What is needed is an integrated perception/planning/motion stack with POMDPs and learned object models that can run on a robust robot with a capable, flexible general-purpose manipulator.