Overview of Major Stages in Pipeline

Robot Pressing Buttons
Mapping Auditory Events to Vision
 Visual Density of Auditory Events
 Detecting Button Affordances

Pressing Buttons Improves Detecting Buttons

Familiar ButtonNovel Button Generalization From a Single Training Button

Pushing Behaviors

Picture that shows the region in which the robot performed exploratory behaviors. The region has a shape of a pyramid, the doorbell buttons lies at the base of this pyramid and the robot starts a behavior from the top of the pyramid.
Region of ExplorationVideo

Videos Showing Press Locations for the Three Exploration Strategies

Random StrategyStimulus-driven Uncertainty-driven
Close-Up Proprioceptive event density for the random strategy (close-up). Proprioceptive event density for the stimulus-driven strategy (close-up). Proprioceptive event density for the uncertainty-driven strategy (close-up).
Video

Videos Showing How Auditory Events and Their Density for Each Button Evolve over Time

Button Close-up picture of one of the doorbell buttons in the experiment. This button is a white small round button, the casing is metallic and the functional component of the button is made from plastic. Close-up picture of one of the doorbell buttons in the experiment. This button looks like a piano key, the casing is metallic and the functional component of the button is made of plastic. Close-up picture of one of the doorbell buttons in the experiment. This button has a small white round pressable part, which is surrounded by a small metal ring. The remaining part of the button's casing is made of plastic and has black color. Close-up picture of one of the doorbell buttons in the experiment. This button has a small black pressable part, which is located on a circular brass casing in a sort of 'ridge', which makes it hard for the large robot finger to press. Close-up picture of one of the doorbell buttons in the experiment. This button has a large brass rectangular pressable part. The whole top face of the button is pressable. Close-up picture of one of the doorbell buttons in the experiment. This button has a small white rectangular plastic functional component. The casing is brass. Close-up picture of one of the doorbell buttons in the experiment. This button has a small white rectangular plastic functional component surrounded by brass casing. Unlike the previous button, the functional component of this button is closer to square and has ridges on its surface.
Auditory
Events
Densities