Aude Billard

During her master's degree at the University of Edinburgh, Billard focused on learning how to build Knowledge-Based Systems, essentially computer programs that are built on fundamental frames, concepts, and logical assertions from the real world but with the ability to perform inferences and derive new knowledge through reasoning systems.[1] Billard’s Master’s thesis, titled “Allo Kazam, do You Follow Me? or Learning to Speak Through Imitation for Social Robots”, focused on the goal of building a system that could develop communication abilities.[5] She developed a system that was capable of learning simple syntactical language and she used two mobile and autonomous robots, acting as teacher and student, to implement the architecture.[5] With an emphasis on simplicity and generalizability, Billard was able to ensure that neither specificities about the environment nor the particular robotic agent were necessary for success of the architecture in practice.[5]

During her graduate work, Billard continued to use the teacher-learner situation as an environment with which to develop robots with social communication abilities.[6] Billard tested her novel learning architecture and found that learning through imitation can be achieved with simple computations and a photodetection system between the teacher and the learner to impart information about movement.[6] However, Billard mentioned in her thesis there are still limitations to learning capabilities when the only means of learning is by imitation.[6] Billard suggested that social learning could be enhanced with more complex cognitive mechanisms that enable a robot to associate one outcome with a subsequent event instead of with simultaneous sensory.[6]

In Billard’s graduate thesis, she described her work implementing imitation learning in autonomous robots through applying her innovative connectionist model designed for robot learning called DRAMA (Dynamical – Recurrent – Associative – Memory – Architecture).[7] To develop DRAMA, Billard used an anti-objectivist framework, distinguishing between cognitive and behavioral skills.[7] This framework ensures that the behavior of one robot takes into account its previous interactions, since behaviors are performed in relation to the social environment with which the robot exists. The framework also takes into account the internal state of the robot, essentially the computations representing the robots own actions.[7] She uses this model in various scenarios in which a human or a robot is the instructor teaching a student robot to build symbolic representations of the world, learn a synthetic proto-language, or perform specific imitative movements.[7]

During her brief postdoctoral work back at EPFL, Billard proposes a theoretical framework with which to understand multi-robot communication systems.[7] She probabilistically modelled a multi-robot system and found that her model was able to accurately predict the ability of the system to process dynamic environmental information and apply it to communication and task learning among other robots.[7]

In 2001, Billard proposed a biologically plausible model of human imitation and discussed its applicability in robot teaching.[10] The model was able to learn the principal features of an arm trajectory in a throwing/catching imitation task, was able to generalize across different demonstrations, was able to learn on-line, and its movements were robust to perturbations.[10] Billard continued to explore more biologically inspired connectionist architectures with which to train robots to learn complex arm movements by imitation.[11] She based her artificial neural networks on brain regions such as the visual and motor cortices and incorporated a decision making occurs region as well. This allowed Billard’s model to imitate a teacher just as well as a human subject would imitate in the same task.[11] Billard continues to base her computational approaches on biological systems and began to explore how implementing neuromodulatory mechanisms in neural networks produce tune-able pattern generation as they would in the human brain.[12] Further, Billard and her colleagues began to implement neurobiological concepts such as homeostatic plasticity, Hebbian reinforcement learning, and hormone feedback into their neural networks to again provide adaptability and flexibility like that exists in the human brain.[13]

In 2006, Billard began adding social cues to interactions between humanoid robots to improve the ability of humanoids to switch between learning and reproduction phases in an imitation framework.[14] She found that implementing a gesture recognition system and using motion sensors, as well as a Hidden Markov Model to extract the essential components of the social cues, produced more life-like behaviors in a social learning task.[14]

Using biologically inspired robotic models, Billard has been able to improve the ability of robots to learn precise and complex motor skills.[15] From 2008 on, Billard highlighted the power of dynamical systems in controlling fine robot movements and generalizing those movements within different contexts.[16][17] Billard and her team used golf putting as a task to explore the ability of the robot to learn complex motions and adapt to changes in position, speed, and target location.[17] When they allowed the robot to learn from the failed attempts at putting the golf ball into the hole, just as a human would learn from failure, the robot began to perform better as it gained information regarding the precise speed patterns and orientations from both successful and failed attempts at sinking a put.[17][18] Shortly after Billard published these astonishing results of the motor learning capabilities of robots, she showed that she could also teach robots to catch objects in flight.[19][20] The video of their robot catching objects in flight has been viewed millions of times on YouTube and their publication in IEEE was the most frequently downloaded document in the journal.[20]

In 2015, Billard and her colleagues used electromyographic EMG recordings to decode grasping intentions at early stages of grasping, as these stages are important for early estimation of final hand position and generation of smooth gestures.[21] With 90% accuracy, they were able to decode three typical grasps, which provides a novel and effective approach to coordinating a subjects arm movements with a robotic hand to generate a natural pattern of motion.[21]

In 2016, Billard and her students won multiple awards for their paper 'Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncertainty'.[22] In this work, Billard and her students proposed a virtual object based dynamical systems control law that could generate autonomous and synchronized movements for a multi-arm robotic system. They were able to validate their approach using a dual-arm robotic system and they found that it was able to adapt and coordinate the motion of each arm to catch flying objects at high speeds and with uncertainty in trajectory.[23]

Billard and her team have also implemented hierarchical knowledge systems to allow robots to learn both high-level complex task plans as well as lower level movements after demonstrations.[24] Their work in 2016 showed that by combining a variant of a Hidden Markov Model with an algorithm that outputs transition probabilities, they were able to learn both low-level motor patterns during specific behaviors as well as the probability of a transition to the next behavior in the sequence.[24] Billard and her colleagues further improved robotic learning of task sequences publishing their methods in the Proceedings of Machine Learning Research in 2017.[25] They combined linear parameter varying systems, to enable the learning of task sequences, with hidden Markov Models, to learn complex control policies of each subgoal/subtask, and were able to validate their approach using two different human demonstrations.[25]

In 2018, Billard and her colleagues developed a method to teach robots to modify their tasks based on human physical interactions and interference.[26] By updating the parameters of their dynamical system to account for desired trajectory versus trajectory of the human interference, they were able to test their approach in real world experiments where robots successfully learned how to adjust their movements in relation to human interactions.[27]