Muhammad Bilal, PhD
Robotics and HRI Researcher
#Open-to-Work
Short Biography
Muhammad Bilal completed his PhD in Human-Robot Interaction at the University of Melbourne, Australia. His research is at the intersection of robotics and spatial computing, with a focus on improving how novice users teach robots novel tasks through learning from demonstration. This includes developing augmented reality feedback systems that guide users in providing high quality demonstrations, enabling robots to learn effectively in real-world settings. His work has been published in leading international robotics venues, including IROS 2024 and HRI 2026. Prior to his doctoral studies, he served as a Research Assistant and subsequently as Team Lead at the Human-Centered Robotics Lab, National Centre of Robotics and Automation, Pakistan.
My Journey in Robotics (2017 - Present)
News
2026 — Completed PhD at the University of Melbourne, Australia, under supervision A/Prof Wafa Johal. (PhD Thesis Link)
2026 — Paper accepted at ACM Designing Interactive Systems (DIS) 2026. (Received Honorable Mention Award – Paper Link)
2026 — Paper accepted at HRI 2026. (Paper Link)
2025 — Gave an invited talk at Monash University. Thanks to Dr. Michael for the invitation.
2024 — Won the Robot Competition at HRI 2024.
2024 — Paper accepted at IROS 2024. (Paper Link)
2023 — Started PhD at School of Computing and Information Systems, the University of Melbourne, Australia.
2022 — Paper accepted at Science Progress – Sage Journals. (Paper Link)
Research Projects
Real-Time Feedback System for Kinesthetic Robot Teaching
Teaching a collaborative robot is often more difficult than simply showing it what to do. In practice, novice users often provide demonstrations that are incomplete, inconsistent, or poorly suited to the robot’s learning process. A key reason is that human teachers often struggle to understand what the robot has actually learned at any given moment. Unlike human students, robots do not naturally signal confusion, uncertainty, or partial understanding. To address this gap, robot learning must be treated as a dialogue in which the robot actively communicates its current state to the teacher through clear and accessible feedback, such as visual, haptic, or auditory signals. This research focuses on new communication interfaces that help robot “students” express what they understand, what they do not understand, and when they need help, making teaching a shared and responsive process rather than a guessing game. In this work, we address the overarching question:
How can robots communicate internal constraints to users in human-robot kinesthetic interactions?
User Experiment
We first conducted a focus group study (N = 9) to identify effective ways of visualizing key robot information, including joint limits, self-collisions, and manipulability. Guided by these insights, we designed an AR-based real-time feedback system and evaluated it in a between-subjects user study (N = 36) on a 7-DoF collaborative robot. Participants performed two tasks, including insertion and drink pouring, with the second task enabling assessment of participants’ learning across tasks.
We designed the following three conditions to test the proposed feedback system:
Offline Feedback — Participants demonstrated tasks without real-time guidance. After the first three demonstrations, they observed the robot’s reproduced execution and used this offline feedback to adjust subsequent demonstrations.
Real-Time Feedback — Participants received continuous AR feedback during demonstrations, showing key task-relevant states to guide movements in real time. No execution playback was provided between rounds.
Combined Feedback — Participants received both real-time AR cues and offline execution playback, enabling immediate guidance during demonstrations and reflection on the robot’s reproduced motion afterward.
Key Findings
- Novice users who received real-time AR feedback not only performed better on the initial task but also retained these benefits when the feedback was removed in the second task, demonstrating measurable learning.
- Feedback supported users in developing a better understanding of the robot’s motion constraints, leading to improved robot task performance even without visual assistance.
Resources
Citation
Muhammad Bilal, Tharaka Ratnayake, D. Antony Chacon, Nir Lipovetzky, Denny Oetomo, and Wafa Johal. Design and Evaluation of AR-Based Real-Time Feedback System for Kinesthetic Robot Teaching. In Proceedings of the ACM Designing Interactive Systems Conference (DIS), pp. 332–356, 2026. DOI: 10.1145/3800645.3812951
This paper received Honourable Mention Award.
Robot Kinematic Redundancy in Learning from Demonstration
Kinematic redundancy, where a robot has more joint degrees of freedom than required for a given task, provides greater flexibility during kinesthetic demonstrations. However, this flexibility also introduces additional complexity, as novice users must navigate a larger set of feasible robot configurations. This project investigates how the robot's degree of redundancy (DoR) affects human performance during kinesthetic demonstrations and, in turn, how the resulting demonstrations influence robot learning and task execution.
To study the impact of robot redundancy, we artificially reduced the robot's DoR by locking one of its joints. We then conducted a within-subject user study with 24 participants, comparing two conditions: high DoR (unconstrained) and low DoR (constrained). Our experimental setup also incorporated a motion capture system to record participants' hand interactions with individual robot joints, enabling detailed joint-level analysis of physical interaction during teaching.
Participants completed two manipulation tasks: button pressing and cuboid block insertion.
User Experiment Videos
This participant first performed button pressing task under constrained condition followed by unconstrained condition.
This participant first performed insertion task under unconstrained condition followed by constrained condition.
Key Findings
- Constraining the robot's redundancy significantly increased participants' mental workload.
- Demonstrations took longer to complete under the low-DoR condition.
- Reduced redundancy resulted in more failed demonstration attempts.
- Participants physically interacted with the robot's joints more frequently when redundancy was reduced.
- Demonstrations collected under the low-DoR condition led to poorer subsequent robot learning and task execution.
To support reproducibility and future research, we also released InteractLfD, a comprehensive dataset of human kinesthetic demonstrations collected during this study.
Resources
Citation
Muhammad Bilal, D. Antony Chacon, Nir Lipovetzky, Denny Oetomo, and Wafa Johal. Investigating the Impact of Robot Degree of Redundancy on Learning from Demonstration. In Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 825–833, 2026. DOI:10.1145/3757279.3785606
Quantifying Demonstration Quality in Learning from Demonstration
Learning from Demonstration empowers novice users to teach robots daily life tasks without writing sophisticated code, thereby promoting the democratization of robotics. However, novice users often provide sub-optimal demonstrations, which can potentially impact the robot’s ability to efficiently learn and execute the tasks. Prior research has assessed the quality of demonstrations by evaluating the robot’s task performance; however, the approach remains insufficient to qualify individual demonstrations, leaving the reason for classifying demonstrations as high- or low-quality unknown. Therefore, this simulation-based study aims to quantify the quality of individual demonstration at each step by incorporating motion-related quality features such as manipulability and joint-space jerk.
Our results illustrate a strong correlation between ranked demonstrations and the quality of task execution. Interestingly, we observed that the quality features have a significant impact on task performance, particularly when the provided demonstrations exhibit diversity in terms of quality. Overall, this analysis enables quantifying the quality of individual demonstrations based on motion-related quality features, thus improving learning from demonstration.
Key Findings
This study makes three primary contributions:
- Utilisation of motion-based features to quantify the quality of individual demonstrations before learning process.
- Transferability of quality features from provided demonstrations to the corresponding LfD model generated motions.
- Quality features can serve as an indicator of task performance when used in a diverse set of demonstrations.
Resources
Citation
Muhammad Bilal, Nir Lipovetzky, Denny Oetomo, and Wafa Johal. Beyond Success: Quantifying Demonstration Quality in Learning from Demonstration. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5120–5127, 2024. DOI:10.1109/IROS58592.2024.10802187