Organized by IEEE-RAS TC on Performance Evaluation & Benchmarking of Robotic and Automation Systems (TC PEBRAS)
Summarizing, the topics to be addressed in the program during the summer school are the following:
• Design of Experiments in Robotics
• Execution of Experiments in Robotics
• Experimental scenarios to evaluate performance, demonstrate
generality, and measure robustness
• Well-grounded experimental methods
• Replication of Experiments in Robotics
• Relationship between benchmarking and replication of experiments
with robots
• Reporting experiments in Robotics
• The R-Article process on the IEEE Robotics and Automation
Magazine
• Examples of Good Experimental Practice
• Evaluation of Experimental Robotics Work
• Evaluation and Benchmarking in HRI
• Comparison of experimental methodology in neurosciences and in
Robotics
• Comparison of experimental methodology in Biology and in
Robotics
• Benchmark standardization
• Benchmarking autonomy, cognition and intelligence
• Scalable autonomy measurements
• Metrics for sensory motor coordination and visual servoing
effectiveness and efficiency
• Performance metrics based on Shannon entropy related measures
• Performance metrics based on dynamical systems methods
• Performance modeling of the relationship between a task and the
environment where it is performed
• Performance Metrics for Response Robotics
• Success metrics in bio-inspired Robotics
• Design of Robotics competitions
• Design of Robotics challenges
• Integration of experimental methods, benchmarking, challenges
and competitions for a better evaluation of results
• Epistemological issues
