Verification of Autonomous Systems

Scope

The robotics and autonomous systems communities have recently seen a significant and rapid increase in both the development of robots for commercial use and in interest in using robots in a wide range of novel applications. As these robotic systems, vehicles, and even embedded devices move towards much greater autonomy, we will require techniques for verification that provide much higher confidence than usual. Consequently, the analysis and test processes used for traditional systems must be significantly enhanced to provide increased confidence in this next wave of autonomous systems. The need for well understood and effective verification techniques will become vital as we move to commercial applications such as “driverless cars”, incorporate complex AI technologies, and utilize these systems in safety-critical scenarios.

Activities

IROS 2024 Workshop on Building and Evaluating Ethical Robotic Systems

Workshop: Building and Evaluating Ethical Robotic Systems

Best paper recipient: Tammy Zhong

Paper title: Towards Hierarchical Planning with Social Norms and Ethical Considerations

For more information on this workshop, please visit this link.

First place abstract award - What does Responsible Robotics mean?: Stretching roboticists’ horizons from an academic, government and philosophical perspective

Name of workshop: “What does Responsible Robotics mean?: Stretching roboticists’ horizons from an academic, government and philosophical perspective”

Name of award: First place abstract award

Name of recipient(s): Amar Halilovic

Title of position paper abstract (2 pages): “Industry May Realize Robotics, but Academia Holds It Accountable”

For more information on workshops, visit this link.

Registration award - What does Responsible Robotics mean?: Stretching roboticists’ horizons from an academic, government and philosophical perspective

Name of grant: Registration award for online attendance to workshops at ICRA 2024

Name of recipient(s): Emily Collins

For more information on workshops, visit this link.

Dinner at ICRA 2024

Dinner for TC members attending the workshops on Monday 13th, 2024.

Attendants: Pekka Makela, Sarah Lund-Christensen, Nico Hochgeschwender, Helen Beebee, Daniel Kilov, Dejanira Araiza Illan, Johann Makela, Andrei Ionescu

Location: Sawada Hanten Yokohama Landmark Tower, Japan

Attendants at the dinner, photo taken by Andrei Ionescu
Attendants at the dinner, photo taken by Andrei Ionescu

Ongoing Activities

Seminars

  • We are currently holding a monthly Verification of Autonomous Systems Seminar Series via Zoom on the first Thrusday of the month.  If you would like to receive invitations and information about this series, please join our TC using the “Join Us” link above.  It features presentations from TC members at both introductory and advanced levels.  If you would like to present, please contact the TC Co-Chairs using the information at the “Contact” link above.
    • Upcoming speakers are posted below in the “Upcoming Events” section
    • Slides from previous seminars will be posted below as they become available through the links in the “Past Events” section.
  • Number of seminars organized to date (2023): 24, with 43 invited talks
  • Number of attendands: ~20 to 80

Members Annual Meetings

  • We hold an annual review meeting with our TC members, to seek for feedback and call for participation in our new and exciting upcoming activities.
  • Past meetings: December 2020, September 2021, September 2022

Newsletters

  • We send out a periodic newsletter with updates to this site and any information or calls submitted by TC members or external parties
    • If you have something to submit, please email the Co-Chairs and it will be included in the next Newsletter
    • Content types:
      • Information about upcoming events – lectures, seminars, symposia, workshops
      • Calls for papers, workshop and tutorial proposals, participation, and grant proposals
      • Requests for information and/or support
      • News – new publications, meeting updates, where to find slides and presentation content for past events
  • Number of issues published to date: 12

Working groups

Outreach Activities

Magazine Articles

  • We published a TC Spotlight Article in IEEE RAM, March 2022, available here.

Support and Organization of Workshops attached to International Conferences

Reaching Under-represented Communities

  • Relevant travelling grants will be made available for upcoming workshops and events.
  • Current active effort for the recruitment of TC members and junior co-chairs from Africa and Asia.

UPCOMING EVENTS

Verification of Autonomous Systems Seminar Series

 

April 4th 2024, 9 am EDT

Houssam Abbas, TBC

Taylor Thomas Johnson, TBC

 

May 2nd 2024

 

June 6th 2024

 

Technical Committee Members Meetings

September 5th 2024

PAST EVENTS

Verification of Autonomous Systems Seminar Series

*** To download the available slides, please click on the respective seminar title ***

 

May 7, 2020

Signe Redfield, “Autonomous Behavior Specification

Dejanira Araiza-Illan, “On verification, safety and manufacturing in Industries 4.0

 

June 4, 2020

Michael Fisher, “Steps to Verifying Autonomous Systems

Mauricio Castillo-Effen, “Trustworthy Autonomous Systems

 

July 2, 2020

Youcheng Sun, “Validation and Verification of Learning-Enabled State Estimation System for Robotics

Chris Allsopp, “Developing a Taxonomy of Errors for the Integration of Autonomous Components

 

August 6, 2020

John S. Baras, “Trusted Autonomy Theory and Applications”

Jim Kapinski, “Application of lightweight verification methods to delivery devices” (slides not available)

 

October 1, 2020

Joanna Olszewska, “On V&V of Intelligent Vision Systems”

Charles Lesire-Cabaniols, “Formal methods for verification and control of software architectures”

 

November 5, 2020

Marco Roggero, “Design and verification of algorithms for object detection and tracking using lidar data”

Brian Logan, “Synthesising provably correct programs for autonomous systems”

 

January 7, 2021

Michael Winikoff, “Towards a Framework for Certification of Reliable Autonomous Systems”

Jyotirmoy Vinay Deshmukh, “Safe Deep Neural Network Controllers for Autonomous Systems” (slides not available)

 

February 4, 2021

Geoffrey Biggs, “Software libraries for increasing the reliability of C++ software”

David Tate, “Formal and Empirical Verification of Autonomy for Assurance”

 

March 4, 2021

Tom Freund, “Urban Air Mobility: Verifying trust”

Liren Yang, “Synthesis-guided Adversarial Example Generation for Gray-box Autonomous Systems with Sensing Imperfections”

 

April 1, 2021

Aurora Schmidt & Marin Kobilarov, “Verified Assured Learning for Unmanned Embedded Systems (VALUES)” (slides not available)

Ali Narenji Sheshkalani, “Verification of visibility-based properties on multiple moving robots in an environment with obstacles”

 

May 6, 2021

Sergiy Bogomolov, “Trusted Autonomous Systems: Verification Meets Falsification” (slides not available)

Angelo Ferrando, “ROSMonitoring: A Runtime Verification Framework for ROS” (slides not available)

 

June 3, 2021

Rosemary Monahan, “Formalising Verifiable Requirements in an Aerospace Use-Case”

Ufuk Topcu, “Autonomous Systems in the Intersection of Formal Methods, Controls and Learning” (slides not available)

 

October 7, 2021

Sadra Sadraddini, “Coordination of Large Heterogeneous Teams from Temporal Logic Specifications” (slides not available)

Kevin Leahy, “Efficient Motion Planning with Collision Avoidance Guarantees” (slides not available)

 

November 4, 2021

Bowen Weng, “On guaranteed safety assurance of automated driving systems (ADS): a set invariance perspective” (slides not available)

Max Cohen, “Safe Exploration in Learning-based Control of Autonomous Systems” (slides not available)

 

December 2, 2021

Mohammad Reza Mousavi, “Trust in Autonomous Systems through Verifiability”

Simos Gerasimou, “Evolutionary-Guided Controller Synthesis for Autonomous Systems” (slides not available)

 

February 3 2022

Ebru Aydin Gol, “Cause Mining and System Repair with STL”

Technical Committee members and co-chairs: Discussion on educational activities

 

March 3 2022

Cem Hatipoglu, TBC

Technical Committee members and co-chairs: Discussion on industrial focus working group

 

May 5 2022

Gabriele Ferri, “From SAUC-E to the European Robotics League: supporting robotics through competitions”

Andrea Censi, “Designing everything together”

 

June 2 2022

Manuela Veloso, ” Advancing AI and multi-robot systems through the RoboCup competition”

Liam Paull, “Duckietown and the AI Driving Olympics”

 

July 7 2022

Joseph Zeid, “Performance Testing to Support Grasping, Manipulation, and Robotic Assembly”

Craig Schlenoff, “The Agile Robotics for Industrial Automation Competition”

 

October 6 2022

Kerianne Hobbs, “An Assurance Case argument for Neural Network Control Systems bounded by Run Time Assurance”

Cristian-Ioan Vasile, “Robust and Relaxed Satisfaction of Temporal Logic Specifications”

 

February 2 2023

Discussion on Industrial Activities for 2023

 

March 2 2023

Hugo Araujo, “Causality for CPS”

Wendy Otieno, “Stealth Attacks in Deep Learning Systems”

 

April 6 2023

Matt Windsor, “RobotCert: Property Specification in Robotics”

TC updates

 

July 6 2023

Alan Winfield, “An Ethical Black Box for Social Robotics: a draft Open Standard”

Dejanira Araiza Illan, “Assessing quality and reliability of robtic software for industrial applications: challenges and solutions” (slides not available)

 

November 2 2023

Signe Redfield, “IEEE P2817 – “Guide for Verification of Autonomous Systems” Standard Development Working Group”

 

December 7 2023

Ho Chit Siu, “STL: Surprisingly Tricky Logic (for System Validation)”

 

March 7 2024

Ichiro Hasuo, “Proving Safety of Automated Driving Vehicles: Formalization of RSS with Program Logic

Christof Budnik, “Ensuring Human Safety in Autonomous Manufacturing: A Staged Approach to Risk-Driven Verification”

 

Technical Committee Annual Members Meetings

 

December 3, 2020

Annual Technical Committee Members Meeting (2 hours)

 

September 9, 2021

Annual Technical Committee Members Meeting (2 hours)

 

September 1, 2022

Annual Technical Committee Members Meeting (2 hours)

 

September 7, 2023

Annual Technical Committee Members Meeting (2 hours)

Membership Data

  • Number of members registered by December 2020: 200
  • Number of members registered by September 2021: 450
  • Number of members registered by September 2022: 544