Introducing a New Taxonomy of Behavioural Competencies to Demonstrate Driving Skills

Automated Driving Systems (ADS) have the potential to transform transportation, but their success depends on meeting strict safety regulations and standards. A key aspect of this is the Safety of the Intended Functionality (SOTIF), which is essential for identifying and mitigating risks that extend beyond traditional functional safety measures. SOTIF, as defined by ISO 21448, focuses on addressing hazards that can arise from a system’s design or performance limitations, even when no defects are present. This includes scenarios where the system operates as intended (no E&E malfunctions) but is not able to handle unexpected situations due to limitations in its design. Ensuring the safety of AI-driven systems requires not just adhering to SOTIF,  but also incorporating emerging standards and guidelines, including ISO 8800, UL 4600, and applicable guidance and best practice on the safety performance of machine learning (ML) systems.

The Role of Behavioural Competencies in Enhancing ADS Safety

To support SOTIF analysis, the concept of Behavioural Competencies (BCs) emerges as a foundational element. BCs are the driving skills that an ADS must demonstrate to ensure safe operation. These skills encompass both explicit rules, such as stopping at a red light, and implicit rules, like giving space to allow another road user to merge into traffic. BCs are essential for proving that an ADS can apply these rules effectively across various driving scenarios, strengthening the Design and Verification and Validation (V&V) process, and ensuring comprehensive safety across diverse driving conditions.

Addressing Inconsistencies in the Industry’s Use of BCs

The EU ADS Regulation 2022:1426 highlights the need for standardised taxonomies for Operational Design Domains (ODDs) and BCs to support a robust verification and validation framework. While ISO 34503, based on BSI PAS 1883, proposes an ODD taxonomy, there is no standardised approach for defining BCs, which weakens their effectiveness in ensuring the safe operation of ADS in a consistent and recognised way. The lack of standardisation is evident in how BCs are defined and applied. Some BCs consider ODD parameters, while others do not. For example, BCs like “navigate rail crossings” are specific to certain ODD attributes, whereas others like “detect and respond to encroaching vehicles” are more general and do not specify ODD attributes. Additionally, some BCs focus on specific manoeuvres, such as “U-Turn,” while others require broader skill sets, like “detect and respond to citizens directing traffic after an incident.” These inconsistencies lead to confusion and complicate the assessment of ADS performance.

Moreover, many existing BCs are not atomic, meaning they cover more than one particular driving skill. For instance, the BC of “car following through intersections, including stop and go, lead vehicle changing lanes, and responding to emergency braking” involves multiple skills, such as adapting speed to preceding traffic and lane following, making it difficult to clearly define and evaluate each competency. Recognising these issues, the Drive Safe AI project saw the need to redefine BCs to create a more coherent and effective framework. The effort is directly shaping the upcoming BSI/PAS 1891, a Taxonomy of Behaviour Competencies.

The Rationale for a New Framework

To address these challenges, we sought out to redefine BCs with a focus on dissociating them from ODD parameters as much as possible. The goal was to establish BCs based on the essential skills needed to navigate any road network topology safely, regardless of specific ODD attributes. This approach allows for a more universal application of BCs, making them relevant across diverse driving environments.

Our new framework introduces a more structured and granular approach to defining BCs. By limiting the number of BCs and creating both high-level and detailed levels of abstraction, the framework provides a clear and unambiguous definition of each BC. 

The building blocks of this framework are based on road network topology, focusing on segments (streets, roads, motorways) and nodes (roundabouts, intersections) that connect them. This ensures that BCs are closely aligned with the fundamental elements of any road network, allowing for more accurate and consistent V&V processes across various ODDs.

A challenge in ADS V&V is that system performance is often assessed based on ODD coverage. However, simply showing that an ADS can operate within a specific ODD doesn’t ensure it can handle all situations safely. The true test is whether the ADS can understand and apply both the written and unwritten rules of the road across various ODDs.

For an ADS to earn trust and wide acceptance, it must demonstrate comprehensive driving skills across different scenarios. BCs are essential in evaluating the ADS’s ability to learn and apply these rules effectively.

By separating BCs from the ODD, Wayve’s approach focuses on the ADS’s ability to understand and apply driving rules independently of specific conditions. This allows for a more robust evaluation of the ADS’s capabilities, ensuring it can handle unexpected variations in any scenario.

Developing a Taxonomy for Behavioural Competencies

Our goal was to create a framework capable of assessing an ADS’s ability to navigate safely in any driving situation, regardless of any specific ODD conditions. We established the following key requirements for our taxonomy:

1. ODD Agnostic: The taxonomy needed to be as independent of ODD variations as possible, allowing us to focus on the core understanding of driving concepts, such as traffic lights, without being constrained by the specific ODD in which these concepts might be encountered, such as bad weather.
2. Universal Applicability: It had to cover all possible road layouts, ensuring its use in any region or country.
3. Multiple Levels of Abstraction: The taxonomy should function at different levels of detail to serve various purposes, from high-level company reporting to detailed SOTIF analysis and mapping of testing coverage.
4. Atomic Behavioural Competencies: The taxonomy needed to feature atomic BCs, allowing us to break down complex manoeuvres, such as taking the third exit of a roundabout, into specific driving skills, like yielding at the entrance, changing lanes, and using turn signals within the roundabout.
5. Granularity for SOTIF Analysis: The taxonomy had to provide enough detail to support in-depth SOTIF analysis and performance measurement, enabling focused problem resolution and improvement of ADS safety.

The taxonomy we have developed meets all these requirements, offering a comprehensive framework for evaluating ADS driving skills independent of ODD constraints while supporting the rigorous safety analysis required for their successful deployment.

Defining Behavioural Competencies on Two Levels

To create a comprehensive framework, we have defined BCs on two distinct levels: Strategic and Tactical. This dual-level approach allows us to capture both the high-level goals of driving scenarios and the specific manoeuvres required to achieve them.

Strategic Level BCs

At the strategic level, BCs guide the ADS’s overall approach to driving scenarios based on the intended goal. For example, consider a roundabout. Depending on whether the goal is to turn left, go straight, or turn right, the ADS must position itself appropriately before entering the roundabout. These strategic-level BCs include:

• The ability to turn left at a roundabout (all exits before 12 o’clock).
• The ability to go straight at a roundabout.
• The ability to turn right at a roundabout (all exits after 12 o’clock).

Tactical Level BCs

Beyond strategic-level competencies, there are tactical rules governing specific manoeuvres within a scenario. For example, when navigating a roundabout, tactical-level BCs might include:

• The ability to position in the correct lane before entering the junction.
• The ability to yield to dynamic elements while driving through the roundabout.
• The ability to use appropriate indicators when navigating and leaving the roundabout.

The diagram below illustrates the Behavioural Competency Taxonomy. First, we classify BCs by the road topology element they relate to. Next, we differentiate BCs based on the driving goal for that element. Finally, we break down BCs by the tactical decisions required at each stage of navigating the road topology, such as approaching, passing through, and exiting.

Components of Tactical-level BCs

To structure the definition of tactical BCs, they can be formulated as a single instance or a combination of the following capabilities:

• Signalling: Communicating to other road users the ADS’ intent, such as activating turn signals or hazard lights. 
• Positioning: Ensuring the vehicle in the right position with regards to:
  • Other road users, e.g., maintaining appropriate longitudinal and lateral gaps
  • Road markings, e.g., stopping before a “Keep Clear” zone or a pedestrian crossing.

• Manoeuvring: Changing lanes or direction, e.g., turning left at an intersection or doing a U-turn.

For example, the tactical BC “The ability to change lanes to be in the correct lane for a given exit at a roundabout” can be structured as [lane change] <manoeuvring> + [turning on/off the indicators] <signalling>.

Application of BCs in V&V Activities

Our goal is to use both strategic and tactical-level BCs not only in the design phase but also to drive V&V activities. Demonstrating that an ADS can safely exhibit these BCs shows that it has learned both the written and unwritten rules of the road. Integrating these BCs into V&V ensures that the ADS can generalise its driving skills across various ODD conditions.

After validating the ADS within a limited set of ODD conditions, the next step is to test its ability to adapt (or generalise) to variations in different ODD attribute classes, such as scenery, environment, and dynamic elements. For example, the system might be tested in scenarios with changes in the number of lanes at a roundabout, varying weather conditions, or different traffic dynamics. This comprehensive approach allows us to thoroughly evaluate the ADS’s capability to handle diverse driving situations.

Considerations and Challenges

Defining BCs with sufficient granularity for detailed analysis is challenging, especially given regional variances in driving rules. Additionally, scaling this approach across a full ODD requires a systematic and thorough strategy. Establishing a common, agreed-upon taxonomy for BCs is crucial for standardising approaches and ensuring that regulators can rigorously test and verify the safety of ADS behaviour.

Conclusion

BCs are essential in demonstrating that an ADS has not only learned driving rules but can also apply them safely across various scenarios, regardless of ODD variations. Integrating BCs into the V&V framework ensures robust and reliable ADS performance.

We encourage you to explore how BCs can be applied in your own ADS development. Stay tuned for future blogs, where we will delve deeper into our V&V strategies for assessing the ADS’s ability to generalise across various ODD conditions.