Sabotage: A Simulation-Based Fault Injection Tool Framework

As automated driving vehicles become more sophisticated and pervasive, it is increasingly important to assure its safety even in the presence of faults. Fault Injection (FI) has been recognized as a potentially powerful technique for the safety assessment and corner-case validation of fault-tolerance mechanisms in automated driving systems. The major aim of performing FI is not to validate functionality, but rather to probe how robust the vehicle is−or their components are−to arbitrary faults under unforeseen circumstances.

To address these concerns, we developed and used a simulation-based FI framework so called “Sabotage”, coupled with the Dynacar vehicle simulator, which includes models of automated functions and high-fidelity multibody dynamics −i.e. engine, transmission, steering system, braking system, aerodynamics−.

Simulation-based fault injection is a technique that uses a series of high-level abstractions or models representing the system under study to evaluate its dependability during system design phases. Thus the system is simulated on the basis of simplified assumptions to (a) predict its behavior in the presence of faults, (b) to estimate the failure coverage and timing of fault tolerant mechanisms, or (c) to explore the effects of different workloads−i.e. different activation profiles. This kind of dependability means is called fault forecasting, and it is of primary interest for Sabotage.

Sabotage is a simulation-based fault injection tool framework based on the well-known FARM environment model. The FARM model is composed of: (1) the set of Faults to be injected, (2) the set of Activations exercised during the experiment, (3) the Readouts to define observers of system behavior, and (4) the Measures obtained to evaluate dependability properties.

The attached Figure shows the Sabotage building blocks and the flow of models to perform fault forecasting during vehicle simulation at early design phases. It can be seen that Sabotage extends the Dynacar vehicle simulator.

The Sabotage framework operates as follows. First, a Workload Generator generates (a) the functional inputs to be applied to the system under test (SUT), and (b) the set of faults−so called Fault List−to be injected in a faulty version of the SUT. Then a Fault Injector uses both, the fault list and a fault model library, to create the Faulty SUT. Once a clean version of the SUT (Golden SUT) and the Faulty SUT are available, the Dynacar environment is invoked to run simulations under the pre-configured vehicle scenario. Finally, a Monitor tracks the execution of the simulation experiments and aggregates the collected data to report the fault injection results.

Workload Generator: This block is in charge of two main activities: setting the experiments and driving scenarios, and generating the fault list. Due to our focus on early design phases, safety analyses provide the basis to specify the operational situations, which determines the vehicle and circuit driving scenarios (i.e. location, road conditions, environment conditions and the like). Dynacar manages a scenario catalogue that includes up to 150 configurable parameters so it can emulate a range of vehicles and prepare driving scenarios and automatic driving cycles. The scenario configuration block in Sabotage also addresses the inclusion of extra readout model blocks in the target system (SUT) to facilitate the logging process of output data.

Besides, the activity of fault list generation creates a subset of faults that can be injected in a reasonable time but still able to provide significant results. Our strategy to identify a representative fault subset is to use the target system malfunctions or failure modes e.g. omission or commission from safety analyses, instead of injecting an exhaustive or random fault set. The kinds of faults in the subset include permanent, intermittent and transient faults.

Fault Injector: The Fault List can be used to produce a Faulty SUT only in terms of reproducible and prearranged fault models. Fault models are characterized by a type (e.g. omission, frozen, delay, invert, oscillation or random), target location (e.g., electronic component pin or signal in a software model), triggering time (e.g. event-driven or after a given time), and duration. To create a Faulty SUT, the Fault Injector injects a saboteur per fault entry from the Fault List together with the associated fault models coded as templates in a fault library. Saboteurs are extra components added as part of the model-based design for the sole purpose of FI experiments.

Monitor: After having included the required amount of saboteur blocks, the Faulty SUT is ready to be simulated, as well as the Golden SUT. The Monitor tracks the execution flow of the golden (faulty free) and faulty simulation runs via the readouts collection activity. As the Workload Generator introduced extra readout blocks to the original SUT, the Monitor observes the same data and events into de Golden and Faulty SUT to be compared by the data analysis activity. Dynacar acts as a simulation platform that incorporates different vehicle subsystem models, potentially developed in any of the following languages: Simulink, Matlab, or dSpace. Users can choose between using the 3D driving environment (Human in the Loop) for virtual driving, or a default driving environment. Finally, we are able to report the corruption effects either quantitatively or qualitatively for fault forecasting and fault removal, as described in the next section.

The uncertainty related to automated driving functions makes safety analysis definitely not sufficient, requiring additional virtual and simulation solutions. Forthwith, FI establishes itself as a way of completing and verifying previously carried out safety analyses. Given that figuring out the system reaction under the effect of real faults can be really a burden-some issue, these experiments arise as a viable solution.

Our approach has been evaluated on a case study for the model-based design of steering control functions embedded in urban vehicles. We focused on forecasting the fault detection interval for permanent faults based on the maximum lateral error and steering saturation. By setting different fault durations, the fault detection time at system/component level and FTTI (Fault Tolerant Time Interval) are obtained. These values determine the required level of fault tolerance−e.g. redundancy or graceful degradation−not to lose the control of the vehicle.

The Sabotage tool framework is currently been developed in the AMASS ECSEL project

Share this post