An important challenge for researchers and professionals related to automated vehicles is to ensure that the operators of these vehicles are capable of perceiving not only the capabilities, but also the limitations of the automated systems (Flemisch et al., 2017).Until Automated Driving Systems (ADS) become capable of performing all types of driving tasks in all types of road conditions, human drivers will be responsible for taking control when the system fails or reaches its operational limits (Varotto et al., 2015). Several studies underline that, as the level of automation increases, the role of the driver remains crucial, and traffic safety is increasingly dependable on the combination of the performance of the human driver and the automation (Merat et al., 2012; Petermann-Stock et al., 2013).

In order to be able to evaluate the safety of new vehicle systems for automated driving, traffic simulations can be used, since most of these systems are not yet widely implemented in real traffic and accidents don’t occur often in practice. Also, when an accident has happened, often the exact cause is unknown. Traffic simulation can address this gap, however it requires realistic modelling of both the human driver, the automated vehicle systems, as well as reliable safety indicators. For safety assessment, the moment when the driver needs to take over control from the automated system is crucial. In order to assess how (un)safe such a takeover is, an indicator is needed that reflects the (un)safety of that specific situation.

Existing research has proposed taxonomies for such transitions of control, depending on who initiates the transition and who is in control after the transition. There are four types of transition (Klunder et al., 2009; Lu & Winter, 2015; Lu et al., 2016):

This research focuses on the third type of transition, which is considered to be the most complex and safety critical one, as it involves mechanisms of human adaptation in emergency scenarios under limited time. The most common situations for an AIDC takeover are when there is a sensor failure, when the system reaches its boundaries (in terms of speed, acceleration/deceleration), e.g. to avoid a stopped vehicle, when the vehicle exits its operational design domain (ODD), e.g. approaches an off-ramp, or when the system deliberately initiates a takeover in order to increase the alertness of the driver.

By that time, the system sends a warning (takeover stimulus) to the human driver, who has to take over within a certain time to ensure safe operations. Such a time sequence has been described by Zhang et al. (2019): the time from the moment of the warning, until the moment of driver’s first intervention, is called Take Over Time (TOT). The last moment at which the driver can take action is called ‘system limit’. The time between a warning and the system limit is called Time Budget (TB).

Typically, when the Take Over Time is smaller than the Time Budget, the situation is considered to be safe. However, comparing the TOT with the TB is not always a reliable safety metric, as both metrics are dependent on the driver’s perception and reaction times. These times are usually taken as mean values, not accounting for the variability in human information processing. Moreover, in certain conditions, it may be the case that after the driver took over, the deceleration force applied, or the maneuver performed, were not sufficient to avoid a collision. While in traffic simulation models this type of ‘poor’ driver performance is seldom considered (Calvert & Arem, 2020; Lint & Calvert, 2018), in reality drivers may not perform in an optimal way due to e.g. misconception of the situation, wrong cognitive processing of the available information, or simply poor task execution (Kyriakidis et al., 2017). Both TOT and TB metrics are not designed for capturing that ‘poor’ and potentially unsafe task performance. Therefore, a simulation experiment in which TOT and TB satisfy the aforementioned condition (TOT < TB), may not necessarily ensure that critical situations will be evaded in practice.

This research aims to develop and test a set of new Key Performance Indicators (KPIs), in order to estimate driver performance and its safety consequences more accurately. More specifically, we propose two new metrics for takeover performance; in these metrics, in addition to the perception and reaction time, the actual task execution time is taken into account. To test these metrics, we use a traffic simulation experimental setup with mixed traffic of conventional vehicles and Cooperative Adaptive Cruise Control (CACC) platoons. The simulation output is used to calculate the share of critical conflicts and virtual ‘crashes’ on the basis of the new proposed metrics. The results are analysed with respect to automated vehicles’ (AV) penetration rate, and vehicle class.

The remainder of this paper is structured as follows: section 2 presents a literature review of simulation studies on mixed traffic, as well as a review of takeover performance in automated driving, its determinants and its safety implications. Section 3 presents the methodology of this research, including the development of the new metrics. Section 4 includes the experimental setup of the simulation. Section 5 presents the results of the analysis. Finally, section 6 includes a discussion on the results and limitations of the study; conclusions are drawn in section 7.

For many AV applications, the safety impacts in mixed traffic (conventional and automated vehicles in different penetration rates) are assessed by means of traffic simulation, e.g. Arvin et al. (2020); Dahl et al. (2018); Liu et al. (2018); Cacciabue and Carsten (2010). A number of Surrogate Measures of Safety (SMoS) (often also referred to as SSM—surrogate safety measures) are typically used to assess the criticality of virtual conflicts observed in the simulated network. These include Time-to-Collision (TTC), Post Encroachment time (PET), Deceleration Rate (DR), maximum speed, speed differentials or variations of the above (Gettman & Head, 2003).

Table 1 summarizes a number of recent simulation studies on the safety of CACC in mixed traffic. The selection criteria for the summarized studies were studies of the last five years published in a scientific Journal, with a clear safety focus. A detailed review of these studies is beyond the scope of this paper, for this the reader is referred to Papadimitriou et al. (2022). It is observed that the majority of studies, by using different SMoS and thresholds, report significant safety benefits from the increase in AV penetration rates in the simulated networks, including highways and junctions, e.g. Papadoulis et al. (2019); Virdi et al. (2019); Rahman and Abdel-Aty (2018); Ye and Yamamoto (2019). One study (Sinha et al., 2020) found little or no safety benefits for conventional vehicles from the increased AV penetration rate, and argued that AV ‘behaviour’ thresholds derived from conventional, human naturalistic driving should be revisited.

Indeed, the safety thresholds used in these simulation studies, e.g. for deceleration, TTC or PET, vary among studies are not standardized in general. It is noted that none of these studies examine specifically the impact of authority transitions on the network safety. Whilst SMoS are available (and still developing) for regular driving, their counterparts for control transitions are still in early stages of development.

There are several design factors and human factors that are involved in a safe authority transition. These have been extensively reviewed in recent review studies and meta-analyses (Winter et al., 2014; Zhang et al., 2019) and only a summary of main findings is presented in this paper (see Table 2).

Many studies have shown that in low level automation, where drivers are still expected to stay engaged in the driving task, reduced cognitive awareness is still observed. The required shift in attention is demanding by itself, and prone to errors and delays, especially if a secondary task is involved (Borojeni et al., 2016; Calvert et al., 2020; Merat et al., 2012).

Several studies have examined the impact of the modality of a takeover request (TOR) on takeover performance, e.g. ‘why messages’ (describing reasons) and ‘how messages’ (describing action) in Koo et al. (2015), visual, auditory or tactile warnings in Naujoks et al. (2021); Politis et al. (2018); Baldwin and Lewis (2014), peripheral light displays in Borojeni et al. (2016), comparison of the modalities of the driving task and the non-driving task in distracted driving in (Petermeijer et al., 2016). In the meta-analysis ofZhang et al. (2019) it was found that when auditory or tactile takeover request was used, the difference in TOT was on average -1.41 s.

In the meta-analysis of Zhang et al. (2019), TOT varied from 0.69 s to 19.79 s. The mean TOT was 2.72 s with a standard variation of 1.45 s. The results greatly vary depending on several drivers and other factors, such as prior experience and time budget magnitude. Time Budget in particular was found to strongly affect TOT, resulting in a difference of +1.35 s. Prior experience also influenced TOT (drivers showed on average -1.00 s TOT when they were taking over for a second time). On the other hand, using a handheld device or being involved in a visual non-driving task increased TOT by +1.33 s and +0.29 s respectively. High traffic density had a marginally significant effect of +0.49 s on TOT.

Table 2

Source	Method		Surrogate Measure of Safety (SMoS)		Results
	Simulation	Meta-analysis	Reaction/takeover time	Other
(Winter et al., 2014)		+	+	+	Reaction time to critical events increases with secondary tasks and higher levels of automation.
(Zhang et al., 2019)	+	+	+		Takeover time ranges from 0.7 to 20 s, with mean 2.72 s. In general, longer time budget results in longer TOT.
Mac Donald et al. (2019)	+	+	+		Takeover time increases per 0.27 s for each 1 s of extra time budget available.
(Papadimitriou et al., 2020)		+		+	In addition to TOT, takeover quality is an ‘umbrella’ term for many logitudinal/lateral control or driver behaviour indicators affecting the safety of takeovers.

The above literature review indicates that the heterogeneity in human perception and reaction time in different situations may result in large variability in TOT. A large share of this distribution is far beyond the desired time gaps that an advanced CACC system can achieve (i.e. 0.6 to 0.9 s). This means that, for instance, in a hypothetical immediate full stop of the front vehicle, a collision would be inevitable, as the available time for a driver to react is not sufficient.

As also noted by Zeeb et al. (2016), TOT may not be the most important element when assessing the performance of an authority transition, because some studies have shown that shorter TOT may come with lower quality. The general term ‘takeover quality’ (TQ) includes different longitudinal and lateral control indicators, but also driver factors such as hazard perception and situational awareness. As described in the review of Papadimitriou et al. (2020), TQ has been found to be a better safety indicator in terms of lateral control, number of steering corrections, and in non-emergency transitions, whereas other indicators (e.g. TOT) could not capture the differences between emergency and non-emergency situations. Ruscio et al. (2015) underline that a driver’s perceiving and cognitive awareness affects his ability to mentally process a takeover request, and thus the takeover performance too; therefore, the standard approach that uses reaction times (TOT < TB) seems to be inefficient to ensure the safety of a transition, despite the fact that it is, in general, a good estimate regarding the criticality of incidents.

Overall, existing studies do not adequately link TOT with the actual safety consequences; for instance, it is not known how TOT correlates with the SMoS that can be empirically observed through traffic simulation. On the other hand, the existing simulations of CACC and mixed traffic use several common SMoS with different thresholds among studies, but do not test the conditions of authority transition. Hence the reported safety benefits do not cover this safety-critical case. In this paper, we argue that, by taking into account the braking time of the driver, it is possible to link takeover performance with critical conflicts or crashes.

Traffic simulation models do not allow virtual ‘crashes’ to occur by design; their input parameters and model specifications ensure safe interactions, as in most cases the modelling purpose is that of traffic efficiency. The safety impacts of traffic scenarios in general, and takeover scenarios in particular, are based on an assessment of the criticality of certain conflicts by means of SMoS. In this research, we assume that, by incorporating more elements of human driver behaviour in simulation, and allowing for virtual crashes to be observed by post-processing the outputs, it is possible to assess the safety implications of authority transitions in a more realistic way.

The present study is based on the data generated by the CACC simulation model created by Xiao et al. (2018). This model highlighted the influence of system deactivations in CACC mode on traffic flow on bottlenecks. This resulted in a dataset, which includes different types of authority transitions. Also, the CACC model manages to capture driver-system interactions in a realistic way on the basis of data. More specifically, the longitudinal response of the CACC model is an empirical model that is derived from vehicle data measured from a field test consisting of four CACC vehicles on public roads. The experimental scenario is intentionally kept simple to avoid ‘noise’ and allow the emphasis of this research to be placed on the already complex translation of traffic flow impacts to safety impacts.

Many studies regarding authority transitions in automated driving have been performed in the past with little to no focus on the identification of optimal transition times with respect to concrete safety outcomes. The main contribution of this research, compared to previous studies performed in this field, is the introduction of a new method for identification and assessment of conflicts’ criticality in authority transitions. To the best of the authors’ knowledge, it is the first time that actual (and variable) maneuver execution times are considered for the evaluation of critical incidents.

A main finding of our study is that by increasing the penetration rates, safety due to authority transitions’ implications is compromised. Other studies performed so far claim that safety is increased by the deployment of AV in general (Papadoulis et al., 2019)—but none of these studies has tested in their simulation the impact of authority transitions in particular. Moreover, although several studies have reviewed the factors affecting takeover performance (Papadimitriou et al., 2020), and found several traffic and environmental determinants (e.g. higher traffic density, higher level of automation and adverse weather increase TOT), no study has associated takeover performance with penetration rate of AVs. Our study suggests a deterioration of safety metrics during authority transitions in higher penetration rates. The prevalence of this type of accident mechanism may have been underestimated in the past.

A main reason for this deterioration is that the desired time gaps under which active automation systems operate is significantly lower than those that human drivers are comfortable to drive in. This difference in the time gaps seems to be critical when it comes to this type of safety maneuver, since the available time for the driver to react and perform an avoidance action is limited. The expansion of time gaps to higher values, which are more in line with the capacities of human drivers in case of deactivation, should be considered especially when the ODD of the corresponding vehicle has also a finite capacity. For example, an automated vehicle that is equipped with a CACC system capable of achieving times gaps of 0.6 s, operating under SAE level 2 (limited ODD), could be programmed in such a way that its minimum value is not used, and the time gaps allowed are closer to those of human drivers are comfortable driving with (although this would have a negative effect on throughput). Overall, the results—although exploratory—imply the significance of a harmonized AV system ensuring smooth transitions, and the importance for adequate time for drivers to react.

Another interesting finding is the strong relation of the TC with the initial speed of the vehicle. This seems reasonable because the higher the velocity of a car is, the more time is required to decelerate to a point that a collision is avoided. However, this time is also dependent on the behavior of the leading vehicle. If in a traffic scenario the leading vehicle starts accelerating again before coming to a full stop, there will be no reason for further deceleration of the following vehicle, therefore avoiding a potential collision in less time. This is the case when shockwave traffic jams appear and disappear. A way to deal with this problem is the use of a dynamic design of takeover requests’ strategies with respect to warning times, where the stimulus may be triggered depending on the vehicle’s speed.

In addition, a rather unexpected finding was the fact that different desired time gaps did not seem to significantly differentiate from each other in terms of the safety implications of the transition. It was assumed that vehicles capable of achieving smaller time gaps would result in higher numbers of critical incidents. A possible explanation is that the algorithm used in this study was designed in a way that vehicles drive with an active CACC mode under desired time gap of 0.6s as much as possible. Thus, no random distribution between different classes and operation modes existed, making the sample for the rest of the classes insignificant.

The developed model in this study has some limitations. Firstly, the data that were used as input for this study were derived from a simulation study that was initially designed to assess traffic efficiency and not traffic safety. This means that several KPIs had to be either calculated outside of the model or based on similar studies. More specifically, the reaction times which were extracted from the previous simulation model were equal to zero. This means that by the time a collision warning was triggered, the driver was assumed to immediately start braking at the same time step. This is not realistic, as the total time that the driver kept braking until the -2 m/s² threshold was not the actual braking time but the entire Time to Control (TC). An adjustment had to be made based on the TOT derived from (Shladover et al., 2015). The same applies to the Time Budget metrics. These limitations led to different assumptions for the calculation of various metrics, which need further exploration and validation in future research.

Another potential limitation comes from the assumption that automation would automatically reactivate again after the -2 m/s² threshold. It might be the case that the braking action performed till that threshold would not be sufficient to avoid a collision, and the automation takes over control of the vehicle before the completion of the safety maneuver. This means that if this setting for reactivation is not implemented, the actual number of critical incidents and crashes might have been significantly higher.

In addition, for the calculation of the two thresholds for the corresponding two ΔΤ values (STB-TB and TB-TOT), but also for the TOT and TB themselves, the mean values of relevant metrics where used. In reality, collision risk is not determined by such mean values, but rather by the outliers of their distributions instead, as these extreme values better represent poor driving behavior.

In order to demonstrate our proposed methodology, we had to adopt or adjust threshold values from the literature, because standard values do not exist for TOT—and no threshold values exist for our new proposed KPIs. It is thereby acknowledged that the proposed values may be improved in further research. Moreover, a sensitivity analysis with respect to the parameters of the simulation would provide additional insights, however unfortunately the authors do not have access to the dataset for further analysis at this stage. Such sensitivity analysis of safety outcomes to simulation parameters should be prioritized in future research.

The objective of this research was to provide insights into the safety implications of authority transitions in AV operations, by taking into consideration the available time between a collision warning and the potential collision. Two new safety indicators, namely Time to Control (TC) and Safe Time Budget (STB) are introduced, based on the fundamental safety indicator of Time to Collision; these new indicators combine the perception/reaction times and the actual braking times, allowing for a more accurate safety evaluation.

From a methodological perspective, the proposed indicators would be useful not only for safety evaluations in microsimulation studies, but also for safety evaluations of real-world pilots, small scale driving tests or driving simulator studies with new vehicle systems involving authority transitions.

From a modelling perspective, by incorporating the actual braking times, more emphasis is put on human behaviour, which can be the critical determinant between a safe vs. an unsafe outcome in authority transitions. Our study reveals the added value and the potential of post-processing of existing simulation datasets for safety purposes, as safety is seldom the primary goal of a simulation model. In this sense, our proposed method and indicators could be applied to other AAC/CACC simulation outputs, by researchers that wish to test a more realistic assessment of the safety impacts of automation disengagements in their network. Further research should be conducted to explore the potential of this method in other contexts or scenarios.

Moreover, our results highlights the known limitations of simulation models and the challenges to realistically represent human behavior and its safety implications. Previous studies on AV safety did not take into account authority transitions in their simulations. Therefore, while the safety benefits shown in their results may be valid for regular AV driving, the negative safety impacts of authority transitions are not taken into account.

From an engineering and policy perspective, the current and future findings in this field show directions for improvement for vehicle manufacturers to redesign automated driving systems in a way that will improve overall safety, not only under normal conditions, but also under emergency situations, e.g. extending the ODD of automated systems, so that they are capable of addressing more traffic conditions (safety-critical situations, low-speed operations), making them more resilient to disturbances, and limiting the amount of system deactivations and thus preventing safety problems. Transport planners and road operators should be aware of the human limitations in takeover tasks, and be able to account for these variations in their simulation models to ensure safety in mixed traffic.

The authors declare no competing interests.

No dedicated funding was received for this project.