Maturity measurement in road traffic injury prevention
1
,
Letty
Aarts
2
,
Dan
Campsall
3
,
Laurent
Carnis
4
,
Veronique
Feypell
5
,
Tanya
Fosdick
3
,
David
Shelton
6
,
Martin
Small
7
,
Anna
Vadeby
8
Handling editor: Aliaksei Laureshyn, Lund University, Sweden
Reviewers: Hanna Wennberg , Swedish Transport Administration, Sweden
Tor-Olav Nævestad, Institute of Transport Economics, Norway
Nicholas John Ward, Leidos, the United States of America
Received: 2 January 2024; Accepted: 10 March 2024; Published: 28 March 2024
Abstract
Road traffic related death and injury continues to be a major challenge globally. Unsafe road use is particularly evident in low- and middle-income countries while also being a growing concern for private sector organisations. The Safe System approach is recognized internationally as the leading approach to improving road safety and previous work has codified the essential management functions and interventions evident in its successful implementation. Tracking the development of Safe System adoption within the public and private sectors is of interest for several reasons. This paper presents recent development and use of road safety maturity frameworks and discusses the utility of these approaches for road safety practitioners and researchers.
Keywords
frameworks, maturity, models, Safe System
Introduction
The world's best performing countries in road safety have seen a steady decline in road traffic related death and injury, generally beginning in the early 1970s, and now recording fatality rates around 20 to 30 deaths annually per million population (ITF, 2022). In contrast, at a global level, fatalities and injuries continue to climb despite these best-in-class successes evidencing that road traffic injury is a solvable problem.
The World Health Organization reports that over 1.3 million people are killed annually in road use and that in some countries, the rate exceeds 300 deaths per million population (WHO, 2023). The economic costs of such losses have been estimated at up to 7% of Gross Domestic Product (Elvik, 2000; Jacobs et al., 2000; OECD, 2006; Wijnen & Stipdonk, 2016; iRAP, 2021)
Research has confirmed that there is a correlation between the maturity of road safety initiatives and road safety collisions (Amador & Willis, 2014). Road safety in countries with a high level of Safe System implementation, such as Norway (Elvik & Nævestad, 2023), Sweden (Belin et al., 2012; Johansson, 2009), or the Netherlands (Wegman et al., 2006; Weijermars & Wegman, 2011), show how this approach helps to improve road safety effectively. This was shown in many research papers, with examples found in Wegman et al. (2015), Edvardsson Björnberg et al. (2020) and Bliss and Breen (2013). Put another way, the occurrence of death and injury in road use reflects the levels of knowledge, skills, experience, and will that are present within the management and operation of the road system. As a result, measuring and understanding an organisation's ability to manage and operate a safe road system is of great interest. This interest has led to a large body of work to codify the road safety management discipline and transfer good practice to improve performance at a country, region, city, organizational and project level. One notable example is the road safety capacity models developed by the World Bank, commonly used for national road safety capacity reviews (Bliss & Breen, 2013). More recently, attention has moved to codifying the level of maturity evident in road safety management systems (Espiga, 2019) following earlier developments in occupational health and safety management (Filho & Waterson, 2018; Fleming, 2001).
This paper reports progress in the development of road safety maturity frameworks and presents three such frameworks designed for use at a project, organizational or country level. These frameworks can be used by project leaders, road authorities, government representatives, companies, NGOs, and other actors to decide to what extent the road safety problems they face, and the plans they envisage to improve the situation, match the current scientific knowledge about effective interventions. The use of these frameworks helps to prevent implementation of less effective interventions. These frameworks consist of concentrated descriptions of best practices, allowing for adjustment to specific local situations for new road safety challenges. The purpose of bringing these frameworks together in this communication is to make the road safety community aware of their existence, and to encourage their use. Through practical application, these frameworks can then be further refined, ensuring that they act as meaningful tools in the future implementation of Safe System actions.
The main dimensions of each model are presented together with commentary on the theoretical and practical links between the frameworks. Ongoing research in the area is discussed.
Road safety maturity frameworks
It is important to differentiate between being able to identify the characteristics of a given developmental stage and being able to recommend with confidence what is needed to move from one stage to the next. A common intention across the developers of all three frameworks presented here was the desire to contribute usefully to the progression in maturity of road safety management systems which exist in the delivering of sustained measures. The term ‘road safety management system’ is used here to refer to the combination of knowledge, skills and attitudes employed by individuals and organizations together with the enabling environment—the structures of power and influence and the institutions—in which they are embedded (OECD, 2006). From this common starting point grew three different approaches to understanding and measuring road safety capacity maturity. Each approach is presented below as work in progress and describes a current best practice to improving road safety based on scientific research and successful implemented policies.
Whilst these frameworks themselves are not science based, their content is. The interventions and governance structures proposed in all three frameworks are based on evidence from successful Safe System implementation in multiple territories since the 1990s (Academic Expert Group, 2019; Breen et al., 2018; ITF, 2016; Turner et al., 2021; WHO, 2017; WHO, 2021; Wallbank et al., 2021; Weijermars & Wegman, 2011; Welle et al., 2018). The Agilysis framework further draws on maturity assessment models from other safety domains (Breen et al., 2018; Cooper, 2000; Hudson et al., 2000; Westrum, 1993), as well as an evidence-based behaviour change model (Darnton & Horne, 2013). Furthermore, the creators of these frameworks have extensive knowledge and experience of applying Safe System thinking in multiple territories, including understanding the challenges and opportunities for those trying to eliminate road risk. These frameworks, therefore, are based on a high-level of expertise, used to provide supportive tools for practitioners and policymakers.
Safe System framework (ITF)
The International Transport Forum (ITF) recognizes that implementing a Safe System approach is the most effective and efficient way to improve road safety (ITF, 2008). However, introducing a Safe System is not easy, especially in low- and middle-income countries. To guide those seeking to implement the Safe System approach, a theoretical framework has been developed (ITF, 2022).
Aim of the ITF framework
The aim of the Safe System framework is to develop a practical tool to help countries, organisations (of all types) and projects make progress in Safe System implementation based on practical experiences, whatever their project concerns. The aim is to visualize what the Safe System should look like in various contexts and outline the types of activities required at different stages of the Safe System journey toward the elimination of serious road trauma. The framework provides the necessary structure to implement a Safe System's project, policy or strategy using the key components in conjunction with the road safety pillars. The ultimate goal is to turn the Safe System framework into a tool for road safety assessment, counselling Safe System implementations, or Safe System indicators.
Description of the ITF framework
The Safe System framework covers three dimensions:
-
five Safe System key components
-
six road safety pillars
-
three development stages.
The first key component (establish institutional governance) is related to legal, administrative and regulatory characteristics. The next two key components stress the importance of cooperation between partners (share responsibility) and a holistic approach (strengthen all pillars). Only then does the framework focus on the consequences of severe crashes, the strong forces on the body that cause serious or fatal harm and support safe road user behaviour (ITF, 2008; ITF, 2016; WHO, 2021).
The framework yields the following six road-safety pillars: road safety management, safe roads, safe vehicles, safe speeds, safe road user behaviour, and post-crash care (Job & Mbugua, 2020; WHO, 2011; WHO, 2017; Welle et al., 2018).
Together, the two dimensions create a matrix of combinations. In each combination, safety improvements can be implemented and assessed. The pillars define the columns of this matrix, while the key components define the rows (see Figure 1). Safe System maturity levels can be different for each combination of key component and pillar.
To assess progress and identify implementation gaps in any country, region, city, organisation, or project where a Safe System intervention is introduced, three possible development stages are described 1 .
-
Emerging. There is awareness and knowledge of what a Safe System looks like.
-
Developing (originally ‘Advancing’). Interventions and policies are linked and organised by robust institutional governance focused on road safety, transport, and mobility.
-
Mature. Highly sophisticated interventions are used in technical and public areas.
Taken together, these three stages comprise the third dimension of the Safe System implementation framework. They also signify a gradual progression from simplicity to sophistication. This allows users to assess where they stand, for each of the 26 cells in Figure 1, and decide in which order to proceed towards more maturity within the scientific basis of a Safe System.
Characteristics of the ITF framework
The Safe System framework serves several possible purposes:
-
To provide general guidance about the kind of interventions that should be considered by countries, regions, cities, and organisations applying the Safe System approach, depending on their stage of development (emerging, developing, or mature).
-
To analyse the Safe System content of existing cases of Safe System implementation. This can encourage improvement by evaluating lessons learned and collecting information about possible future steps to enhance effectiveness.
-
To assess projects, planned Safe System projects or sets of interventions to help improve their Safe System content, identify opportunities for improvement and provide professional guidance to maximise effectiveness.
The framework stresses the importance of interdependence and multiplier effects between policy interventions and actors. Although it is legitimate to break the road-safety problem into smaller components for analysis and planning purposes as part of a Safe System approach, partners should not take a ‘silo’ approach to road safety pillars. The pillars should be viewed as interlinked parts of the whole system, which implies a broad perspective on strategy and important resources.
Applications
The framework has been applied to several road safety improvement cases from all continents. Some preliminary examples have been published in ITF (2022). Readers are referred to these descriptions for a further impression of how the framework could be used for maturity assessment.
More recent examples have not yet been published but are expected to be referred to once the ITF has published a practical tool for the application of this framework. The purpose of this practical tool is for road safety actors from companies, countries, and NGOs to apply it to their projects or plans, and to guide them towards effective steps and measures. We hope to be able to learn from these applications to further improve the framework.
Safe System Cultural Maturity Model (Agilysis)
Agilysis created the Safe System Cultural Maturity Model (SSCMM) and accompanying diagnostic tool (Campsall et al., 2022; Fosdick et al., 2024) to assist those organizations managing and delivering road safety activities to adopt and deliver Safe System actions. The framework combines Safe System guidance with principles of safety culture from other sectors, such as occupational health, coal and mining, aviation and maritime navigation, and nuclear regulation (Figure 2). To adopt a Safe System approach, an organization cannot just state its ambition; it needs a culture that supports individuals to develop deep-seated motivation arising from belief in the approach and the organization to develop actions that are consistent with its values.
Aim of the Agilysis framework
The SSCMM was created to assess the readiness of public agencies in the adoption and implementation of Safe System thinking and actions. Safe System cultural maturity is diagnosed in several ways: the tool identifies which specific Safe System actions are being delivered, but it also investigates how Safe System thinking is embedded into any organization. Once a level of Safe System cultural maturity has been diagnosed, the organization can use the findings to identify where Safe System culture could be strengthened.
A future aim is to identify methods for strengthening Safe System cultural maturity, providing case studies for others to emulate.
Description of the Agilysis framework
There are many organizations who are seeking, or should be seeking, to adopt Safe System practices. These include road authorities, police forces, fire and rescue services, local and national government, and road safety partnerships. It is not possible to create a Safe System overnight and, as it requires a systemic approach, it cannot be the responsibility of one organization or one individual within an organization.
The culture of an organization is shaped in many ways, including through the beliefs, attitudes, and values of its members (Cooper, 2000; Hudson et al., 2000; Westrum, 1993). Culture is also shaped by the way actions are applied, through dedicated roles and responsibilities (Darnton & Horne, 2013). Lastly, there are the multiple operations of delivery, ranging across engineering; education; enforcement; legislation and regulation; leadership and cooperation; standards and training; investment; and research, monitoring, and evaluation (CCMTA, 2016; Etika, 2018; NZTA, n/d). This tool seeks to help organizations to understand their current level of Safe System cultural maturity so that they can pinpoint where additional roles, resources, and efforts should be concentrated to become more mature.
Figure 3 describes what an organization might say, do, or think at each maturity stage, shown in the columns, and lastly, what they need to do to mature. For each level of maturity, the descriptions in the columns suggest what an organization might look like.
To determine the Safe System cultural maturity level of an organization, a question set was developed covering actions across the Safe System. Each question is accompanied by a set of five statements, which respondents must select from, with the statements related to the five stages of cultural maturity. The statements relate to specific actions but also explore attitudes across the organization, related to leadership, roles, and resources and map across the descriptions shown in the table.
Each statement is scored from 1 (vulnerable) to 5 (advanced) to reflect the five cultural maturity stages. The survey tool is applied online to anonymously collect the data from those across an organization. The scores from statements associated with 22 questions are analyzed to provide averages for the Safe System components and the areas of operation, providing graphical outputs to organizations so as to understand where its members indicate the Safe System culture is more or less mature.
The survey tool is designed to be completed by a representative sample of individuals across an organization, seeking to gain responses across departments and levels of responsibility. The larger the sample size, the more confidence the organization can have that the results reflect its current level of maturity. Engaging a wide array of respondents, playing a mixture of roles both in delivering Safe System activities and supporting those who deliver, will provide depth of insight into how far Safe System culture has been established.
Characteristics of the Agilysis framework
The Safe System requires strong leadership, effective coordination, and established safety culture to be implemented correctly. Organizational culture can influence how Safe System policies and strategies are implemented.
In some organizations, there is no road safety culture at all, little or inappropriate concern; whilst for others, there is a commitment to continue with ‘traditional’ road safety practices, because there is a lack of understanding, resources, and/or will to adopt a Safe System approach. This tool can help organizations diagnose their current level of maturity; it allows comparisons between organizations; can be used to evaluate progression or regression over time; and to support transformation of organizations in specific Safe System areas or actions.
The collection of data from an anonymous tool deployed across the organization provides members from a variety of roles to share how they perceive the prevailing culture of the organization and how it is delivering Safe System actions. By collecting quantitative data, it is possible to measure progress over time and compare maturity levels with similar organizations. A potential weakness is participation, with those less involved in road safety also less likely to engage with the tool or to provide an honest reflection of the situation.
To date, the framework has been applied to two national highways authorities and more than 10 regional road safety partnerships in Great Britain. The survey tool has been used to diagnose where in the Safe System these organizations were performing more maturely and where more progress is required. The analysis also explored the components of the Safe System and the operational measures used within the organization to deliver Safe System actions. The output for one national authority is being used to develop a cultural maturity playbook, providing advice on how Safe System thinking can be embedded across the areas identified as less mature in the survey tool.
Road Safety Maturity Framework (ADB)
Developed by the Asian Development Bank (ADB), the Road Safety Maturity Framework (Small et al., 2023) recognizes the importance of improving road safety as a key response to achieving the targets set out in the UN Sustainable Development Goals. While originally developed for use among countries of Asia and the Pacific, the framework is not geographically limited. Designing effective investments in road safety at a country level is aided by an understanding of the country's maturity in road safety, the priority areas for improvement, and the state of readiness to take action. ADB's framework, Assessing the Maturity of National Road Safety Management Systems, sets out a methodology for country level assessment and comparative analysis.
The framework was developed in consultation with the partner countries which form the South Asia Subregional Economic Cooperation (SASEC) program. Assessments conducted in Bangladesh, Bhutan, India, Maldives, Nepal, and Sri Lanka, as part of the SASEC program, helped test and validate the framework and the methodology followed. The Asian Development Bank is in the process of promoting the use of the model for other developing member countries, including through the Asia Pacific Road Safety Observatory.
Aim of the ADB framework
ADB developed this framework to support developing member countries in their self-assessment of road safety capacity, and to identify the most useful next steps in capacity development. A key result of applying this framework is the common understanding reached between ADB and its member countries on what are the investment priorities for consideration in future programs are. The standardized assessment framework enables comparative assessments between countries and sub-regions and supports the objectives of the Asia Pacific Road Safety Observatory in generating engagement and action by member countries on improving road safety performance. The framework is in use and is expected to be refined on a continual basis.
Description of the ADB framework
Several existing models were considered when developing the framework, starting with the Safe System approach as it has been documented by the International Transport Federation. The Global Road Safety Facility (GRSF) model developed by Breen et al. (2018) was particularly influential because of its emphasis on institutional foundations, which were considered relevant across Asia. The assessment framework was developed from the notion that effective national road safety management systems tend to have three linked elements: (i) well-organized institutional management functions, which; (ii) produce high-quality interventions, which; (iii) in turn, produce the results sought. The UNRSF (2018) framework illustrated the value of concrete expression of actions and options available to demonstrate different levels of maturity.
From this analysis, a set of 15 dimensions were identified against which assessments could be made. The resultant framework establishes five levels of maturity of national road safety management systems (see Figure 4) in terms of three components:
-
overall climate of concern for road safety
-
focus of the safety-related activity
-
government preparedness to invest in road safety.
These five levels of maturity are described across the 15 dimensions (see Figure 5). The Advanced level serves as an ultimate level of maturity which could be discerned clearly from the GRSF framework. On assessment, each of the 15 dimensions may be rated at any of the maturity levels. This allows for the complexity and non-uniformity in maturity development evident within countries. Legislation for example may be assessed as Emerging, while Leadership is rated Maturing.
The complete published framework (Small et al., 2023) contains a description of the level of maturity (column) evident for the particular dimension (row) of road safety under consideration.
Working through the lead agency, the assessment involves a briefing of country respondents who then complete a questionnaire covering all 15 dimensions, rating responses to each question. The scores are averaged and aggregated into one score for each of the 15 dimensions, which is workshopped with respondents. Drawing on experience, research, and country responses, the assessment team record a score for each of the 15 dimensions, adding comments on the most important aspects of each. The maturity ratings for each dimension are used to identify national priorities along with a consideration of those issues likely to generate the greatest traction in countries.
The ADB framework is a technical reference and should be used by professionals technically proficient in road safety management. A team of at least two such professionals is recommended.
Characteristics of the ADB framework
The ADB framework provides an analytical foundation for strengthening national road safety management systems in low- and middle-income countries. The framework has shown itself effective using the same assessment team to compare maturity across multiple countries. Concurrence of results when different assessors are used has not been tested. By describing the levels of maturity with reference to specific actions, milestones, programs, and policies, the framework is relatively accessible to technical and executive leadership. The framework can generate valuable and specific insights on capacity improvement opportunities, and tie insights back into the body of road safety management theory and practice. Five levels of maturity provide the granularity needed to see countries move in maturity in response to capacity building efforts.
Discussion
This paper illustrates the manyfold and intensified cooperative international efforts to provide practical guidelines for countries to improve road safety performance. The three maturity frameworks presented cover different organisational structures (see Table 1). While the ITF framework covers six road safety pillars with only one dimension directed at management, the ABD approach is more heavily weighted to road safety management. In contrast, the Agilysis framework addresses quite another dimension: the safety culture, needed to increase the willingness to change. The ADB approach is especially relevant and powerful for those acting on a national level, aiming for a systematic improvement of the Safe System approach. The ABD tool allows for a large variety in stages of development; hence many countries can apply their methods. The ITF method is mainly developed with a focus on (low and middle income) countries, but might also be used by regions, cities, organisations, and projects. The Agilysis framework has a main focus on organizations, providing them with a method and material for introspection to find out how they can improve towards a safe road traffic system.
|
Framework |
Focus |
Projects |
Organisations |
Cities |
Regions |
Countries |
|---|---|---|---|---|---|---|
|
ITF |
Policy and practice |
✔ |
✔ |
✔ |
✔ |
✔ |
|
Agilysis |
Organisational culture |
✔ |
✔ |
✔ |
||
|
ADB |
Management |
✔ |
✔ |
Currently, ITF and partners are developing a practical tool for those who want to improve the Safe System content of their road safety improvement approach (ITF, n/d). This is done in cooperation with many, such as the ADB, Agilysis and many others. This cooperation revealed that similar approaches can greatly enhance each other. Agilysis have tested their tool with road safety partnerships and two national road authorities in the UK and are looking to extend testing to determine the universality of approach and refine as necessary. ADB have applied their framework across countries in South Asia and will soon assess Central Asian countries with a view to its application across Asia and the Pacific.
The maturity concept could help with strengthening the interventions delivered by these jurisdictions and organizations while developing a common set of notions, assisting them in developing sustained road safety policy. One of the goals is that these available practical guidelines align and strengthen each other, refer to one another, helping practitioners all over the world to use these guidelines to their best ability and to contribute to designing efficient road safety policies and facilitate their implementation.
Although the three maturity frameworks differ in the target audience and the scope as illustrated in Table 4, they are alike in the sense that each strive to invite road safety professionals to apply the frameworks in practice, and share the results with the authors or each other, to allow for further improvements and an even better match between theory and practice.
Conclusion
While on the one hand, theories on how to design the Safe System continue to develop, putting these theories into practice in any Safe System implementation is another matter. Implementing the Safe System is first and foremost a matter of cooperation between professionals, adapting the theory to the practical limitations. The process of cooperating with each other and learning from each other has shown us that there is still room for improvement in aligning methods to provide support in Safe System implementation and, especially, the choice of terminology of such methods (e.g. defining maturity).
These three frameworks aim to help countries, regions, cities, or organizations to learn from those more advanced in Safe System implementation, such as the Netherlands and Sweden, who started on this journey decades ago. ITF reports (ITF, 2008; ITF, 2016; ITF, 2022) show an increasing tendency to bring together practical experiences and options for policy action on the one hand, and common principles of the Safe System approach on the other hand. When most effective, this maturation occurs simultaneously across Management, Organisational Culture, and Policy and Practice. The frameworks are presented in this communication to promote their existence and to encourage their use amongst the global road safety community. The authors of this paper invite readers to assess the future application of these frameworks in order to validate and further improve them.
The three frameworks presented all provide pointers to those seeking to mature as to how they can do so, exploring the same problem from different perspectives. The ADB Framework emphasizes the role of management in committing to Safe System implementation, whilst the Agilysis Framework explores how the prevalent organisational safety culture can influence the likelihood to adopt the required roles, actions, and policies to enact Safe System thinking. The ITF approach is practical in nature and can assist with Safe System implementation by all actors across the system. Through comparing and analysing these frameworks, it has become clear that there are multiple and complementary routes to Safe System maturity. Collectively, these tools can provide a holistic view of a jurisdiction's or organization's maturity and therefore, can (and perhaps should) be used in combination.
Declaration of competing interests
The authors report no competing interests.
Funding
The contribution of SWOV to the development of the ITF Safe System model was based on the funding of the Ministry of Infrastructure and Water Management for the research and knowledge dissemination programme of SWOV. The contribution of VTI to the development of the ITF Safe System model was funded by VTI.
The Agilysis' model was developed using funding from National Highways in England. The research methodology and results were not directed by the funding organisation.
The Asian Development Bank model was funded by the Asian Development Bank.
CRediT contribution statement
Henk Stipdonk: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Letty Aarts: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Dan Campsall: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Laurent Carnis: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Veronique Feypell: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Tanya Fosdick: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
David Shelton: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Martin Small: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.
Anna Vadeby: Conceptualization, Methodology, Visualization, Writing—original draft, Writing—review & editing.