Reassesssment of the response to TSB recommendation A16-13
in PDF [1797 KB]
Oversight of commercial aviation in Canada: SMS assessments
On 31 May 2013, at approximately 0011 Eastern Daylight Time, the Sikorsky S-76A helicopter (registration C-GIMY, serial number 760055), operated as Lifeflight 8, departed at night from Runway 06 at the Moosonee Airport, Ontario, on a visual flight rules flight to the Attawapiskat Airport, Ontario, with 2 pilots and 2 paramedics on board. As the helicopter climbed through 300 feet above the ground toward its planned cruising altitude of 1000 feet above sea level, the pilot flying commenced a left-hand turn toward the Attawapiskat Airport, approximately 119 nautical miles to the northwest of the Moosonee Airport. Twenty-three seconds later, the helicopter impacted trees and then struck the ground in an area of dense bush and swampy terrain. The aircraft was destroyed by impact forces and the ensuing post-crash fire. The helicopter’s satellite tracking system reported a takeoff message and then went inactive. The search-and-rescue satellite system did not detect a signal from the emergency locator transmitter. At approximately 0543, a search-and-rescue aircraft located the crash site approximately 1 nautical mile northeast of Runway 06, and deployed search-and-rescue technicians. However, there were no survivors.
The Board concluded its investigation and released report A13H0001 on 15 June 2016.
TSB Recommendation A16-13 (June 2016)
Transportation companies have a responsibility to manage safety risks in their operations. Compliance with regulations can only provide a baseline level of safety for all operators in a given sector. Since regulatory requirements cannot address all risks associated with a specific operation, companies need to be able to identify and address the hazards specific to their operation.
In the traditional oversight model, companies are not required to have formalized systems in place to continuously manage safety at a systems level. Oversight is accomplished using an inspect-and-fix approach. In this approach, the regulator's role is focused on finding instances of regulatory non-compliance, which the operator must correct. The impact on safety of this approach is limited for 2 main reasons.
First, it is not possible for the regulator to examine continuously all aspects of an operation. Challenges in detecting non-conformances have been identified in a number of previous TSB investigations (e.g. A12W0031, A12C0154, and A13W0120). For example, in A13W0120, although TC had assessed the operator involved as high risk, and was conducting frequent surveillance activities, the focus had not shifted to verifying regulatory compliance, and remained at a systems level. As a result, TC's surveillance did not identify the unsafe operating practices that contributed to the severity of the occurrence.
Secondly, if the systemic causes of non-conformance are not identified and addressed, it is likely that unsafe conditions will persist. The TSB has also previously identified this pattern in a number of investigations (e.g. A10Q0098, A10Q0117, and A13H0002). For example, in A13H0002, the report identified weaknesses in the oversight of an operator with a transitioning safety management system (SMS). In this case, the operator experienced difficulty producing acceptable CAPs and meeting its proposed implementation time frames. This resulted in repeated delays in addressing deficiencies. During this period, TC postponed any additional surveillance activities pending CAP implementation. The suspension of surveillance activities while waiting for the CAP process to run its course effectively reduced the frequency of oversight for an operator that was considered high risk and left the operator with less than the planned level of oversight for an extended period.
When implemented properly, SMS provide a framework for companies to effectively manage risk and make operations safer. Regulatory requirements for companies to implement SMS are the first step in ensuring that all operators are capable of meeting their safety responsibility. It is for this reason that the TSB has echoed calls from ICAO and the worldwide civil aviation industry emphasizing the advantages of SMS.
Still, even with SMS requirements, companies will vary in degrees of ability or commitment to effectively manage risk. Less frequent surveillance, focused on an operator's safety management processes, will be sufficient for some companies. However, the regulator must be able to vary the type, frequency, and focus of its surveillance activities to provide effective oversight to companies that are unwilling or unable to meet regulatory requirements or effectively manage risk. Further, the regulator must be able to take appropriate enforcement action in these cases.
Operators with a mature, effective SMS, along with a corresponding safety culture and abilities, may be the subject of less-frequent, systems-level oversight. In contrast, companies that have not demonstrated the capability to effectively manage risks at a systems level should be subject to more frequent surveillance, with a greater emphasis on ensuring compliance with regulations. As an operator's systems mature and become more effective, the frequency of oversight may be reduced and the balance of oversight can shift from the compliance-based model to more systems-level surveillance activities.
In the investigation of the Ornge RW accident at Moosonee, Ontario, the TSB found that TC's approach to surveillance activities did not lead to the timely rectification of non-conformances. It also found that TC inspectors believed that tools other than a corrective action plan to guide the operator back into compliance were either unavailable or inappropriate for use with a willing operator. As a result, the operator's willingness to address surveillance findings superseded concerns about the operator's capability to address the deficiencies in post-surveillance decision making. In addition, the investigation found that the training and guidance that was provided to TC inspectors contributed to uncertainty, which led to inconsistent and ineffective surveillance of Ornge. Ultimately, although TC was conducting frequent and detailed surveillance, the approach to returning the operator to a state of compliance was not well matched to the capabilities of the operator.
The investigation also noted that although TC was relying heavily on the CAP process, the operator was not required to have an SMS and, as a result, had not demonstrated to TC that it had the processes in place to effectively manage safety.
The TSB has previously identified these issues; safety management and oversight is a multi-modal item on the TSB Watchlist, which identifies those issues posing the greatest risk to Canada's transportation system. The Watchlist proposes the following solutions in this area:
- TC must implement regulations requiring all air operators to have formal safety management processes, and TC must oversee these processes.
- Companies that do have SMS must, in turn, demonstrate that it is working—that hazards are being identified and effective risk mitigation measures are being implemented.
- Finally, when companies are unable to effectively manage safety, TC must not only intervene, but do so in a manner that succeeds in changing unsafe operating practices.
The investigation into this accident and other recent occurrences emphasize the need for operators to be able to effectively manage safety. More than 10 years after introducing the first SMS regulations for airline operators and the companies that perform maintenance on their aircraft, SMS implementation has stalled. While many companies, such as Ornge RW, have recognized the benefits of SMS and voluntarily begun implementing it within their organizations, approximately 90% of all Canadian aviation certificate holders are still not required by regulation to have an SMS. As a result, TC does not have assurance that these operators are able to effectively manage safety.
Therefore, the Board recommended that
The Department of Transport conduct regular SMS assessments to evaluate the capability of operators to effectively manage safety.
TSB Recommendation A16-13
Transport Canada's response to Recommendation A16-13 (September 2016)
Transport Canada agrees in principle with this recommendation.
The suite of surveillance tools that TC uses, namely assessments, program validation inspections and process inspections, are effective in verifying compliance with the Canadian Aviation Regulations, including SMS requirements.
TC has adopted a systems-based approach to all its surveillance activities. In other words TC has added management system principles and quality assurance techniques to surveillance activities across all sectors not just those subject to safety management system requirements. Under the systems-based approach, TC’s civil aviation safety inspectors use the systems as an entry point and sample end products to verify that the system is working effectively and is in compliance with the Canadian Aviation Regulations.
While TC continually evaluates its tools to ensure they continue to be effective and makes updates, as required, the department is confident in its approach of using a combination of surveillance tools to verify regulatory compliance.
TSB assessment of Transport Canada’s response to Recommendation A16-13 (December 2016)
In its response, Transport Canada indicated that it adopted a systems-based approach to all its surveillance activities and that it is using a suite of surveillance tools to verify compliance with the Canadian Aviation Regulations, including SMS requirements. TC also indicated that it is confident in its systems-based approach to verifying regulatory compliance. The Board recognizes that TC has undertaken a number of change initiatives aimed at improving its oversight program, and that some progress has been made.
In a recent briefing to the Board, TC has also reiterated its commitment to verifying regulatory compliance at appropriate intervals and effectively carrying out enforcement, as required. The Board is encouraged by the concrete enforcement actions recently taken by TC on issues that were identified through its surveillance activities. However, TC’s response does not fully address the underlying safety deficiency that led to this recommendation. Achieving minimum regulatory compliance does not necessarily guarantee that all commercial aviation operators are capable of effectively managing safety within their organization. TC must also confirm that operators have a mature, effective SMS and are managing safety risks effectively.
The Board takes note that TC has recently undertaken a Program Evaluation and Update Project aimed at taking stock of the various transformation and improvement initiatives implemented to date. Once completed, this evaluation project will assist TC in refining the various elements of its surveillance program, including regular SMS assessments of the capability of operators to effectively manage safety. Although the numerous actions taken by TC may address the risk associated with this safety deficiency, more work remains to be done:
Therefore, the response to the recommendation is considered to be Satisfactory in Part.
Transport Canada’s response to Recommendation A16-13 (June 2018)
TC agrees with the recommendation.
The Department’s Evaluation and Advisory Services (EAS) group is evaluating civil aviation related Safety Management Systems (SMS). The report detailing the outcomes of this evaluation will be completed in summer 2019. As part of this evaluation, the EAS has conducted a stakeholder survey in 2017. They will also lead case studies on seven airlines and interview stakeholders and TC staff with SMS requirements.
The Department has also published a variety of guidance materials regarding SMS. For example, in 2016, the Department published information aimed at smaller sized-operations, such as private operators who are also subject to SMS requirements. Further, through a quality assurance review in consultations with industry, TC published material in 2017 outlining the establishment and maintenance of Quality Assurance Programs that comply with the CARs.
TSB reassessment of Transport Canada’s response to Recommendation A16-13 (September 2018)
In its response, Transport Canada (TC) indicated that it has taken actions to address the safety deficiency identified in Recommendation A16-13. These actions include the following:
- In 2016, TC published safety management system (SMS) guidance material, including information aimed at smaller-sized organizations;
- In 2017, TC published material on the establishment and maintenance of quality assurance programs that comply with the Canadian Aviation Regulations; and
- In 2017, TC’s Evaluation and Advisory Services (EAS) group conducted a stakeholder survey as part of its evaluation of civil aviation SMS.
Additionally, TC expects to complete the EAS evaluation report in summer 2019.
The Board recognizes that TC continues to take steps to improve its oversight program; however, TC’s response does not address the need to conduct regular SMS assessments to evaluate the capability of operators to effectively manage safety. As a result, TC’s actions to date do not significantly reduce or eliminate the safety deficiency associated with Recommendation A16-13.
Therefore, the response to Recommendation A16-13 is assessed as Satisfactory in Part.
Transport Canada’s response to Recommendation A16-13 (October 2019)
TC agrees in principle with this recommendation.
Following the Surveillance Program Evaluation and Update undertaken in 2016, several initiatives have been implemented to improve the way in which Transport Canada Civil Aviation (TCCA) evaluates the capability of operators to effectively manage safety and employs enforcement methods that [are] commensurate with the operator’s willingness and abilities.
TCCA has made adjustments to the surveillance program, and is using more efficient oversight tools to effectively determine compliance and risk. In addition, the risk based planning methodology has been updated and surveillance is being conducted on a broader range of operators at more frequent intervals. In particular:
- Systems level inspections are no longer the default surveillance methodology. Instead, process inspections are conducted which target specific areas of higher risk. Process inspections involve a combination of compliance verification and an evaluation of processes to ensure that results are consistent. The scope of the process inspections is selected to target areas of higher risk or areas of concern specific to individual operators. In the event that issues are identified, surveillance teams may broaden the scope of the inspection or recommend that reactive surveillance be initiated.
- Reactive surveillance may include compliance inspections, process inspections, Program Validation Inspections (PVIs), or Safety Management System (SMS) assessments. The type of inspection selected is based on which tool is most suitable to respond to the information or event that triggered it, and is dependent on the specific circumstances encountered.
- SMS Process Inspections worksheets are now available. These worksheets are designed to confirm that established Safety Management Systems processes within an enterprise are: being used, producing consistent results and that those results are meeting the objectives set by the CAR.
As a result of these changes, surveillance teams are better equipped to vary the type, frequency, and focus of surveillance activities to address the operator’s needs and provide effective oversight.
Guidance materials, inspector tools, and training has been amended to reflect surveillance program enhancements. This includes updates to SI SUR 001 — Surveillance ProceduresFootnote 1; SI SUR 028— Surveillance Planning Instructions - Fiscal Year 2019-2020Footnote 2; SI SUR 029— Writing Findings for Non-ComplianceFootnote 3; the development of new process inspection surveillance worksheets for all disciplines; and the development and delivery of Surveillance Update Training for all TCCA inspectors.
As of September 2019, 388 inspectors, managers and officers have received the surveillance update training. It is expected that the majority of inspectors will have received the training by the end of this fiscal year.
TSB reassessment of Transport Canada’s response to Recommendation A16-13 (March 2020)
The Board is pleased that Transport Canada (TC) continues to undertake efforts to improve its oversight program, in particular that guidance, tools and training have been improved to assist inspectors in their oversight duties. To date, TC has taken the following actions:
- In October 2017, published Staff Instruction (SI) SUR-027 — Oversight Advisory Board (OAB);
- In April 2018, published SI SUR-029 — Writing Findings for Non-Compliance;
- In January 2019, updated SI SUR-001 — Surveillance Procedures;
- In January 2019, updated SI SUR-028 — Surveillance Planning Instructions — Fiscal Year 2019-2020;
- Developed new process inspection worksheets; and
- Developed and delivered surveillance update training to Transport Canada Civil Aviation (TCCA) inspectors, managers and officers.
When safety management systems (SMS) were introduced in 2005, the original intent was to serve as an additional layer of oversight, helping ensure the safety of the travelling public. However, almost 15 years later, SMS is still not mandatory in all sectors of commercial aviation. Since assessments apply to SMS enterprises, companies must first be required to have an SMS in order for TC to conduct an assessment on that company. Therefore, the underlying safety deficiency associated with this recommendation is not being addressed for operators currently not required to have an SMS.
In its response, TC stated that SMS assessments have been replaced by process inspections as the default surveillance methodology. While there are benefits to an increased focus on regulatory compliance, which may be easier to accomplish through process inspections, this approach does not address the underlying safety deficiency associated with this recommendation. Simply verifying regulatory compliance and processes in a cross-section of an organization does not guarantee that commercial aviation operators are capable of effectively managing safety within their organization. Although TC has the ability to broaden the scope through reactive surveillance, there is a risk that problem areas may be overlooked by limited-scope surveillance activities. TC must also confirm that operators have a mature SMS that is capable of managing safety risks effectively.
Therefore, the Board considers the response to the recommendation to be Satisfactory in Part.
Transport Canada’s response to Recommendation A16-13 (September 2020)
TC agrees with this recommendation.
TC is currently conducting a review of SMS implementation with consideration for the future extension of the program in mind, as stated in the response to recommendation A16-12.
An SMS assessment is used as a reactive surveillance tool when safety intelligence gathered through planned surveillance or other means warrants it. TC continues to update and improve its surveillance methodology to enhance its capacity to evaluate the effectiveness of safety management systems as well as compliance with regulations:
- A new Process Inspection tool aimed at verifying the integrity of critical SMS processes within an enterprise, without having to resort to a full SMS Assessment, is now available;
- Improvements are being made in the field of safety data collection and analysis with the consolidation of the databases containing surveillance information and the introduction of business intelligence and data visualization technologies;
- A methodology to enable the conduct of remote surveillance activities is being developed in response to the challenges posed by the COVID-19 pandemic. It is expected remote surveillance will remain an option for the future and will further enhance TC’s capability to acquire the safety and compliance intelligence required to support its risk-based surveillance program.
TC actively monitors the effectiveness of its surveillance program and is committed to continuously improve its capability to monitor the safety and performance of all its certificate holders.
TSB reassessment of Transport Canada’s response to Recommendation A16-13 (March 2021)
In its most recent response, Transport Canada (TC) states that it is conducting a review of safety management system (SMS) implementation with consideration for extension of the program. In addition, TC’s response highlights the following:
- A new process inspection tool, which is designed to look at critical SMS processes without having to conduct a full SMS assessment, is now available;
- Improvements are being made in the way safety data is collected and analyzed; and
- A methodology for remote surveillance activities is being developed.
The Board is encouraged by TC’s efforts to update and improve its surveillance methodology; however, until TC demonstrates, through surveillance activity assessments, that the new surveillance methodology is identifying non-compliances, and that TC is ensuring that a company returns to compliance in a timely fashion and is able to manage the safety of its operations, the risks associated with the safety deficiency identified in Recommendation A16-13 remain.
Therefore, the response to Recommendation A16-13 is assessed as Satisfactory in Part.
Transport Canada’s response to Recommendation A16-13 (September 2021)
Transport Canada (TC) agrees with the recommendation.
Since the recommendation was issued in 2016, TC completed several actions and published a variety of guidance materials regarding Safety Management System (SMS) assessments. Namely, TC undertook a quality assurance review in consultations with industry and published material in 2017 outlining the establishment and maintenance of Quality Assurance Programs that comply with the Canadian Aviation Regulations (CARs) and published Staff Instruction (SI) SUR-027 — Oversight Advisory Board (OAB).Footnote 5
Guidance materials, inspector tools, and training have also been amended in 2018 and 2019 to reflect surveillance program enhancements which include updates to SI SUR-001 — Surveillance Procedures,Footnote 6 SI SUR-028 — Surveillance Planning Instructions — Fiscal Year 2019-2020Footnote 7 and SI SUR-029 — Writing Findings for Non-Compliance. Footnote 8
In our last update in September 2020, the Department committed to:
- Continue to update and improve its surveillance methodology to enhance its capacity to evaluate the effectiveness of safety management systems as well as compliance with regulations;
- Improve safety data collection and analysis with the consolidation of the databases containing surveillance information and the introduction of business intelligence and data visualization technologies; and,
- Develop a methodology to enable the conduct of remote surveillance activities in response to the challenges posed by the COVID-19 pandemic.
Since the previous update, TC reviewed the oversight tools within the surveillance methodology and is now developing new ones to more effectively communicate non-compliances to Canadian Aviation Directive (CAD) holders. To monitor implementation, TC formed the Finding Review Committee. This initiative performs quality assurance on 100% of findings entered by TC inspectors. Results have informed updates to SI-SUR-029 - Writing Findings for Non-Compliance,Footnote 9 for which a national consultation is planned for Fall/Winter 2021, emphasizing findings of clear violations of rules-of-conduct in order to build an aviation record and be able to demonstrate re-occurrence. Repeated contraventions can be evidence of systemic failure and are used to trigger system level surveillance, enforcement action, and/or certification action in accordance with SI-SUR-006 - Safety Management Systems - Civil Aviation Non-Compliance Event Review.Footnote 10
Also, TC is seeking to improve the effectiveness of SMS oversight as it acknowledges this is a current weakness in the program. TC is developing a proposal for effective safety oversight of management systems, which is currently undergoing internal review and consultation. This proposal includes consistent tools, processes and training for safety oversight of SMS, Quality Assurance Plan (QAP), and Fatigue Risk Management System (FRMS), with a proposed development timeframe over the next two years.
TSB reassessment of Transport Canada’s response to Recommendation A16-13 (March 2022)
Transport Canada (TC) agrees with the recommendation.
Since 2016, TC has amended guidance material, inspector tools, and training to reflect updates to surveillance procedures.
In its most recent response, TC indicated that it is:
- developing oversight tools to more effectively communicate non-compliances to Canadian Aviation Document (CAD) holders. This is being monitored by the Finding Review Committee, which reviews all findings entered by TC inspectors to ensure findings related to violations of rules-of-conduct are clearly documented. This will assist in building an aviation record and in identifying re-occurrence, which can be evidence of a systemic failure used to trigger system-level surveillance, enforcement action, and/or certificate action.
- seeking to improve the effectiveness of safety management system (SMS) oversight, which it acknowledges is a current weakness in the program. TC is developing a proposal for effective safety oversight of management systems, which is currently undergoing internal review and consultation. This initiative is planned to be completed over the next 2 years.
The Board is encouraged by TC’s efforts to update and improve its surveillance methodology; however, the Board is concerned that, as per TC’s surveillance planning and procedures, SMS assessments are only conducted as a reactive surveillance tool. According to TC’s Staff Instruction (SI) SUR-001 - Surveillance Procedures, assessments “will be conducted as a reactive surveillance activity only invoked by the Oversight Advisory Board (OAB).”Footnote 11 In addition to TC’s four categories of surveillance activities (system vs process level, targeted vs compliance level inspections), the Board continues to recommend the use of regular SMS assessments as part of TC’s planned (and reactive) surveillance activities to ensure that SMS are evaluated regularly and updated based on results of integrated and fulsome evaluations and to ensure that an operator has the ability to effectively manage safety. As TC indicated it will not be conducting regular SMS assessments, the risks associated with the safety deficiency identified in Recommendation A16-13 remain.
Therefore, the response to Recommendation A16-13 is assessed as Satisfactory in Part.
Next TSB action
The TSB will continue to monitor the progress of TC's planned actions to mitigate the risks associated with the safety deficiency identified in Recommendation A16-13, and will reassess the deficiency on an annual basis or when otherwise warranted.
This deficiency file is Active.