Reassesssment of the response to TSB Recommendation A16-14

 Recommandation A16-14
in PDF [212 KB]

Oversight of commercial aviation in Canada: Policies, procedures and training

Background

On 31 May 2013, at approximately 0011 Eastern Daylight Time, the Sikorsky S-76A helicopter(registration C-GIMY, serial number 760055), operated as Lifeflight 8, departed at night from Runway 06 at the Moosonee Airport, Ontario, on a visual flight rules flight to the Attawapiskat Airport, Ontario, with 2 pilots and 2 paramedics on board. As the helicopter climbed through 300 feet above the ground toward its planned cruising altitude of 1000 feet above sea level, the pilot flying commenced a left-hand turn toward the Attawapiskat Airport, approximately 119 nautical miles to the northwest of the Moosonee Airport. Twenty-three seconds later, the helicopter impacted trees and then struck the ground in anarea of dense bush and swampy terrain. The aircraft was destroyed by impactforces and the ensuing post-crash fire. The helicopter’s satellite trackingsystem reported a takeoff message and then went inactive. The search-and-rescue satellite system did not detect a signal from the emergency locator transmitter. At approximately 0543, a search-and-rescue aircraft located the crash site approximately 1 nautical mile northeast of Runway 06, and deployed search-and-rescue technicians. However, there were no survivors.

The Board concluded its investigation and released report A13H0001 on 15 June 2016.

TSB Recommendation A16-14 (June 2016)

This, and other investigations have highlighted the need for TC to adapt its approach to regulatory oversight to the competence of the operator. The documentation provided to TC inspectors evolved considerably in the time period covered by this investigation, and TC continues to provide new training to its inspectors as outlined in the Safety action taken section of the report.

However, recent investigations have highlighted the fact that, when faced with an operator that is unable or unwilling to address identified safety deficiencies, TC has difficulty adapting its approach to ensure that deficiencies are effectively identified and that they are addressed in a timely manner.

TC’s risk-based approach to surveillance planning resulted in the operators, which were all viewed as higher risk, being scheduled for more frequent surveillance. However,in A13W0120, unsafe conditions remained unidentified when the surveillance remained focused on processes. In other occurrences, unsafe conditions were allowed to persist for an extended period while TC relied heavily on a CAP process, in which the operators were ill-equipped to participate.

Therefore, to ensure that companies use their SMS effectively, and to ensure that companies continue operating in compliance with regulations, the Board recommended that

The Department of Transport enhance its oversight policies, procedures and training to ensure the frequency and focus of surveillance, as well as post-surveillance oversight activities, including enforcement, are commensurate with the capability of the operator to effectively manage risk.
TSB Recommendation A16-14

Transport Canada’s response to Recommendation A16-14 (September 2016)

Transport Canada agrees that it must continually enhance its oversight policies, procedures and training. In fact, independent to this accident or this recommendation, TC committed in its National Oversight Plan for 2016/2017 to conduct an evaluation of its surveillance program.

Further to that commitment, in July 2016, TC launched a Civil Aviation Surveillance Program Evaluation and Update Project. The purpose of this project is to analyse opportunities for improvement identified via inspector feedback, analyse lessons-learned, and leverage business intelligence to update the surveillance program. The goal of the project is to ensure the surveillance program effectively verifies regulatory compliance at appropriate intervals and is effective in carrying out enforcement action, as required. While the project is scheduled to complete in December 2017, the department will not hesitate to implement enhancements sooner, if warranted.

The Board’s recommendation A16-14 will beused as an input to this evaluation work.

Board assessment of Transport Canada’s response to Recommendation A16-14 (December 2016)

In its response, Transport Canada indicated that it launched a Civil Aviation Surveillance Program Evaluation and Update Program that is scheduled to be completed in December 2017. The Board is encouraged that Transport Canada has committed to evaluating its surveillance program, and to considering opportunities for further improvements in order to ensure the effectiveness of its surveillance program.

In a recent briefing to the Board, TC has provided a detailed update on the various program improvement initiatives undertaken since 2015-16. The Board is pleased to note that TC has implemented some concrete actions such as: the establishment of a National Oversight Office, the implementation of an Oversight Advisory Board, thecreation of a dedicated team working on surveillance policies and procedures, strengthened oversight planning, risk-based decision making, timely enforcement actions, and taking temporary measures that will permit an increase in the number of inspections on higher risk areas while the program evaluation andupdate is being done.

The Board also acknowledges TC’s efforts as it strives to find the right balance between planned and reactive oversight activities, as well as in the use of the various types of oversight tools available. Although TC has implemented numerous improvements it is too early to assess whether or not TC’s actions will adequately address the safety deficiency associated with this recommendation.

Therefore, the response to the recommendation is considered to be Satisfactory Intent.

Transport Canada’s response to Recommendation A16-14 (June2018)

TC agrees with the recommendation.

In July 2016, TC launched a Civil Aviation Surveillance Program Evaluation and Update Project to analyse improvement opportunities identified via inspector feedback and lessons-learned to update the surveillance program.

The objectives are to:

  • ensure the surveillance program effectively verifies regulatory compliance with safety management systems requirements;  
  • confirm that verifications are done at appropriate intervals; and, 
  • verify that enforcement action is taken as required. 

Since launching the project, TC has briefed the Chair and members of the TSB on progress.

While the evaluation of the surveillance program continues, TC has already taken action and has:

  • Introduced leaner, more efficient tools that effectively determine compliance and risk and provide a better balance between system and process level surveillance (e.g., anunannounced inspection at the ramp of the airports used to determine an air operator’s method for complying with regulatory requirements, such as flightcrew certification, instrument and equipment requirements, weight and balance procedure, dispatch procedures and an overall evaluation of the operations being conducted. Ramp inspections may be conducted at point of origin, en route or destination stops). 
  • Introduced Safety Management System verifications as part of inspection tools, other than assessments, which will facilitate more regular checks of enterprises’ SMS to confirm they are working.
  • Made extensive use of data interms of examining performance levels and sampling methodologies to determine surveillance intervals.

TC’s surveillance plan for 2018/2019 covers private operators (604), aerial work (702), heliports (305), flight training units and the general aviation community in addition to the higher impact sectors, such as airlines (705), commuters (704) and their associated Approved Maintenance Organizations.

In April 2018, TC also introduced new guidance to inspectors for post-surveillance work, including the documentation of findings and enforcement procedures. The intent of these updates is to strengthen TC’s approach to documenting findings of non-compliance and to better articulate the issue identified through inspection.  This will support companies (that are in receipt of findings) with their development and implementation of more effective corrective action plans. This new guidance will allow unsafe operating practices to be addressed more quickly and effectively.  In cases where certificate holders do not develop Corrective Action Plans, TC will suspend the relevant Canadian Aviation Document.

Board reassessment of Transport Canada’s response to Recommendation A16-14 (September 2018)

In its response, Transport Canada indicated that its Civil Aviation Surveillance Program Evaluation and Update Project is currently on-going. While the evaluation of the surveillance program continues, TC has introduced leaner, more efficient tools to determine compliance and risk, has introduced safety management system (SMS) verifications, has made extensive use of data to determine appropriate surveillance intervals, and has introduced new guidance to inspectors for post-surveillance work.

The Board is encouraged by these efforts. However, the Board notes that TC’s response makes no mention of air taxi (703) operations as part of its surveillance plan for 2018/2019, and it does not clearly describe what, if any, changes will be made to enhance existing oversight policies, procedures, and inspector training. Although TC’s actions to date may reduce the risk to the travelling public, the risk will not be substantially reduced or eliminated until its oversight policies, procedures, and training have been enhanced.

Therefore, the response to Recommendation A16-14 is assessed as Satisfactory in Part.

Transport Canada’s response to Recommendation A16-14(October 2019)

Since 2016, TC has implemented a number of changes to the surveillance program in order to improve oversight of the aviation industry. Among other enhancements, TC has implemented guidance, tools and training to improve the quality of findings, drive decision-making, and improve the risk-based planning methodology to ensure effective oversight.

Developing accurate, comprehensive, and clear findings are an essential function that supports enforcement action where required. Findings are critical communication tools that enable companies to understand and address the identified non-compliances. Further, robust and well supported findings are a necessary foundation upon which all further enforcement action relies, should an operator prove unable or unwilling to rectify identified issues.

SI SUR 029 was developed to provide inspectors with the enhanced guidance and tools to:

  • accurately determine the existence of a non-compliance;
  • determine what evidence is required to support the finding of non-compliance;
  • clearly describe the nature of the non-compliance; and,
  • identify whether the finding should be at the system level or process level.

It is critical that all of these elementsare in place in order to allow for an effective enforcement response. In addition to the publication of SI SUR 029, Surveillance Update training was developedand delivered throughout 2018, which included instruction related to SI SUR029.

The Oversight Advisory Board (OAB) was established to support efficient and effective decision-making when handling complex oversight decisionsFootnote 1.The process was developed in order to bring decision-makers and subject matter experts together to review concerns and determine appropriate courses of action in relation to complex oversight issues.

The OAB brings several benefits, including:

  • circumventing hierarchical processes for a rapid response;
  • supporting nationally consistent decision-making; and
  • facilitation of information-sharing between operational groups.

Fiscal year 2020-2021 is the third year of transition during which TC is enhancing its surveillance program. The planning methodology relies on the following fundamental concepts: operational latitude, using local intelligence, leveraging inspectors’ professional judgment, and data-driven risk-based decision-making.

SI SUR 028Footnote 2 was updated in October 2018 for fiscal year 2019-2020 and is currently being amended in preparation for 2020-2021 planning. The document details enhancements to the risk-based planning process that have been implemented with a view to continuous improvement.

Changes to the surveillance planning methodology for 2020-2021 includeFootnote 3:

  1. The introduction of a “conceptual National Oversight Plan” which is developed by headquarters, and further refined by the regions based on local intelligence.
  2. An enhanced data analysis tool that will assist in the planning of surveillance activities. This tool uses data obtained from the National Aviation Company Information System (NACIS), the Civil Aviation Daily Occurrence Reporting System (CADORS), as well as qualitative data from the regions to group enterprises into peer groups based on risk level.
  3. The introduction of aqualitative assessment tool to validate the categorization of enterprises using the quantitative analysis using the primary inspector’s perspectives.
  4. Continued use of process inspections as the primary surveillance tool for planned surveillance activities. Systems level surveillance will be conducted on a reactive basis where warranted.

TC’s surveillance plan for 2019-2020 includes surveillance for the following sectors: commercial airlines (705), commuter airlines (704) and their associated Approved Maintenance Organizations, as well as private operators (604), aerial work (702),air taxi (703), heliports (305), flight training units, and the general aviation community.

TSB reassessment of Transport Canada’s response to Recommendation A16-14 (March 2020)

The Board is encouraged by Transport Canada’s(TC) efforts to improve oversight of the aviation industry. In particular, TC has indicated in its response that it has implemented guidance, tools, and training to improve:

  • the quality of findings produced during surveillance activities;
  • the decisions made related to the oversight of commercial aviation; and
  • the risk-based planning methodology.

The necessary guidance and tools can be found in the following documents:

  • Staff Instruction (SI)SUR-027 — Oversight Advisory Board (OAB), published in October 2017;
  • SI SUR-029 — Writing Findings for Non-Compliance, published in April 2018;
  • SISUR-001 — Surveillance Procedures, updated in January 2019; and
  • SI SUR-028 — Surveillance Planning Instructions — Fiscal Year 2019-2020, updated in January 2019.

TC’s emphasis on improving the quality offindings will assist in the conduct of surveillance activities and in post-surveillance decision-making, which may include enforcement action decisions, at the regional and/or national level. TC’s response also highlighted the benefits of the OAB which, when used, can assist with the post-surveillance decision-making process.

TC has provided a general overview of its recent enhancements to its surveillance planning methodology. Specifically, TC refers to operational latitude, local intelligence, and inspectors’professional judgement, which will be used in conjunction with an enhanced data analysis tool and a qualitative assessment tool. In light of the infancy of these changes, it is not possible to fully assess any potential impact that they will have on TC’s oversight program.

TC now relies on process inspections as the primary surveillance tool for planned surveillance, where as systems level surveillance is done on a reactive basis, when warranted. There are benefits to an increased focus on regulatory compliance, which may be easier to accomplish through process inspections; however, simply verifying regulatory compliance in a cross-section of an organization does not guarantee that commercial aviation operators are capable of effectively managing safety within their organization. TC must also confirm that operators are capable of managing safety risks effectively, whether or not they have a safety management system. Therefore, TC’s increased reliance on process inspections represents a reduced surveillance focus, which will have to be assessed over time.

It is likely thatTC’s planned actions and actions taken to date will reduce the risks associated with the safety deficiency identified in Recommendation A16-14. However, withthe information TC provided, it is not possible to determine at this point if these actions will substantially reduce or eliminate the safety deficiency. That determination will be made once TC’s changes have been fully implemented.

Therefore, the Board considers the response to the recommendation to be Satisfactory in Part.

Transport Canada’s response to Recommendation A16-14 (September 2020)

TC agrees with this recommendation.

Following the original commitments related to TSB recommendation A16-14, TC completed the Surveillance Program Evaluation and Update project in 2018. Since then the civil aviation program has continued through the normal continuous improvement cycle as evidenced by the publication of multiple revisions of staff instructions (SI) defining the surveillance program:

  • SI-SUR-001 Issue 8 – Surveillance ProceduresFootnote 4
  • SI-SUR-029 Issue 2 – Writing Findings of Non-ComplianceFootnote 5
  • SI-SUR-028 Issue 5 – Surveillance Planning Instructions – Fiscal Year 2020-2021Footnote 6

The training program launched to explain to the inspectorate the changes resulting from the Surveillance Program Evaluation and Update project concluded in the spring of 2020. The vast majority of the Civil Aviation Safety Inspectors, Technical Team Leads and Associate Directors of Operations participated in the two-day scenario based training that was offered in small groups across Canada to facilitate open discussion about the multiple aspect of their surveillance related duties, including: data gathering for risk-based planning, pre-site, on site and post site activities, enforcement, writing and documenting findings of compliance, quality control and continuous improvement and operational latitude concepts. Recurrent surveillance training is to be included to the list of mandatory training at the next revision of CAD-ADM-05: Required Training for Civil Aviation Employees who Develop, Deliver and/or Support Oversight Activities.Footnote 7

The planning exercise for fiscal year 2019-2020 and fiscal year 2020-2021 was done with the support of local intelligence gathered from inspectors directly involved with the enterprises as well as the risk indicators based on data available from central databases. This methodology enabled TC produce a risk-based surveillance plan that reflects the preoccupations of the inspectorate as well as the interpretation of the risk data available to TC as a whole.

The recent COVID-19 pandemic has proven to be an opportunity to leverage the Targeted Inspection (TI) tool that was introduced as a result of the Surveillance Program Evaluation and Update Project. Being a flexible and nimble tool, the TI methodology enabled TC to assess the impact of the pandemic on Canadian certificate holders and to capture emerging safety risks in a changing landscape. More specifically, the purpose of the COVID-19 TI was to:

  • Assess certificate holders current levels of risk given the changing landscape of aviation operations;
  • Feed data-driven decision-making in order to prioritize surveillance for certificate holders who demonstrate increased levels of risk, and;
  • Actively monitor and respond to the changes imposed during the COVID-19 pandemic.

In a matter of 3-4 weeks from start to finish, the inspection was designed, planned, delivered and the results were analyzed.Footnote 8 The vast majority of the 2166 operating certificate holders contacted were found to represent very low to low risk. These results are consistent across regions, with inspectors providing only 3 recommendations to increase monitoring and 9 recommendations to initiate a reactive surveillance activity as a result of higher risk.

The COVID 19 pandemic is also driving further improvements to the surveillance program due to the restrictions imposed on travel and physical distancing requirements. Remote surveillance and new surveillance tools aiming directly at regulatory compliance verifications are being considered and piloted to limit on-site presence while maintaining an adequate level of oversight.

It is important to point out that this is not a departure for the system-level approach to surveillance. The vision is to limit on-site presence to conduct necessary compliance verification. The documentation review, documentation sampling, interviews and analysis work required tosupport the system level surveillance will continue in a remote (virtual) environment.

TC continues to rely on the development of its Quality Assurance system to support further improvement of its surveillance system. Following the conclusion of the Surveillance Program Evaluation and Update project and several other Transformation Initiatives, TC is committed to invest the resources necessary to ensure that the surveillance program remains effective at mitigating risks associated with Civil Aviation.

With respect to the enforcement component of the surveillance program, TC has recently moved to a multimodal enforcement model. Civil aviation is working actively with the newly created Center for Enforcement Expertise to develop procedures and guidance to ensure effective enforcement process.

Finally, TC has launched the Civil Aviation Oversight Application Rationalization project aiming at reducing the number ofoversight related databases and systems while improving workflow, interconnectivity, analysis and reporting capability. This long term project (estimated at 5 years+) is expected to dramatically improve Civil Aviation’s risk-based planning and risk analysis capability once fully implemented.

TSB reassessment of Transport Canada’s response to Recommendation A16-14 (March 2021)

The Board is pleased that Transport Canada (TC) agrees with this recommendation and is taking steps to improve its surveillance program. In particular, the Board acknowledges that TC has undertaken the following activities:

  • TC has made multiple revisions to staff instructions defining the surveillance program, including:
  • SI-SUR-001 Issue 8 – Surveillance Procedures
  • SI-SUR-029 Issue 2 – Writing Findings of Non-Compliance
  • SI-SUR-028 Issue 5 – Surveillance Planning Instructions – Fiscal Year 2020-2021
  • TC provided 2-day scenario-based training to its personnel to explain recent changes to TC’s surveillance program.
  • TC introduced the Targeted Inspection (TI) tool to allow for a flexible and nimble approach to oversight during the COVID-19 pandemic. Using this tool, TC determined that the vast majority of operators represent a very low to low risk.
  • TC is exploring remote surveillance and new surveillance tools intended to verify regulatory compliance.
  • TC continues developing its quality assurance system to support further improvements to its surveillance system.
  • TC has recently moved to a multimodal enforcement model and is developing procedures and guidance to ensure an effective enforcement process.
  • TC has launched the Civil Aviation Oversight Application Rationalization project to reduce oversight-related databases and systems while improving efficiency.

It is likely that TC’s planned actions and actions taken to date will reduce the risks associated with the safety deficiency identified in Recommendation A16-14. However, in light of past instances where non-compliances have gone undetected, or have been allowed to persist, the Board is unable to assess the validity of the risk assessments made using the new TI tool. Furthermore, TC has not provided a clear framework for how it plans to ensure that the frequency and focus of surveillance activities are commensurate with the capability of the operators to effectively manage risk or how moving to a multimodal enforcement model will result in more effective enforcement of commercial aviation.

While the Board recognizes these positive steps, with the information provided, it is not possible to determine at this point if these actions will substantially reduce or eliminate the safety deficiency.

Therefore, the response to Recommendation A16-14 is assessed as Satisfactory in Part.

Next TSB action

The TSB will continue to monitor the progress of TC's actions to mitigate the risks associated with the safety deficiency identified in Recommendation A16-14, and will reassess the deficiency on an annual basis or when otherwise warranted.

This deficiency file is Active.

Date modified: