Auditor NL 5/12
IS-BAO NEWSLETTER | May 2012
As of 1 May 2012 there were 646 registrants:
Noteworthy: One-third of registrants now have productive SMSs, i.e. they have progressed beyond the formative stage of the program.
This is an update to the 2010 version of this policy. The new helicopter auditor designation requirements are described.
The policy requires auditors:
This was developed to clarify the SMS ongoing performance requirements for IS-BAO registration, and to provide for provisional operator registration under certain circumstances. Specifically, the policy covers issues of failure to progress to or maintain higher stage levels of registration.
|Knowledge of Current IS-BAO Standards
IS-BAO standards and recommended practices are being constantly reviewed to better remain current with ICAO standards and industry best practices. Each January scores of changes are published in a completely revised edition of the standard and interim changes are promulgated throughout the year via policy statements like those above.
|Stage 2/3 Comments
Most auditors are providing better comments for advanced stage audits, especially SMS comments, but some reports and protocols continue to provide insufficient information to provide meaningful feedback to operators and to convince the audit review group of an operator’s readiness for an advanced stage designation. This is important – in case you missed a previous article on this, see Making a Case for Stage 2 and 3 Audits.
Just because it is an advanced stage audit doesn’t mean that we don’t want comments throughout the protocols, either. What’s new, changed, different, better, worse…? Granted, the focus of these audits is SMS performance and maturity but sampling of all aspects of the operation is required; tell us something about what you observe during the sampling process.
|Let’s Talk About Findings…
“…findings are specific instances of non-conformity with the requirements of IS-BAO, operator manuals or operator directives. The findings may relate to one or more minor non-conformities or major non-conformity, but should be of such significance as to warrant urgent action (APM 6.1).”
Some auditors continue to cite individual non-conformities as findings when these items are really evidence of a larger issue that may be properly designated a finding.
For example, non-conformities such as, “4.2.3 – PIC duties not fully described”, “5.3.1 – tems d, h and j of emergency procedures training not mentioned” and “6.4 Weather Minimums – 6.4.2, takeoff weather minimums not specified” are all non-conformities but not necessarily findings when considered individually. Rather, they may constitute a pattern of missing items that could be considered a finding under standard 10.1, requiring an operator to provide an operations manual containing the previously mentioned items. On a larger scale, management’s commitment and responsibilities may be involved if a larger pattern of missed requirements is evident – see standard 3.2.1.
Do all non-conformities have to be turned into findings? Not really: APM 6.2.3 provides guidance: “If it is determined that one or more minor non-conformities are simply slips, lapses or omissions that can easily be corrected and there is no significant underlying problem, rather than making a Finding the non-conformity should be noted in the Analysis of Non-conformities section of the Audit Protocol along with the comment, ‘Action Required’.”
|3.2.3a – Safety Performance Monitoring and Measurement
“The organization shall develop and maintain the means to verify the safety performance … shall be verified in reference to the safety performance indicators and safety performance targets of the SMS.”The operator should generate a set of measurement tools to verify performance. The secret to this one is to set measurable targets/goals. Unfortunately, many SMS goals contain few, if any, means of quantifying progress toward goals. For example, a safety goal to reduce overall risk to its lowest possible level is not measurable unless quantified. A more realistic safety goal may be to — reduce risk to its lowest possible level by:
This is based on and validated by the management axiom: if you can’t measure it, you can’t manage it.
The majority of audits received are now are for the advanced stages. These require evidence of performance, not just having a system in place designed to prepare an operator for conformance. A number of audits describe varying levels of performance but also describe relatively high levels of basic oversights and non-conformities that should have been accomplished at stage 1.If an operator has been operating under IS-BAO for two years yet all personnel have not attended required training courses, safety communication is lacking, SOP are not universally used and/or key standards and limitations are missing from the operations manual, it is hard to believe that safety management is actively targeted and safety risks are being effectively managed (APM 5.4.4). Taken alone these non-conformities are bad enough but together they indicate a pattern that indicates a lack of SMS conformance and effectiveness, and are therefore a significant finding. See APM 6.3.2 and 6.3.3 for additional information.
No one wants to tell an operator they don’t qualify for the next stage, but not doing so may permit a false sense of well-being to continue within the operator’s organization.
Advanced stage audits concentrate on SMS performance but not to the exclusion of the remainder of the IS-BAO standards. The percentage guidelines for emphasis on program effectiveness shown in APM 5.4 are not intended to exclude the necessity of ensuring conformance to all standards. APM 4.2.3 provides guidance in this issue. Checking the program basics, especially performance of essential items, is a required subset of the audit designed to ensure the program fundamentals leading to SMS conformance are indeed in place and working correctly.
|Where do I Put Comments?
At the end of each 2012 protocol chapter the term “Analysis of Non-conformities” has been substituted for the old “Summary of Observations” header. This was done to encourage auditors to do just what the new header says: analyze the evidence, determine severity and justify the finding with objective evidence, as appropriate. Comments and remarks should be placed in the right-hand column of the form. However, if you have no findings to analyze, feel free to use the new analysis section for summary comments for the chapter but preface your remarks with the term “Comments:”
|Bits and Pieces
The following open-book questions are designed to keep you current on the standards and audit procedures. Don’t send us the answers; this is just for you:
1. What items should be covered in the closing meeting?
2. When must a pilot advise the operator of the aircraft’s departure and arrival times?
3. How does an operator know whether its fatigue management system is effective?
4. Is an operator required to explain why safety actions are taken and why safety procedures are changed?
5. If hazard reports are not being used to improve company procedures, training, etc., is this considered to be a critical deficiency for a stage 2 audit?
To confirm your expertise, see the bottom of the page for the references.
Be a discerning and insightful auditor,
Have you reviewed APM 5.8.2 recently?
|Answers: 1) APM 4.11.4 2) Standard 6.3.1 3) Standard 6.13.1d 4) Standard 3.2.4b 5) Standard 6.3.2b|