Best Behavior
The County Fire Authority of Victoria, Australia, is one of the world's largest volunteer-based emergency services organizations, with approximately 58,000 volunteer members who are supported by over 400 career firefighters and officers and more than 700 career support and administrative staff. CFA covers all of rural Victoria (except state forests and national parks) as well as the province's cities and towns. This equates to 2.5 million people and just under one million homes spread across 58,000 square miles.
For many years CFA has undertaken reviews of operational preparedness and response through operations analyses, debriefings and brigade inspections, but there has been limited coordination and consistency among these review tools. More recently, CFA has been the subject of a number of external reviews, including coronial inquiries into the January 1997 Dandenong Ranges Fire where three citizens died, the 1998 Linton Fire where five CFA firefighters died in an entrapment, and various government inquiries into the 2003 bushfires that burnt 2.7 million acres in Victoria and over 4 million acres in adjacent areas of New South Wales and Australian Capital Territory.
The outcomes of these internal and external reviews are usually a list of recommendations for changes in operational systems or procedures. In the past, actions to address these recommendations often were developed in an uncoordinated manner with no reference to similar findings from previous reviews, and monitoring the progress of implementing the actions varied in its effectiveness.
To remedy this, CFA is introducing an operations performance improvement program that incorporates an integrated suite of performance monitoring tools with consistent performance benchmarks and evaluation processes. CFA needed to take a staged approach to the development and implementation of the monitoring processes, as there were insufficient resources to develop all processes simultaneously. Given the workload already confronting CFA managers and firefighters, it would be impossible to implement each of the processes throughout CFA at the same time.
The first phase of the program was the review of the inspection and reporting process to monitor brigade efficiency, and the development of processes for regional and state preparedness monitoring and real-time performance monitoring.
The CFA's chief officer is required to “inspect or arrange for the inspection of all brigades and to report to the County Fire Authority on their state of efficiency or otherwise.” A revised process was introduced in 2004 that focused on determining brigade efficiency through reviewing training, operational readiness, response performance and fireground performance against specified standards. Where it's found that a brigade does not meet the established standard, an action plan is developed and agreed with the brigade to address the deficiency.
Supporting software has been developed on the CFA intranet to provide reports that are required for the inspection, and allows for direct entry of the inspecting officers comments and agreed actions. Planned development of the software will enhance the brigade and region's ability to monitor the brigade's performance at any time, not just at the time of the inspection.
In 2001, the CFA's chief officer determined that he required a means of ensuring that each CFA regional/state emergency coordination center were adequately prepared for their roles, with the initial focus being on wildfire preparedness. A regional/state preparedness monitoring process was developed and implemented prior to the 2002-2003 fire season and has been conducted annually since. The objective of the preparedness monitoring process is to provide an assessment of the level of preparedness of CFA to respond to and manage fires and incidents. A key focus of the program is to develop appropriate action plans to address gaps in planning and share ideas among regional staff members through access to best practice models of documentation, processes and procedures developed by their peers.
An annual review is conducted by a group of senior, experienced operations managers, and is based on a predetermined set of criteria that focuses on the capability of the regional/state emergency coordination center to manage and coordinate incident response.
Real-time performance monitoring is a process developed jointly with the Victorian Department of Sustainability and Environment to monitor the performance of incident management teams at larger wildfires as they are operating. The purpose of the real-time process is to monitor the activities of the incident management structure to promote safety throughout the incident as well as effective and efficient incident management. It also promotes continuous improvement in incident management by effectively measuring operational performances during incidents, and reinforcing established standards and performance measures
CFA and DSE first implemented the process during the 2002-2003 fire season. A group of experienced incident controllers were recruited as members of performance monitoring teams and were briefed on the program.
During each visit, team members are deployed to an incident to review incident management arrangements against a checklist of key performance parameters. The team is required to liaise closely with the incident commander, and conclude each visit with a detailed briefing for the incident controller and that individual's team with respect to significant findings or observations made. In addition, performance monitoring teams are required to provide chief officers of CFA and DSE with an initial report identifying any significant findings within 24 hours, an interim report of findings within 28 days and a final report within 60 days.
Now that the development and implementation of these processes is complete, work has commenced on reviewing processes for debriefs, operations analysis and near-miss investigations.
Debriefing is integral to the process of continuous improvement. CFA has determined that by conducting debriefing sessions it is able to reinforce positive practices and procedures while identifying where things could have been done better.
Prior to the 2002-2003 fire season, CFA conducted debriefs of specific incidents on a regular but limited basis at a local level, with little transfer of any lessons learned to rest of the organization. Following the 2002-2003 fire season, CFA and DSE recognized the need to undertake a coordinated and strategic approach to debriefing the extensive campaign, as the “traditional” approach of getting everyone together to talk about the incident was physically impossible. Ironically, debriefing processes were identified during this program as an area that could be improved, and CFA is currently reviewing this aspect of the performance improvement program. Following the review, new procedures will be introduced to help capture and resolve issues raised in debriefs as they occur.
The use of processes such as after-action reviews as used by the U.S. Wildland Fire Lessons Learned Center, also will be considered. Simple processes for entry of lessons learned into centralized databases at the same time as the fire report is being entered also will be examined.
CFA has conducted operations analysis of significant incidents on an ad hoc basis for many years. Operations analysis has been identified as an important tool for reviewing operational performance. Interim procedures already have been developed that describe terms of reference, business rules for activation and duties of the operations analysis team, reporting formats, and timelines.
Operations analyses are only conducted at a limited number of large incidents because the process is very labor intensive and the number of personnel with the available time and experience are few. There are potential benefits in developing a more streamlined version of the operations analysis process to enable the identification of key issues to occur more regularly. Such a simplified process could potentially be delivered by a broader group of suitably trained and experienced personnel at a larger number of fires. Other fire agencies have achieved this objective by using experienced fire investigators already present at the fire to conduct operations analysis.
CFA and DSE are developing a joint process for the investigation of accidents and near-misses during operations. The process will provide a more detailed analysis of operational issues relating to accidents or near-misses that occur during operational response. The linkages and relationship of the proposed accident/near-miss investigation process to the broader corporate incident reporting processes that operate within both agencies for all agency business (not just operational response) needs clarification.
After the 2003-2004 wildfire season, CFA and DSE resolved to undertake a structured post — fire season survey. This process was based on previous successful implementation of post-fire surveys by DSE, and by the U.S. Wildland Fire Lessons Learned Center following the 2003 fire of Southern California.
The survey was developed to seek the opinions of senior operational personnel and incident management team members within both agencies, regarding their knowledge, and views about the effectiveness of implementation of the key actions that arose from the debrief process of the 2002-2003 fire season. A combination of closed and open questions were used, and respondents were given the opportunity to make additional comments. Individual's responses to the survey were collated and analyzed both in total and on agency-specific lines.
Given the recent introduction of the survey, comparison against benchmarks is limited at this time. This technique appears to be capable of providing reliable, strategic feedback on operational performance in a very efficient manner compared to the very resource intensive debriefing processes. It is proposed to continue to use this process to assist in identifying performance trends over time
As described above, CFA has been the subject of a number of external reviews of operational performance, and various government inquiries into the 2003 bushfires. The outcomes of these internal and external reviews are generally a list of recommendations. In the past, actions to address these recommendations were often developed in a disjunct or uncoordinated manner with no reference to similar findings from other reviews, and monitoring of progress in implementing the actions was difficult. To address this issue, CFA has developed a performance improvement monitoring system on the CFA intranet. The system allows users to review all operational reviews, any recommendations relating to the review, and progress on implementing actions to address the recommendations.
There are many types of actions usually identified in response to recommendations and findings generated by a review. These generally take the form of amendments to legislation, agreements, policies, procedures, guidelines, practices, or equipment. These then flow through to amendments to training materials, and education of current personnel regarding the changes through operational updates, memos, briefings, articles in internal newsletters, etc. The challenge for any organization attempting to implement improvement is to coordinate all of these processes to obtain a clear and ongoing change in behavior. Other challenges that are faced by CFA in implementing the operations performance improvement program include:
- Maintaining momentum and corporate commitment to development and implementation of the program.
- Developing effective lessons learned processes to share learnings with other agencies.
- Establishing effective linkages to corporate performance improvement processes.
- Developing effective processes for routine analysis of small “bread and butter” incidents both during and after the incident.
- Improving functionality and efficiency of the intranet performance improvement monitoring system.
- Evaluating the relevance and usefulness of human factors and situational awareness concepts and implementing accordingly.
The systems and processes being developed in the operations performance improvement program hopefully will address these challenges effectively.
Greg Esnouf is a deputy chief officer and manager of operations performance improvement with the Country Fire Authority, Victoria, Australia.
