TOTAL RECALL

By Michael T. DeGrosky

Whether viewed as a tool, a technique or a process, an after-action review uses an appraisal of experience to improve performance by reproducing success and preventing recurrent errors. An AAR enables key participants in a mission-critical activity to review their assignments, identify successes and failures, and look for ways to continue successful performance or improve deficient operations in the future.

The U.S. Army first developed the AAR as a learning method in the mid-1970s to facilitate learning from combat training exercises. It has since become standard Army procedure in both training and operations, providing an avenue for feedback, a means of promoting evaluation, and a mechanism for improving unit cohesion.

Many organizations have adopted the AAR as procedure, in many cases adapting it to their own needs; and one can see the process at work in diverse environments including military, government, medical, industrial, retail, service, and not-for-profit organizations. The AAR has gained broad acceptance among organizations operating in high-risk environments, in which common human error can produce unacceptable outcomes.

Among those organizations, elements of U.S. wildland fire agencies have been conducting AARs since the late 1990s, with the process entering these agencies through the evolving National Wildfire Coordinating Group leadership-development training curriculum. Through the influence of the NWCG leadership training initiative, a significant part of the wildland fire work force now understands the purpose and intent of the AAR and conducts some type of review process.

However, anecdotal evidence suggests that effective AAR practice hasn't penetrated wildland fire operations as thoroughly as might be hoped, and too few resources are optimally conducting AARs. As a vehicle for capturing and learning from experience, the AAR provides an effective tool of continuous learning for an organization.

Still, there are ways to improve AAR practice within wildland fire agencies, and three strategic actions necessary to systematically and comprehensively use the AAR process.

By learning from collective experience, organizations can capture and spread knowledge and apply learning so that they may understand events and improve performance. One might consider these traits as characteristic of learning organizations. A learning organization is one “… skilled at creating, acquiring, interpreting, transferring, and retaining knowledge, and at purposefully modifying its behavior to reflect new knowledge and insights,” as David Garvin wrote in his book, Learning In Action — A Guide to Putting the Learning Organization to Work.

In the broader context of organizational learning, the AAR provides organizations with a simple, powerful tool enabling them to continuously learn from their daily experiences. Consequently, the AAR not only arms the learning organization with a useful field-level technique for making learning routine and improving the effectiveness of personnel, but adoption of the AAR process can move the organization toward broader organizational learning.

The AAR concept entered the federal wildland fire agencies when Interagency Hotshot Crew superintendents gained exposure to the process through nascent human factors and leadership training in the late 1990s. A group of IHC superintendents began conducting and consequently modeling and pioneering the concept in their agencies. As the NWCG leadership training initiative evolved, matured and gained the full support of agency management, thousands of emerging leaders were introduced to the AAR process.

In 2002, the NWCG included AAR guidance in the Incident Response Pocket Guide and the Wildland Fire Lessons Learned Center began planning AAR train-the-trainer workshops. Owing to the success of these initiatives, a significant part of the wildland fire work force knows the purpose and intent of the AAR, and many crews, teams, modules and organizations conduct some type of AAR process.

However, while little hard data exists, field experience suggests that, while the AAR concept has made its way into fire agencies, skilled AAR practice hasn't thoroughly penetrated wildland fire operations. While some resources have made the AAR routine and have become quite skilled at it, relatively few fire units are conducting AARs routinely or using optimal practice.

That isn't to suggest that the agencies have underachieved in their effort to adopt the AAR as a technique for reviewing experience with the intent of improving performance. Indeed, the AAR concept also evolved slowly (over 20 years) in the U.S. Army. AAR practice entered the wildland fire agencies via a grassroots effort. Much of this effort was directed at borrowing techniques from other industries and disciplines, with emphasis placed on rapid integration rather than optimal design, acceptance and performance.

Consequently, after a short five-year period, a significant portion of the wildland fire work force engages in some type of AAR process, though practical performance may be falling short of known best practices. Relatively little hard data exist to definitively describe the practical experience with the AAR method. However, available data suggest the following:

Fire agencies have adopted the AAR as a technique, not as a process within the context of, or contributing to, a broader organizational learning environment. It appears that many firefighters have been trained to understand the mechanics of AAR conduct as a procedure without embracing the desired outcome of the AAR, that being purposefully modified behavior reflecting insight and knowledge gained by reviewing experience. Consequently, the benefits that have accrued have been sub-optimal.

AAR conduct is irregular. Curt Braun of the University of Idaho surveyed 399 fire personnel on two Type-1 fires in 2003. When asked if they had ever participated in an AAR, 60.9% of “overhead” respondents indicated that they had not. In contrast, 81% of responding agency hand crew personnel indicated that they had participated in an AAR. Conversely, only 16.6% of respondents from contract hand crews had been involved in the review process.

The dichotomies between agency hand crews and overhead and between agency and contract hand crews reflect the vector through which the process is entering the work environment, that being the NWCG leadership training curriculum. Braun also asked how many times the respondent had participated in an AAR that fire season but, unfortunately, did not report the results due to problems with the data.

There's an implication for future research here. While we know that fire crews have adopted the AAR, we don't know how routinely fire personnel engage in the AAR process. However, evidence suggests that AARs often are conducted as one-off, infrequent events, not routinely as a discipline or standard procedure. AARs contribute to performance best when seen, not as an event, but as an ongoing practice, a disciplined approach to improving performance over time.

When asked, a significant portion of fire personnel report that they use debriefing techniques other than the AAR, or informally conduct reviews without employing the practices established by the leadership training and published in the Incident Response Pocket Guide. For example, field interviews with 19 firefighters on a 2003 fire found only one respondent (the superintendent of a crew working toward IHC status) familiar with, and routinely using, established AAR practice.

Helibase personnel interviewed on this incident indicated that that they conducted debriefings using the form in the Interagency Helicopter Operations Guide, which is similar to the standard AAR approach, but more specific to helicopter operations. However, notable was the fact that most personnel interviewed on this incident were unfamiliar with the terms “after-action review” or “AAR.” While most indicated that they typically conduct some sort of debriefing with crew members on fire assignments, it appears that without using standard practice, these “AARs” may be missing the intended purpose of the process.

These findings align with those of a separate effort to interview eight experienced Type-2 crew bosses. While more familiar with the existence of the AAR process and IRPG guidance, only two of these eight crew bosses routinely used standard AAR practice as taught in the NWCG leadership training and found in the pocket guide. Like other personnel interviewed elsewhere, most either used debriefing techniques other than the AAR or informally conducted AARs without following standard practice as published in the pocket guide.

A similar field study found that the AAR method might not have effectively penetrated the wildland fire-use environment. Findings from that incident suggest that very few WFU resources are routinely conducting AARs as designed or intended. On the incident studied, observers saw little evidence that the AAR process was being used as designed by the Fire Use Management Team, district personnel, experienced division supervisors, or by most line personnel. A few line resources and district personnel were conducting “standard” AARs.

Currently, no systematic approach exists for preparing agency personnel to use the AAR tool at multiple agency levels or across a full-range of work environments. As mentioned earlier, AAR practice entered the wildland fire agencies, first through a grassroots effort, and later via the NWCG leadership training curriculum and a small effort to conduct AAR train-the-trainer workshops. None of these efforts has sought, or received, a high level of management support or commitment.

As a result, though AAR practice has become widespread within NWCG member agencies, agency managers haven't actively encouraged or supported AAR practice. In fact, uncoordinated management actions have, more often than not, interfered with organization-wide acceptance, integration and performance. Consequently, it should be no surprise that AARs are common practice in portions of the wildland fire work force while remaining nearly absent in others, and that approaches to AAR conduct vary dramatically.

More than 6,000 people have been exposed to the AAR concept through the NWCG leadership training curriculum and the L-380 (Fireline Leadership) training in particular. This training represents, by far, the most significant mechanism for introducing the AAR concept to fire personnel. However, it should be noted that this training primarily enables participants to understand the purpose of the AAR and prepares them to effectively participate in an AAR. The training does little to prepare participants to facilitate the process. AAR train-the-trainer workshops conducted by the Wildland Fire Lessons Learned Center strive to address that need but have reached very few people. AAR experts widely accept that skilled facilitation is essential to effective AAR practice.

Braun reported that when asked to “Please give an example of one (or more) thing(s) you dislike about the AAR process” the responses of 84 survey participants (21% of all participants) fit into one of five categories:

• Unintended/undesirable effects,
• Time and timing issues,
• Inappropriate AAR facilitation,
• Redundancy or
• Process modifications.

Braun categorized only 5% of responses to this question as “inappropriate AAR facilitation.” However, on reexamination of the data, one realizes that the quality of AAR facilitation would bear directly on approximately 30% of the responses to this question.

Practical performance is falling short of known best practices. It appears that the time has come for the NWCG and its member agencies to enhance and intensify their effort if they are to realize the full benefit of the AAR process, both as a useful field-level technique for improving performance and as an element of broader organizational learning efforts. Actions called for include:

Adopting a culture of continuous learning. Learning organizations succeed because people at all organizational levels share information and learn from experience. Leaders in these organizations promote learning first by modeling, or in other words, learning on a personal level. Second, leaders advance learning by helping others in their units learn. Finally, the leaders of learning organizations create and contribute to an organizational culture promoting learning. Within this context, the AAR is a process for learning from experience, capturing and spreading knowledge, sharing information, and purposefully modifying behavior reflecting insight and knowledge gained by reviewing experience.

To achieve the most benefit, organizational leaders must focus on why they conduct AARs; consistently communicate that rationale to their personnel; and, once an AAR is done, disseminate learning to others who may be embarking on similar actions. Without adopting such a learning culture, there exists a danger of encouraging fire personnel to go through the motions of an AAR without clarity of purpose, turning AARs into a non-thinking ritual that doesn't adequately review experience, cause learning, or result in modified behavior that improves performance.

Methodically making AAR practice routine, consistent, and as important as other organizational activity. AARs contribute to performance best when seen, not as an event, but as an ongoing practice, a disciplined approach to improving performance over time. By creating a discipline to capture and apply learning over time, the effects of AARs are cumulative. The AAR process is most likely to improve organizational performance, and is most likely to be sustained, when there's a high level of management commitment and AAR practice is encouraged and supported. AAR conduct must become regular or routine, and personnel must understand known best practices and conduct their AARs in accordance with them.

While some within wildland fire agencies fear standardization and formal organizational adoption, experience suggests that the corollary, irregular and informal conduct, may actually represent a greater threat to the credibility and importance of the AAR in the eyes of the average firefighter. The NWCG and its member agencies will know that they have achieved a lasting; sustainable process for understanding events and improving performance when fire units routinely conduct AARs as a discipline or standard procedure, rather than one-off, infrequent events. AARs contribute to performance best when seen, not as an event, but as an ongoing practice, a disciplined approach to improving performance.

Systematically preparing people to lead a review by developing their facilitation skills. The NWCG leadership-training curriculum prepares participants to effectively participate in an AAR. However, only a portion of the work force attends this training, and the training does little to prepare participants to facilitate the process. AAR train-the-trainer workshops conducted by the Wildland Fire Lessons Learned Center reach relatively few people.

Currently, no systematic approach exists for preparing agency personnel to facilitate the AAR process, and generally speaking, facilitation skills are lacking in the agency work force. Since AAR experts agree that skilled facilitation is essential to effective AAR practice, this represents a situation requiring attention. Needed is an AAR train-the-trainer strategy, the goal of which would be to develop a sufficient cadre of AAR trainers nationally. The plan should create that cadre strategically and systematically, with the intent of producing a sufficient number of qualified trainers who are geographically and organizationally distributed while simultaneously maintaining standards and quality.

The after-action review is a process technique that uses a review of experience to avoid recurrent mistakes and reproduce success. As a vehicle for capturing and learning from experience, the AAR provides an effective tool of continuous learning for the organization. It's within this organizational learning context that I've endeavored to suggest ways to improve AAR practice within wildland fire agencies. If they are to realize the full benefit of the AAR process, both as a useful field-level technique for improving performance and as an element of broader organizational learning efforts, it appears the NWCG and its member agencies must enhance and intensify their efforts to integrate to process into fire operations.

Actions called for include adopting a culture of continuous learning; methodically making AAR practice routine, consistent, and as important as other organizational activity; and systematically preparing people to lead an AAR by developing their facilitation skills.

These conclusions are based on available data collected both anecdotally through field studies, and through a single quasi-experimental study. To fully understand the nature and extent of AAR use in NWCG agencies will require more, and more systematically collected, data. There is a need to comprehensively survey fire personnel about their AAR experience and practices.

---

Michael T. DeGrosky is the chief executive officer of the Guidance Group, a consulting organization specializing in the human and organizational aspects of the fire service. His interests include leadership, strategy, and bringing the concepts of learning organizations and high-reliability organizing alive in fire organizations. He currently completed a master's degree in organizational leadership. He can be reached at info@guidancegroup.org.

The WIldland Fire Center for Lessons Learned offers the following tips for conducting after-action reviews:

• Schedule shortly after the activity is completed.
• Pay attention to time. Use the 25% / 25% / 50% suggested time allotment.
• Focus on what not who.
• Establish clear ground rules. Encourage candor and openness — this is dialogue not lecture or debate. Focus on items that can be fixed; keep all discussions confidential.

The leader's role is to ensure there is skilled facilitation of the AAR.

The AAR should ask four questions:

1. What did we set out to do? (Spend about 25% of total time on this question and the next.)

   • Establish the facts.

   • Purpose of the mission and definition of success:

     • Key tasks involved.
     • Conditions under which each task may need to be performed (weather, topography, time restrictions, etc.)
     • Acceptable standards for success (explaining what “right” looks like).

2. What actually happened?

   • Continue to establish the facts.
   • Participants should come to agreement on what actually happened.
   • Pool multiple perspectives to build a shared picture of what happened.

3. Why did it happen? (25% of total time)

   • Analysis of cause and effect
   • Focus on what not who.
   • Provide progressive refinement for drawing out explanations of what occurred. This will lead into developing possible solutions.

4. What are we going to do next time? (50% of total time)

   • Solutions will arise naturally once problems are identified and understood.
   • Focus on items you can fix, rather than external forces outside of your control.
   • Identify areas where groups are performing well and should sustain. This will help repeat success and create a balanced approach to the AAR.