Preparedness & Cyber Risk Reduction Part Six: Evaluate & Improve

With the goal of reducing cyber risk and by supporting effective incident response, heretofore in our series on Preparedness, we have explored the different components of the Preparedness Cycle — the continuous cycle of planning, organizing, training, equipping, and exercising. In this second to last post in the series, we’ll briefly look at the last two parts: evaluating and taking corrective actions. For common understanding, let’s start with two exercise-specific definitions via the FEMA Preparedness Toolkit:

  • Evaluation: “Exercise evaluation is the cornerstone of an exercise and maintains the functional link between exercise and improvement planning. Through exercise evaluation, organizations assess the capabilities needed to accomplish a mission, function, or objective. Effective exercise evaluation involves planning for exercise evaluation, observing and collecting data during exercise conduct, analyzing data, and reporting exercise outcomes.”
  • Improvement Planning: “Exercises afford organizations the opportunity to evaluate capabilities and assess progress toward meeting capability targets in a controlled, low-risk setting. An effective corrective action program develops improvement plans that are dynamic documents, with corrective actions continually monitored and implemented as part of improving preparedness.”

For those desiring a “deep dive” into exercise evaluation and improvement planning, review the guidance in the 2013 Homeland Security Exercise and Evaluation Program (HSEEP). That will provide details on the process of developing and conducting evaluation and improvement planning and documentation, addressing ideas such as Exercise Evaluation Guides (EEGs), data collection, after action reporting, and developing an improvement plan and corrective action program. Below, I’d like to share a few ideas for additional consideration.

Do What Works

The HSEEP guidance above provides specific approaches that work. Using well-established standards like Core Capabilities and EEGs provide common terms and references, and help promote consistency in evaluations and documentation. All good! However, not every exercise is resourced (nor really requires) the complete HSEEP approach. HSEEP is guidance and should be treated as exactly that. If you want to irritate an exercise pro, tell them you want an “HSEEP-compliant exercise” and watch their eyes roll into the deepest parts of their skull … What is critical is that you plan for evaluations hand-in-hand with training and exercises and that you have a deliberate approach. Your organization may have some specific ways you like to capture and report information or you may need to be mindful of certain sensitivities. More often, you have to contend with being under-resourced and need to manage the best evaluation you can with what you have available in both people and time. What is most important is that you know what you have available, deliberately plan as part of the training and exercise development process, and ensure evaluation does occur and is documented. If you do that, however exactly you have to do it, you’re doing pretty well!

Get Buy-In

As noted in our mini-series on exercises, exercises tend to get the most attention. Exercises are fun! — evaluations are much more boring, and can be contentious, and frustrating… Getting buy-in early and from the right people can save planners (particularly junior personnel) a lot of grief and greatly help support an effective and value-added evaluation. We want to gain buy-in into our approach to the evaluation, as well as to the activities supporting the evaluation and improvement planning. So, who do we need buy-in from? Well, ideally, everyone. But given we can’t court every leader and participant, it is good to try and ensure that your exercise sponsor is on-board, as well as those that will help conduct the evaluation.  For events like After Action Meetings (AAM; again, refer to HSEEP guidance for details), know who some of the key players and influencers are and work with them to help them understand what you’re doing, where it’s going, and to get their support for the process and your efforts. And know who you’re going to be putting some focus on and get ahead of potential tensions and flare-ups — but engage them privately before doing so publicly. If you’re about to go into an AAM and know that a certain organization or department is about to hear some things they won’t like, talk to them ahead of time (which hopefully you’ve done in developing the evaluation) and agree to how you may approach some of the more difficult areas. They may still not like your approach, but by engaging them, you may get more support, or at least less objection (and sometimes you won’t, and it might get ugly…). In both developing the evaluation process and in conducting the evaluation and after action activities, building support and getting others to invest in what you’re doing can grease the process and make it a lot more successful.

Seek Continuous Improvement

One of my favorite books is the classic Animal Farm and like Boxer, the hardworking but rather dim horse in that story, my typical approach to things is to put my head down, block out the noise and tell myself, “I will work harder.” After many years of ugly running and punishing my Achilles, I started cycling about a year ago. Applying my usual approach, I try to muscle through every challenge, which has some utility. But, when I take the time to look at my stats, assess parts of the ride and how I tackled them, compare with previous workouts, and otherwise assess and evaluate my performance, I’m able to better understand how I did and how I can improve. My goal is to keep getting better. In Animal Farm, Boxer’s valiant efforts end in the care of the “Horse Slaughterer and Glue Boiler,” and I’d prefer a smarter, more positive outcome. By properly planning and preparing for my ride evaluation, taking the appropriate amount of time to review, assess, and evaluate my performance, I am able to work towards continuous improvement and hopefully reaching the desired level of physical fitness. Hopefully… The same approach should be applied towards exercises and preparedness broadly. Develop a multi-year plan (as discussed in previous posts in this series), establish goals and milestones, plan but be flexible, and seek to continuously improve the readiness and resilience of your organization through effective evaluation, corrections, and improvement planning.

With this post, we’ve worked our way through the Preparedness Cycle! In the concluding segment to this series, I’ll talk to Jeff Peters as we conclude this series on the Preparedness Cycle, some common issues, best practices, and more.

Preparedness & Cyber Risk Reduction Part Five C: Operations-Based Exercises

As we continue in our series on Preparedness, and concluding this mini-series on exercises, in the section that follows, we’ll look at different types of operations-based exercises as we continue to explore some of the ways our fictional character, Johnny, and his colleagues at Acme Innovations can take a progressively challenging approach to exercise design as they strive to better prepare for and decrease the risks associated with the threat of ransomware.

As with the previous post, the quotes below are taken from the Homeland Security Exercise and Evaluation Program (HSEEP). In our last post, we addressed some of the discussion-based exercises Johnny and the Acme team would be conducting. Moving on to more complex and realistic operation-based exercises, Johnny is ready to try some simple drills.


“A drill is a coordinated, supervised activity usually employed to validate a specific function or capability in a single agency or organization. Drills are commonly used to provide training on new equipment, validate procedures, or practice and maintain current skills. For example, drills may be appropriate for establishing a community-designated disaster receiving center or shelter. Drills can also be used to determine if plans can be executed as designed, to assess whether more training is required, or to reinforce best practices. A drill is useful as a stand-alone tool, but a series of drills can be used to prepare several organizations to collaborate in an FSE. For every drill, clearly defined plans, procedures, and protocols need to be in place. Personnel need to be familiar with those plans and trained in the processes and procedures to be drilled.”

Using the newly validated Annex as reference, and based on the same scenario that was previously exercised, Johnny conducts several short drills to validate that personnel understand and are able to execute roles, responsibilities, and procedures detailed in the Annex. With leadership approval, Johnny leads three unannounced drills over the course of a two-week period. One drill involves several individuals reporting a suspected ransomware infection on their device to different parts of Acme in order to test recipients’ ability to properly receive and understand the messages, as well as communicate the suspicious incident to the proper POCs within the time frame determined in the Annex. A second drill exercises the leadership decision making processes upon notification of a suspected ransomware incident. The third drill allowed participants the opportunity to practice reestablishing files from back-ups following a notional ransomware infection.

Functional Exercises

“FEs are designed to validate and evaluate capabilities, multiple functions and/or sub-functions, or interdependent groups of functions. FEs are typically focused on exercising plans, policies, procedures, and staff members involved in management, direction, command, and control functions. In FEs, events are projected through an exercise scenario with event updates that drive activity typically at the management level. An FE is conducted in a realistic, real-time environment; however, movement of personnel and equipment is usually simulated. FE controllers typically use a Master Scenario Events List (MSEL) to ensure participant activity remains within predefined boundaries and ensure exercise objectives are accomplished. Simulators in a Simulation Cell (SimCell) can inject scenario elements to simulate real events.”

Following the drills, and with opportunities to make some minor refinements to the Annex and some retraining on key tasks, Johnny is approved to plan a three-hour FE that implements the procedures detailed in the Annex from initial identification of a suspected ransomware incident in real time. In a scheduled and announced exercise that includes all appropriate personnel, the Acme team wants to assess what they are successfully able to accomplish in a finite period of time and to gauge if they are able to properly follow procedures under the stress of an expanding outbreak.

Full-Scale Exercises

“FSEs are typically the most complex and resource-intensive type of exercise. They involve multiple agencies, organizations, and jurisdictions and validate many facets of preparedness. FSEs often include many players operating under cooperative systems such as the Incident Command System (ICS) or Unified Command. In an FSE, events are projected through an exercise scenario with event updates that drive activity at the operational level. FSEs are usually conducted in a real-time, stressful environment that is intended to mirror a real incident. Personnel and resources may be mobilized and deployed to the scene, where actions are performed as if a real incident had occurred. The FSE simulates reality by presenting complex and realistic problems that require critical thinking, rapid problem solving, and effective responses by trained personnel. The level of support needed to conduct an FSE is greater than that needed for other types of exercises.”

Here, Gary adds that ideally, “A full-scale cybersecurity exercise could include using a simulated cyber range environment to replicate an organization’s network, allowing for testing of response activities to simulated attacks or incidents.” It is important to try and make exercises — particularly operational exercises — as realistic as possible, and following Gary’s advice here can help challenge participants in as realistic a manner as possible.

For this year Acme has determined they are going to keep the exercise internal, and not include external subject-matter expertise that would be employed in the event of an incident beyond their team’s ability to internally manage. Following the FE, and some other exercise events that are already planned for this year, Johnny is tasked with integrating a ransomware attack into a more complicated FSE for next year that will include an additional scenario variable and the inclusion of external personnel in several areas.

Parting Thoughts

Whatever your organizations’ cyber risk focus, taking the time to plan and resource an effective, progressive exercise program can go a long way in supporting effective preparedness, and ensuring timely and successful response to incidents. The ability to properly respond to an incident can save an organization a lot of time and money — minimizing downtime and helping to minimize impacts, while supporting a quick return to normal operations.

While exercises are critical and provide an awesome opportunity for rehearsals to real incidents, the greatest value of an exercise actually comes not during, but after the event. As with Organizing and Equipping, another too-often neglected part of preparedness, follows the conduct of the exercise — the Evaluation and Improvement process, which will be Part Six in our ongoing series on Preparedness & Cyber Risk Reduction!

Preparedness & Cyber Risk Reduction Part Five B: Discussion-Based Exercises

Continuing our series on Preparedness, and this mini-series — exercises (see previous post for the intro to exercises) — this installment and the next build on our introduction, and in the section that follows we’ll look at different types of discussion-based exercises as we consider some of the ways our fictional character, Johnny, (introduced in our previous post on training) and his colleagues at Acme Innovations can approach progressive exercise design as they look to decrease the risks associated with the threat of ransomware.

The quotes below are taken from the Homeland Security Exercise and Evaluation Program (HSEEP). To start, we break exercises up into two categories – Discussion-Based and Operations-Based Exercises – and we typically progress from one to the other as we build capabilities and increase complexity, although there is certainly room for some back and forth.

  • Discussion-based exercises include seminars, workshops, tabletop exercises (TTXs), and games. These types of exercises can be used to familiarize players with, or develop new, plans, policies, agreements, and procedures. Discussion-based exercises focus on strategic, policy-oriented issues. Facilitators and/or presenters usually lead the discussion, keeping participants on track towards meeting exercise objectives.”
  • Operations-based exercises include drills, functional exercises (FEs), and full-scale exercises (FSEs). These exercises can be used to validate plans, policies, agreements, and procedures; clarify roles and responsibilities; and identify resource gaps. Operations-based exercises are characterized by actual reaction to an exercise scenario, such as initiating communications or mobilizing personnel and resources.”


“Seminars generally orient participants to, or provide an overview of, authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas. As a discussion-based exercise, seminars can be valuable for entities that are developing or making major changes to existing plans or procedures. Seminars can be similarly helpful when attempting to assess or gain awareness of the capabilities of interagency or inter-jurisdictional operations.”

Johnny wants to ensure his colleagues understand ransomware and some of the examples of incidents and best practices that he can share. After talking with some of his coworkers, contacts at other companies, and local government partners through the state fusion center, he develops a half-day seminar event. The Ransomware Seminar includes a mix of panels and presentations. The agenda covers what ransomware is, and a short presentation by the Acme security team on other types of cyber extortion. Two guest speakers discussed case studies from real ransomware attacks they endured. Government partners (coordinated via the fusion center) and the Acme security team shared government and industry best practices. In closing, the Acme CISO shared final thoughts to help encourage ideas in preparation of the next exercise event.


“Although similar to seminars, workshops differ in two important aspects: participant interaction is increased, and the focus is placed on achieving or building a product. Effective workshops entail the broadest attendance by relevant stakeholders. Products produced from a workshop can include new standard operating procedures (SOPs), emergency operations plans, continuity of operations plans, or mutual aid agreements. To be effective, workshops should have clearly defined objectives, products, or goals, and should focus on a specific issue.”

Shortly after the Ransomware Seminar, Johnny conducts an Acme Ransomware Response Planning Workshop. The event includes selected members from Acme’s security team, several executives and line managers, legal representatives, members from IT support, business continuity, incident response teams, and other selected personnel.

“During the planning of any type of cyber-focused exercise, an organization should strive for inclusion of a wide variety of personnel from various departments such as these to properly develop a realistic, focused exercise that addresses cross-cutting organizational issues.” – Gary Benedict, Section Chief of the DHS National Cyber Exercise & Planning Program

The group reviews highlights from the seminar with the purposes of establishing clear planning guidance and an outline of how Acme wants to respond to a ransomware incident. The actual procedures will be developed after the workshop, but informed by decisions made at the exercise.

Tabletop Exercises

“A TTX is intended to generate discussion of various issues regarding a hypothetical, simulated emergency. TTXs can be used to enhance general awareness, validate plans and procedures, rehearse concepts, and/or assess the types of systems needed to guide the prevention of, protection from, mitigation of, response to, and recovery from a defined incident. Generally, TTXs are aimed at facilitating conceptual understanding, identifying strengths and areas for improvement, and/or achieving changes in perceptions.”

“Whether its conducted with external partners or just with internal staff, a TTX environment encourages open discussion and often networking of key personnel, ensuring understanding of roles and responsibilities and preventing the notion of ‘exchanging business cards during a disaster.’” – Gary Benedict, Section Chief of the DHS National Cyber Exercise & Planning Program

After completing the “Acme Ransomware Response Annex” to the Acme Incident Response Plan, Johnny develops a TTX based on a real-world ransomware outbreak and a fictional incident at Acme. The TTX includes many of the same personnel involved in the workshop, with a few additional players. This time, rather than exploring how they may want to respond, the participants exercise the Annex to gain familiarity with now-defined expected roles and responsibilities, and to validate that the Annex properly and effectively addresses the incident. Following the TTX, Johnny develops and After Action Report and… wait (!), we’ll cover that in the next installment of this series!


“A game is a simulation of operations that often involves two or more teams, usually in a competitive environment, using rules, data, and procedures… Games explore the consequences of player decisions and actions. They are useful tools for validating plans and procedures or evaluating resource requirements. During game play, decision-making may be either slow and deliberate or rapid and more stressful, depending on the exercise design and objectives. The open, decision-based format of a game can incorporate ‘what if’ questions that expand exercise benefits. Depending on the game’s design, the consequences of player actions can be either pre-scripted or decided dynamically. Identifying critical decision-making points is a major factor in the success of evaluating a game.”

Based on time and resources, and his assessment of utility for this threat, Johnny will not conduct a ransomware game. While he’d like to see the entire exercise series progression, he determines that after the TTX, Acme will move into some short, focused drills. Drills, and other operations-based exercises, will be addressed in our next installment, as we continue our discussion on exercise types and wrap-up this mini-series on exercises.

Preparedness & Cyber Risk Reduction Part Five A: Intro to Exercises

Returning to our ongoing series on Preparedness, this post addresses what is probably the most fun part of preparedness — exercises! A championship football team needs to be complete — with great linemen to fight in the trenches, defensive players to dominate their side of the ball, skills players and special teams to razzle and dazzle and put up points, and then there’s the quarterback — the attention getting centerpiece of nearly every team. Champions in preparedness also need to have success through every part of the Preparedness Cycle — the continuous cycle of planning, organizing, training, equipping, exercising, evaluating, and taking corrective actions to support effective incident response — but exercises, like quarterbacks, seem to always garner a lot of attention and can be seen to make or break the rest of the program.

To help highlight some areas with expert insight, I’ve asked a colleague to share some wisdom as well. Several areas below include comments from my colleague, Gary Benedict, who serves as the Section Chief of the Department of Homeland Security’s National Cyber Exercise & Planning Program.

What Are Exercises?

As we have noted in previous parts of this series, our focus on preparedness is to effectively support our efforts to reduce organizational risks — the “potential for an unwanted outcome resulting from an incident, event, or occurrence, as determined by its likelihood and the associated consequences.” Exercises play a critical role to that end. The Homeland Security Exercise and Evaluation Program (HSEEP) defines exercises as instruments “to train for, assess, practice, and improve performance in prevention, protection, mitigation, response, and recovery capabilities in a risk-free environment. Exercises can be used for testing and validating policies, plans, procedures, training, equipment, and interagency agreements; clarifying and training personnel in roles and responsibilities; improving interagency coordination and communications; improving individual performance; identifying gaps in resources; and identifying opportunities for improvement.”

HSEEP “provides a set of guiding principles for exercise programs, as well as a common approach to exercise program management, design and development, conduct, evaluation, and improvement planning. … Through the use of HSEEP, exercise program managers can develop, execute, and evaluate exercises that address the priorities established by an organization’s leaders. … These priorities guide the overall direction of a progressive exercise program, where individual exercises are anchored to a common set of priorities or objectives and build toward an increasing level of complexity over time. Accordingly, these priorities guide the design and development of individual exercises. … Through improvement planning, organizations take the corrective actions needed to improve plans, build and sustain capabilities, and maintain readiness. … HSEEP exercise and evaluation doctrine is flexible, scalable, and adaptable, and is for use by stakeholders across the whole community.”

I really appreciate the HSEEP methodology because it is logical, repeatable, helps us all use common terms, and is flexible. FEMA has written, “Exercise practitioners are encouraged to apply and adapt HSEEP doctrine to meet their specific needs.” We won’t get into all the weeds of exercises here, but the current version of HSEEP can be accessed here and I encourage anyone involved in the planning of exercises to take time to get familiar with this document.

Types of Exercises

When we look at the Preparedness Cycle, exercises are usually placed in the sequence noted above — planning, organizing, training, equipping, exercising, evaluating, and taking corrective actions. That is the right place for them to be; however, they may also be used to help inform planning and can be very effective for that purpose. One important idea to understand is that an effective exercise program should progress through a series of successive and increasingly complex exercises leading up to the desired level of proficiency and preparedness. “This progressive approach, with exercises that build upon each other and are supported at each step with training resources, will ensure that organizations do not rush into a full-scale exercise too quickly. Effective planning of exercises and integration of the necessary training will reduce the waste of limited exercise resources and serve to address known shortfalls prior to the conduct of the exercise” (HSEEP).

In planning the progressive schedule of exercises, it is important that exercises are conducted at a cadence that allows organizations to learn from previous exercises and make appropriate procedural refinements before engaging in more challenging exercises. This can be a particular challenge for large organizations with broad regulatory accountability, especially ones that are also trying to support external exercises such as with government or their information sharing communities. Exercise planners often know where some of the likely trouble areas for an organization may be — for many exercises capabilities such as communications and planning repeatedly come up, for example — and should work with their organization to provide enough time to learn and improve before progressing to more complex activities and repeating the same mistakes.

From his years of experience in cyber and physical security exercises, Gary adds that the progressive, “building block approach should be documented into a multi-year Training and Exercise strategy (which we referred two in part two of this series under Preparedness Planning). A critical component to the success of this approach is also having senior leadership approval and buy-in. Exercise strategy can be influenced by organizational ongoing risk analysis, so exercise planners should allow some flexibility in the strategy to be adjusted as the risk landscape evolves.”    

What follows are brief descriptions of the different exercise types, and some ideas on how they may fit into a cybersecurity exercise program. To do that, we’ll continue the adventures of our Preparedness Champion, Johnny, and his company Acme Innovations (see previous blogs for reference).

If you recall from our previous post on training, Johnny and his colleagues at Acme Innovations had identified the threat of ransomware as a very concerning risk for Acme. In the next installment of this series we’ll be looking at the different exercise types as we consider some of the ways Johnny may develop a progressive exercise program to build preparedness and be ready for a potential ransomware incident.

Preparedness & Cyber Risk Reduction Part Four: Awareness and Operational Training

In our ongoing series on Preparedness & Cyber Risk Reduction, we’ve discussed an “Introduction to the Preparedness Cycle” and we’ve explored the topics of preparedness and operational planning, and organizing and equipping. In our sustained effort to reduce risk through proper preparedness, we’ll tackle the next critical step in the Preparedness Cycle — training.

To effectively support our efforts to reduce organizational risks — which we defined as the “potential for an unwanted outcome resulting from an incident, event, or occurrence, as determined by its likelihood and the associated consequences” — we want to ensure our personnel are properly trained. Obviously, an organization conducts a variety of types of training and not all is relevant to preparedness (though a lot does impact broader risk management, such as some of the training that may be delivered by Human Resources). The focus of this article is specifically on two types of training: Threat Awareness Training and Operational Training.

FEMA states that, “Training provides first responders, homeland security officials, emergency management officials, private and non-governmental partners, and other personnel with the knowledge, skills, and abilities needed to perform key tasks required by specific capabilities. Organizations should make training decisions based on information derived from the assessments, strategies, and plans developed in previous steps of the Preparedness Cycle.”

I agree with FEMA’s definition, but, regrettably, it is incomplete. Our approach is to certainly encourage Operational Training to arm personnel “with the knowledge, skills, and abilities needed to perform key tasks required by specific capabilities,” often relating to the plans, procedures, systems, and equipment we put in place in the previous steps of the Preparedness Cycle but, in addition to Operational Training, it is critical that personnel have a sound understanding of the threat environment. Our motto at Gate 15 is to apply a “threat-informed, risk-based approach to analysis, preparedness and operations.” To do that right, training needs to include efforts aimed at educating personnel on the varied threats they may encounter in the workplace (and perhaps more broadly). We consider that Threat Awareness Training.

Organizations face a wide array of threats to their operations, people, and facilities. With limited time and resources, training can’t address every threat. To help prioritize training activities and emphasis, leaders should apply a threat-informed but risk-based approach to planning, developing, and conducting training. That means understanding the threats, conducting a risk assessment, and prioritizing the greatest risks as primary areas of focus. In today’s environment, an organization, we’ll call it Acme Innovations, may conduct a risk assessment and determine that Acme’s greatest areas of concern are hostile events at the workplace, a severe earthquake, a significant data breach, and being infected with ransomware.

If you recall from Part Two of this series, we said that, ideally, organizations will have a Preparedness Champion who can help develop and maintain a multi-year training and exercise program. This program — informed by a prioritized risk assessment — should detail a training schedule and progressively challenging exercises over a few years’ period. Developing a multi-year preparedness program helps protect time and allows leaders to plan for and commit funding to support the activities. Even if the details of an event change, the time and resources will be there. In Part Two of this series we met Johnny, who it turns out, is Acme Innovations’ Preparedness Champion. Johnny’s multi-year preparedness program includes Acme’s preparedness priorities, focused on the four areas noted above. For this post, let’s focus on the concerns around ransomware.

  • Operational Training: Based on Acme Innovations’ preparedness priorities, Johnny’s multi-year preparedness program includes a deliberate approach to reduce the risks associated with the threat of ransomware. Johnny has worked with colleagues from across Acme Innovations to develop a Ransomware Response Plan, which they’ve included as an annex to the broader Acme Innovations Incident Management Playbook. The Plan includes specific actions for personnel to take upon identifying a possible ransomware infection. Those include immediate individual actions to take, who to report the incident to, what actions Acme’s security team and IT support teams are to take, key decisions and who is responsible for them, and other details developed through the process of Operational Planning. Over the next three months, Johnny and Acme’s corporate trainer are conducting training on that plan and the expected actions of all involved parties, to ensure Acme personnel understand their individual and team responsibilities in the event of a possible ransomware infection. At the end of the three-months, Johnny is leading a tabletop exercise with key leaders and responders to validate the plan and he wants everyone to know their roles and responsibilities ahead of time. But, wait – that’s the next part of this series!
  • Threat Awareness Training: While Johnny and the trainer are training the organization on how to respond to a possible ransomware incident, Johnny knows that ideally, Acme will avoid getting infected in the first place. So, Johnny has done his homework, he’s looked at some of the great online resources that address ransomware, and he’s working closely with Acme’s security team to better understand how ransomware works and how it may be delivered. With his colleagues, he’s developed a deliberate Threat Awareness Training Plan to educate Acme Innovations personnel on what ransomware is, how it can enter a network, what the implications of that are, how individuals can help to reduce the risk of infection, and other nuggets he’s learned through his discussions and research. With that, he’s excited as he starts implementing his plan and educating his coworkers! Once again, good job, Johnny!

As we noted in Part Three, FEMA describes the core capability cybersecurity as protecting, and if needed, restoring, “electronic communications systems, information, and services from damage, unauthorized use, and exploitation.” Johnny’s efforts are directly supporting that for Acme Innovations but Acme knows that their threats are more than just cybersecurity. We identified Acme’s concerns above but for your organization, whether your emphasis is on health issues – such as the impacts of a potential pandemic, or natural disasters – maybe annual spring flooding or perhaps you’re in an area that is more likely to experience high-impact hurricanes, or physical security threats – such as workplace violence, the same approach to training applies. Addressing your prioritized risk concerns, both Operational Training and Threat Awareness Training should be included in your multi-year preparedness program.

But, wait, we said above that we only have limited time and resources for training. How do we get all this done?!?! Well, different organizations will approach training, and all aspects of preparedness, differently and will allocate varying amounts of time for it. Some will choose to conduct annual training days, whereas others will approach things in smaller, more frequent iterations. Some will conduct all training with internal resources, whereas some may bring in professional trainers for some, or all, of the training. Different approaches will make more or less sense for different organizations and for different threat concerns. Hurricane training may be something your organization does on an annual basis, whereas ransomware training may come in the form of a quick update every couple weeks. It will be up to you to determine the approach that makes the most sense for your organization based on your understanding of the threats, your risk assessment, your priorities, and your available time and resources.

With cybersecurity, there is abundant information available online about the array of potential threats and easy-to-find examples of real incidents. There is also a lot of great information on how to conduct training. Yes, of course, our team at Gate 15 is happy to help you develop your multi-year preparedness program and to support your operational and threat awareness training (!) but you can leverage some great free resources to help inform and support your program as well.

What’s most important is that as an organization, you follow Acme’s example, assign a responsible champion and dedicate the necessary time to plan and conduct both operational and threat awareness training. As you progress through the Preparedness Cycle, each step builds and enhances the work of the previous step. Having effective plans, people, and equipment in place, it is vital that you give them the necessary training to understand the threats your organization is most concerned about, the risks those threats pose, and the actions you need them to take. This further enhances our preparedness and resilience, minimizing the impacts of incidents and facilitating a quick return to normal operations when they do occur. With a solid training program in place, we move onto the next step in the Preparedness Cycle. In the next installment we’ll address exercises, where we have the opportunity to test and validate all the good work we’ve done!

Preparedness & Cyber Risk Reduction Part Three: Organize & Equip

In Parts One and Two of this blog series, “Introduction to the Preparedness Cycle,” we took a general look at threats, risks and preparedness, as well as a slightly deeper look into planning — both preparedness and operational planning — to minimize the likelihood and impacts of the undesired threats that have the potential to develop into disruptions and other “unwanted outcomes.” Such outcomes could impact organizations’ people, information, operations and/or facilities, and it is our goal to be ready and resilient — ideally preventing the incidents, but, more in some cases, minimizing their impacts and facilitating a quick return to normal operations.

One approach to supporting preparedness — which we defined as a continuous cycle of planning, organizing, training, equipping, exercising, evaluating, and taking corrective actions to support effective incident response — is to apply a deliberate process to reduce our risks. That deliberate process is the Preparedness Cycle. As we continue through that Cycle, we now move on to the next step in the process – Organizing and Equipping. I often feel like this step is an unloved child in the preparedness family — frequently glossed over as planning, training, and exercise usually get more attention. In reality, this step is critical and is more present in our day-to-day operations than the rest of the preparedness activities.

FEMA states that, “Organizing and equipping include identifying what competencies and skill sets people delivering a capability should possess and ensuring an organization possesses the correct personnel. Additionally, it includes identifying and acquiring standard and/or surge equipment an organization may need to use when delivering a specific capability.” And for our purposes, we’re focused on cybersecurity, which FEMA describes as the core capability focused on protecting “(and if needed, restore) electronic communications systems, information, and services from damage, unauthorized use, and exploitation.” There is absolutely no way we will succeed in achieving that core capability if we do not have the right people and equipment in place.

So where do we begin? Well let’s start with some reality checks and then let’s move on to ice cream, a far more enjoyable topic!

  • Reality Check #1: There is no silver bullet and no automated solution that will alone protect your network and information.
  • Reality Check #2: Intelligence and information — even deep web and dark web intelligence — are useless unless they are understood, analyzed and applied towards decision-making and action – primarily in the form of preparedness and operations.
  • Reality Check #3: No two organizations are the same. Some are alike — based on size, industry, clients, etc. — but they’re not the same. In most cases, their organization and equipment needs may also be alike, but not the same.
  • Reality Check #4: Security will require an appropriate blend of technology and human resources.

And an important note, when we speak to “equipment” here, for organizations looking to address their cybersecurity preparedness, it is not just hardware but also the software, technology and services we may apply towards our security operations.

With that – to ice cream! I’m a big fan of ice cream. Seventy-five percent of the reason I workout is probably to indulge in an extra scoop of delicious creamy heaven. I like almost every flavor and am open to nearly every topping – there are a lot of great combos I can enjoy! But for me, for my tastes, it is tough to beat three scoops of chocolate chip cookie dough ice cream coated with a generous portion of shredded Heath Bar and just the right amount of caramel and fudge. Oh, yes, that does it. But that’s me. You may be a chocolate or butter pecan person. Or a marshmallow or peanut butter sauce person. Maybe you like the combined flavor of strawberry ice cream with pineapple sauce on top. The potential combinations are endless and there are probably many that would be to your liking, but probably one or two you really, really like. And, over time, maybe your tastes change. Maybe a little more fudge. Maybe switch out the mint chocolate chip for pistachio … as your needs change, so too must your ice cream sundae!

And so it is with security. There are a lot of great tools and resources out there. Some awesome technology solutions and some great talent. But not all are right for you and your organization and those that work today, may not fit tomorrow as your organization, and the threat environment, change. As you try different things to get to know your likes and dislikes with your perfect sundae, so too must you sample and experiment with the right composition of human and technology resources for your cybersecurity in order to achieve the desired capabilities. And to the aforementioned idea that, organizing and equipping “includes identifying and acquiring standard and/or surge equipment an organization may need to use when delivering a specific capability,” we need to think of potential areas where we may need enhanced support. Perhaps if we suspect we have malware on our network or if we experience a data breach. Wherever we assess risks that we want to be able to operationally address (as opposed to something we’d accept and address via insurance, for example) and do not have the organic in-house capabilities, we need to be able to surge, with internal or external resources, to meet the potential situation.

In addition to our preferences, we also have to respect competing demands and requirements. Whether buying ice cream and groceries or security solutions and hiring talent, we have to respect the constraints of our budgets and choose smartly. I need to both eat and maintain security daily. Sometimes we can buy steak, sometimes we can’t. But we can eat. With security, maybe you can’t get that sophisticated phishing training service you wanted right now, but you can put together an in-house threat awareness program and threat identification and reporting incentive program. Maybe you can’t hire a malware analyst right now, but you can register with the FBI and submit issues into their iGuardian program or join your appropriate information sharing group and leverage their resources and capabilities. In organizing and equipping, as with all preparedness, we should start with basic steps and progress towards a desired endstate. Maybe the goal is a world-class security operations center with validated incident response capabilities. Great! But maybe that starts with some free, basic subscription services, portal registrations and hiring a junior analyst. How fast, and how robust, need to be based on your organization, your risk assessment, your available resources and your goals.

That being said, of course (!) you should consider SurfWatch Labs’ Threat Analyst and Cyber Advisor products and Gate 15’s support for your cyber exercise program … of course, of course, but you already knew that those were as critical to your security sundae as the ice cream itself!

The process of organizing and equipping – like all aspects of threat, risk and preparedness management, is continuous and needs to be regularly reassessed. As you continue into the Preparedness Cycle, as you run drills and more complex exercises to test your team and processes, and as you encounter real events and incidents, there will be numerous opportunities to document successes and opportunities for improvement. These should help you refine your people and processes, and your organization and equipment as well. But, before we get into exercises, we need to give our personnel effective training on the plans, procedures, systems, and equipment we have in place. And that will be the subject of the next installment in this series!

Preparedness & Cyber Risk Reduction Part Two: Preparedness and Operational Planning

In part one of this blog series, “Introduction to the Preparedness Cycle,” we took a general look at threats, risks, and preparedness. Recognizing that there will be undesired threats that develop into disruptions and other “unwanted outcomes” impacting our organizations’ people, information, operations, and/or facilities, we want to be ready and resilient — ideally preventing the incidents, but more likely trying to minimize their impacts and facilitating a quick return to normal operations. To support that, we can apply a deliberate process of preparedness to address our threats, physical and cyber, and reduce our risks – the “potential for an unwanted outcome resulting from an incident, event, or occurrence, as determined by its likelihood and the associated consequences.”

We defined preparedness as a continuous cycle of planning, organizing, training, equipping, exercising, evaluating, and taking corrective actions to support effective incident response. This post addresses the first step – planning. There are actually two important aspects to planning – Preparedness Planning and Operational Planning — and ideally, an organization will do both.  

Preparedness Planning

There are a number of ways to mitigate risks. In some instances, we assess the risk as low or the cost of mitigation as too much, and we decide not to do anything at all, accepting the risks and moving on. In some cases, we get insurance to help manage the potential consequences of an incident. In some cases, we determine to take preparedness actions to decrease risks. In those cases, preparedness – planning, as well as training and exercises – needs to be thought of like insurance in that you don’t pay insurance once and stop. You pay it month in and month out, use it or not. Same thing with preparedness. It needs to be scheduled and recurring. Plan for it, do it regularly, keep doing it. With our insurance bill, we plan for it, allocating time and resources to make sure we pay it. Again, with preparedness, we need to plan our activities and set aside the time and resources to conduct them.

Ideally, organizations will have a preparedness champion who can help develop and maintain a multi-year training and exercise program. This program – informed by a prioritized assessment of risks — should detail a training schedule and progressively challenging exercises over a few years’ period. Its not set in stone and needs to be flexible enough to be updated as threats evolve and risks are regularly reassessed. However, the near events should be locked in, with events further away scheduled, but tentative, pending confirmation or refinement. Developing a multi-year preparedness program helps protect time and allows leaders to plan for and commit funding to support the activities. Even if the details of an event change, the time and resources will be there.

In cyber preparedness, we may, for example, assess that our greatest risk is a significant data breach. And let’s say Johnny has been assigned as our Preparedness Champion. Johnny, taking his task seriously, investigates and finds that there is no plan for responding to a data breach. As such, he determines this is a priority. He talks to his leadership team and they determine that their goal is to have a validated process for responding to a data breach in 18 months. Wait, what — 18 months?!? Well, as with insurance, most of us don’t make one payment annually, we break it out over a manageable schedule and period of time. To be realistic, preparedness has to be approached similarly. Now, priority efforts may be addressed more aggressively, and some things taken much slower, but that is a decision that leaders need to weigh in on – informed by a sound understanding of the threat environment and based on a prioritized assessment of risks. For example, after the recent WannaCry outbreak, some leaders may be reassessing their patching processes and wanting to fast track and exercise new processes and procedures. Returning to our champion, Johnny develops a series of activities to plan, train staff, and exercise the data breach response plan, through a series of scheduled, progressive activities going from developing a plan, to conducting staff training, to a series of increasingly challenging exercises – a tabletop exercise, a drill, and a full-scale response – all completed within the specified 18-month period. Johnny documents his plan, gets leadership approval and resources, and executes, leading his team to the desired state of readiness by the required suspense. Good job, Johnny!

Operational Planning

This is the actual development of plans and procedures. There are different levels of planning and though they may sometimes be given different names, the four basic types of planning are: strategic, operational, tactical, and, contingency. Some may have additional steps, use different names, or stack them in a different order. For purposes of simplicity, we’re not going to address strategic planning, and for this discussion we’ll roll the rest up under operational planning – which in this context I mean as the development of plans and procedures. This is when the organization develops the plans and procedures that they will use to train their personnel and from which they will actually base their response actions. The National Incident Management System notes that, “All emergency management/response personnel and their affiliated organizations should develop procedures and protocols that translate into specific, action-oriented checklists for use during incident response operations.”

To develop his plans, procedures, and checklists, Johnny didn’t know where to start. So, he did the smart thing and looked for viable templates that he could work off, such as those provided by the Federal Trade Commission or the European Union Agency for Network and Information Security. He refined these plans to fit his organization, their people and capabilities. Along the way, Johnny also conducted several interviews to inform his draft plan. And, while we’re not at the part of this series addressing exercises yet, Johnny was. He even conducted a tabletop exercise to validate his draft plan! We’ll come back to that in part five of this series. When he was done, Johnny was able to provide his coworkers with a well thought out, validated data breach response plan and corresponding actionable checklists.

An important note, there are too many variables for any organization to address every possible threat or variation of an incident. In both physical and cybersecurity, and for pandemics and other threats — it is great to have detailed plans and protocols. However, no organization can get to a 100% solution for every situation. Having plans is important but so is building in flexibility and innovation. After hearing from some of the more experienced team members, Johnny developed a basic incident response plan, accepting that he and his co-workers would have to be able to adjust to the reality of events “on the ground.” Your plan is almost never going to be based on the exact situation you find yourself in. Plan well, be deliberate, but also be prepared for a little bit of backyard football, being able to make game time decisions when needed. Matt Stafford’s coaches don’t tell him to throw that sidearm ball, but sometimes, he has to adjust to get the ball in his receiver’s hands. Know the right form, but be ready to toss the sidearm when you have to.

In the next installment of this series, we’ll take a look at the next step in the Preparedness Cycle – organizing and equipping.

Preparedness & Cyber Risk Reduction Part One: Introduction to the Preparedness Cycle

Bad things happen. Whether we’re dealing with our personal or professional business, life seems to always have a variety of bumps and obstacles that pop up in our path. We should anticipate that these disruptions will arise and prepare ourselves to move through them as successfully and efficiently as possible while minimizing the impacts the disruptions cause. In dealing with the wide spectrum of threats that can cause operational disruptions to our organizations – regardless of whether they are health or natural catastrophes, terrorists or cybercrime – a key part of successfully overcoming the impacts of incidents is taking the time to properly prepare. Preparedness can be defined as a continuous cycle of planning, organizing, training, equipping, exercising, evaluating, and taking corrective actions to support effective incident response.

In today’s cyber threat environment, it seems many organizations are struggling to determine how to mitigate the array of cyber threats and associated risks they are facing. In a fast paced, frequently changing environment, one could be overwhelmed trying to determine how to prepare for and respond to the attacks and incidents that could arise. But alas! There is hope! While the Preparedness Cycle is often thought of in relation to “traditional” threats – hurricanes, explosives and earthquakes, for example – it is just as valid an approach to take in confronting cyber threats and works just as well to reduce the associated risks and impacts of such events.

But let’s back up. Threats, risks – what are we talking about? Malware, ransomware, cyberattacks, phishing, whaling (did you say whaling?), espionage, insider threats, denial of service, social media… what am I going to do with all these threats?! Or are they risks?

Let’s start with lexicon. Terms matter. So, let’s start with some basic definitions. I like references because then I can blame someone else for the typos… in 2010, the Department of Homeland Security’s Risk Steering Committee developed the “DHS Risk Lexicon” providing sound definitions for a number of key terms. Let’s look at the two most fundamental: Threats and Risks.

  • Threat is defined as a “natural or man-made occurrence, individual, entity, or action that has or indicates the potential to harm life, information, operations, the environment, and/or property.”
  • Risk is defined as the “potential for an unwanted outcome resulting from an incident, event, or occurrence, as determined by its likelihood and the associated consequences.”

As we try to understand our cyber threat environment, we have to gain an appreciation for the many occurrences, individuals, and entities that have the potential to cause harm. This can be developed in a number of ways and the means by which we gain a sound understanding of the threat environment and how to conduct a risk assessment could be entire blog series’ of their own. For today, we’ll just assume you’re maintaining threat awareness via great resources like SurfWatch Labs’ and Gate 15’s blogs and Twitter feeds … and that you’ve then assessed those threats in relation to your organizational interests and that you’ve developed a prioritized assessment of your risks.

No organization is able to specifically address every threat and risk, nor to address them all as thoroughly as we’d like. By prioritizing our risks, and recognizing that you only have limited time and resources to work with, you can then find ways to “get the most bang for the buck” in determining how to approach preparedness activities. Some risks, you will choose to simply accept. Some will get addressed via insurance. Others will be addressed by using the Preparedness Cycle and a deliberate process of planning, training, organizing and equipping, exercising and evaluating and improving. In the next few installments of this blog series, we’ll take a look at each one of these parts of the Cycle and ways you can progressively reduce your cyber risk via proper preparedness.