The Dangerous Questions IT Never Wants to Hear

Being able to deliver a rapid response to problems is a core need for the modern enterprise. While many network “issues” can suffer a delay in response time, true problems such as security breaches and network outages cannot. Let’s look at two situations that require a rapid response, five questions that IT never wants to hear, and one example of a solution that IT can implement.

In the case of a suspected security breach, you need to know if you were actually breached or not, where the breach occurred, and what was compromised. According to the 2016 “Verizon Data Breach Investigations Report” (DBIR), almost 68% of breaches happen over the course of several days, so a rapid response to security threats can definitely help minimize the cost of a breach.

If you have just been breached, this is not a situation in which there is nothing you can do. The faster you isolate the attack vector, the faster you can limit the amount of damage to your business. The key is that you need to know right away when you are breached. Unfortunately, the Ponemon Institute’s 2015 Cost of Cyber Crime Study shows that it is actually taking businesses longer to resolve cyberattacks. It now takes an average of 46 days to resolve a cyberattack, which is one day longer than it took last year. This represents an increase of 30% over the last 6-year period and results in a corresponding increase in the cost for a cyberattack.

The situation is also serious for network outages. According to the 2016 “Cost of Data Center Outages” study conducted by the Ponemon Institute, the average cost of a data center outage is $740,357 and lasts for about 95 minutes. This results in a cost of approximately $7,793 per minute of downtime. A rapid response is needed in this situation as well to control costs and keep your mean time to repair as short as possible.

In both situations, you need information and you need it fast. The question is, where and how do you capture the critical data that you need and then how fast can you do it? 
While time is money, time also affects customer satisfaction and reputations (especially IT department reputations).

When a crisis happens, whether it’s a cyberattack, component failure, or application failure, IT needs to quickly respond to the problem. This includes using diagnostic tools to analyze the problem.

If the tools aren’t already available in the network, or if they need to be moved, the IT engineer is faced with the five following questions that need an immediate response:

1. Will this require approval?

2. Where should I insert the tools?

3. Will the insertion of the tools cause other problems?

4. Will the tool data actually be useful?

5. Will the data I collect even be accurate?

Change board approvals require time, and, even in emergency situations, this can take hours. If it is not an absolute emergency (or at least not perceived to be an absolute emergency), the delay can be weeks. Even if you have test equipment on hand, you will face a time delay to get approval to modify the network.

“Where should I insert the tools?” is the next common question. This often becomes an architectural exercise to determine where to get the correct information. In an emergency, you may not have the time to sit down and create a visibility architecture—it needs to already be in place. Otherwise, it is a guessing game of where to collect the data.

The next question is, “Will the insertion of the tools cause other network problems?” This is especially important when inserting inline tools that affect all network data from that point on. The last thing you want is to break something else.

Another concern is whether the tool data will be useful. Different threat vectors can hide their tracks and even delete data after the attack is over. So, are the tools you plan to insert in the network going to capture the critical data needed? If the attack is ongoing, then the answer may be yes. If not, the best case scenario might be to sift through log data instead of inserting new tools into the network.

The fifth fundamental question is whether the data collected is even going to be accurate. For out-of-band monitoring situations, if you’re not using TAPs (Test Access Points), then SPAN (Switched Port Analyzer) ports will only give you summarized data—not the full data. This means that critical (bad) data right before an incident happens is probably omitted from the SPAN data, and you may not have the data you need to accurately diagnose the source of a problem/attack. SPANs also have the lowest priority for data-switching functionality. Therefore, if the switch is running at full processor capability (due to a DDOS attack), packets on the mirroring port will get dropped and you will miss important data.

The most cost-effective solution to these issues is to include a visibility architecture in your network design. A visibility architecture is simply a coherent plan about how to gather monitoring and security data from your network. It typically consists of TAPs, network packet brokers, and monitoring/security tools that are arranged in key positions across your network to collect the type of data you want. This helps maximize monitoring effectiveness by ensuring proper access to the data you need, when you need it. As a bonus, once you have designed and implemented a visibility architecture, there are typically no further change board approvals or other delays in your acquisition of monitoring data. This gives you access to the data you need 24x7. n