Summary
When looking at the number of concurrent connections on an Exinda under the System graphs, it is useful to be able to determine whether a high number of connections are normal for the environment the Exinda is located in.Overview
Looking at the number of Concurrent Connections on the Monitor > Systems page, under the "Connections" tab will show how many connections are initiated through the Exinda. This graph shows two separate metrics - the number of concurrent connections at any time in the time period, and the number of new connections per second.The concurrent connections are the total number of distinct flows through the Exinda. For example, even though a single webpage is loaded, there are also requests for images and for embedded content going along with it. The Exinda counts each of these as a distinct 'connection', so loading a single webpage might actually create 10 new connections. This is not just for HTTP - this is all network connections, so for items such as SMB, print sharing, DNS, Active Directory, etc, they will all register their own connections.
The number of new connections per second is just as it seems, the number of new TCP/UDP connections registered on the Exinda per second.
The number of connections can be estimated through looking at the number of flows on reporting graphs. While it will not show all the flows, based on the reporting limits set during setup, it will show top applications and the number of flows associated with them, if looking at the Applications reporting. Similarly, the number of flows per host can be seen when looking at the Hosts reporting.
Connections, as mentioned, can come from all users, all types of network traffic, at all times. While there are certain things to watch out for, seeing a high number of concurrent connections is not inherently a dangerous thing. When thinking that there is a problem on the network, look at the following:
- An extremely high spike in new connections per second, way beyond normally established traffic patterns. If the normal rate if 1000 and suddenly the rate jumps to 300000, it is indicative of host(s) sending or receiving a large number of connections.
- A rhythmic spike in connections beyond the norm. This includes a 'spike' of connections per second, or concurrent connections at a regular interval, ie every hour, or every day at a specific time, if it hasn't been the trend in history (or has an unexplained start)
Cause
In the first case, if there is a large spike of new connections per second, even if they don't translate to 'concurrent connections' (as some might be aborted, or refused), it can be indicative of a network attack or a port scanner working on the network. These connections are active TCP connections being started (with a packet called a SYN). The Exinda marks this as a connection regardless whether the originating host gets an ACK. A common type of Denial of Service attack is a SYN Flood, where SYN packets are sent to devices at a rapid rate to overwhelm them. The Exinda appliances are not immune to this like security devices can be. Seeing a sharp spike of new connections per second could indicate that this type of attack is occurring.If a rhythmic spike is occurring beyond the norm, this could be indicative of a process, scheduled job or something else on a timer that is scheduled to run every X minutes, corresponding to the spike in connections. This can be something such as SNMP polling of devices, a process phoning home with a lot of connections, a network batch job, etc. There are plenty of options. While it is difficult to tell from the historical data which machine might be having these types of job, the Hosts graph can show the total number of flows. Isolating that time range to the time of the increased number of connections can help narrow it down further.
Resolution
If a suspected attack is occurring on the network, there is not much the Exinda can do to handle it. It is not a security device; best practices is to put the appliance behind a firewall so that these attacks can be mitigated by a security device. Additional logging on the Exinda will also help determine whether it's an attack directed at the Exinda appliance itself.?In the second case, the resolution would be to stop the job of the process in order to attempt and smooth out those spikes. If that works, and it is known not to be malicious traffic, it is possible to reenable it and if possible, create an application definition for it to more carefully monitor it in the future.