When a firewall starts refusing new connections while its traffic graph still reads nearly idle, its state table has filled. That is a protocol attack, the exhaustion class a thorough DDoS test is built to exercise between the volumetric and application layers. It does not saturate bandwidth or abuse application logic; it weaponizes the bookkeeping that stateful devices keep for every connection they see.
How a protocol attack works
Every device that tracks connections holds a finite table. A firewall keeps a connection-tracking entry per flow, a load balancer keeps a session, and the origin kernel keeps a backlog slot for each half-open handshake. Each entry costs a little memory and stays allocated until the connection closes or times out. Fill the table and the device has nowhere to record the next connection, so it refuses it, including the legitimate one.
The vectors differ only in which table they aim at. A SYN flood sends the opening packet of a handshake and never the final acknowledgement, so half-open records pile up in the kernel backlog. ACK, RST, and FIN floods force per-packet state lookups on connections that do not exist. Connection floods complete real handshakes and then hold them, one conntrack entry at a time. Fragmentation attacks make the reassembly buffer do the work instead.
The tell is the unit. Protocol attacks are counted in packets per second, not bits, because it is the arrival of another packet needing another table entry that does the damage, not the bandwidth those packets carry. A few hundred thousand tiny packets a second is trivial traffic on a bandwidth graph and more than enough to exhaust a mid-range firewall.
Why it matters under attack
This is the class that provisioning bandwidth does nothing to stop. A link can have ample capacity and the service still falls when the smallest state table in the path fills, and that table often belongs to a middlebox in front of the origin rather than the origin itself. The device that fails first is rarely the one anyone was watching.
The graph lies, which is the dangerous part. Link utilization reads normal, packet counters look ordinary, and the first symptom is connections being refused. A team instrumented only for volume can watch every dashboard stay green while the service is already unreachable, because the resource that ran out was memory in a table, not room on the wire.
There is a bitter irony in which device fails. The stateful security appliances bought to defend the perimeter, the firewall and the inline load balancer, are themselves the ones holding the tables an attacker aims to fill, so the more state a box tracks the more attractive a target it becomes. This is also why protocol vectors are a staple of multi-vector floods: they cost almost nothing in bandwidth, so an attacker can run a SYN flood underneath a volumetric one and exhaust the state tables long before the pipe fills, forcing the defender to solve two different failures at once.
What a DDoS test measures
A protocol test characterizes the packet-rate ceiling of every stateful device in the path, not just the origin, because the weakest table sets the limit for the whole chain. It drives the handshake and per-packet vectors and watches which table gives way first, the layer of first failure that a datasheet never tells you.
Then it confirms the mitigations hold: whether SYN cookies let the kernel answer a handshake without reserving a backlog slot, whether conntrack and accept-queue limits are tuned for the real connection rate, and whether an upstream device sheds the spoofed traffic before it reaches the tables it would otherwise fill. Separating this state-exhaustion layer from the volumetric and application classes is the structuring principle behind DDoS resilience testing.