The network operations center observed that the available memory on the primary core router was draining steadily away. Over three days the usage climbed from 40 percent to 85 percent; were it to reach 100 percent, the router would crash and bring a major network outage down upon us.

My task was to name the process consuming the memory and to steady the router before it ran dry.

I logged into the router’s command-line interface and, by way of diagnostic commands, surveyed the memory allocation of each system process. The routing protocol process for the border gateway protocol (BGP) was consuming nearly all of what remained. Turning to the release notes for the router operating system, I found a known bug: a memory leak provoked when a specific type of malformed BGP route is received. I applied a route filter to block the malformed routes.

The memory usage stopped climbing the moment the filter was in place, yet the leaked memory was not recovered of itself. I scheduled a maintenance window in the night to restart the routing process; after the restart, usage returned to 40 percent and the network remained stable. A week later I upgraded the router operating system, fixing the bug for good.