Hi list
We're having severe trouble with an old installation of checkpoint's fw-1
(ver. 4.1 sp6) on nokia ipso 3.5-FCS14. ("upgrade", i hear you say. yes,
we're at it, but it will take more time than we have to solve this)
Symptoms: At random times the box gets 100% loaded, i.e.
- a constant 0% cpu idle time reported by vmstat (with 99% in system, 1% in
user mode)
- shell is almost unresponsive (charactes on prompt echoed at a rate of one
per 30 seconds, commands never returning, login taking about 10 minutes)
- box practically stops forwarding any traffic
- interface throughput drops to zero on ALL interfaces (measured in retrospect
on switch and in ipso interface statistics)
- box takes about 3 seconds to reply to icmp echo-request
- box stops sending out OSPF and VRRP packets, resulting in failovers and OSPF
recalculations, then occasionally sends one out, resulting in another session
of failovers and routing convergences.
Then, after 10-30 minutes, suddenly all returns to normal (80% idle), nothing
in the logs except for the OSPF messages from ipsrd.
On one occasion I managed to get a "ps auxw" through, and it showed monitord
using 10% cpu and fwd using 8% cpu, all other processes were using below
0.1%. But I assume this only showed userspace processes.
From all that I see, it's the kernel using all resources, or a hardware
fault.
Now: is anyone aware of a tool or method to get more information about what is
causing this load? Has anyone heard of an attack that might cause these
symptoms? Can anyone remember a bug in that version of fw-1/ipso that might
result in this behaviour? Generic observations?
I'd be grateful for any pointer.
thanks
/markus
pgpoNqF05KG8E.pgp
Description: PGP signature
