Cordgrass IP: degraded routing through one upstream network
Resolved. Loss returned to zero at 16:47. The upstream network confirmed a faulty line card at 17:00 and replaced it. We have restored its normal preference.
to 17:00 EDT
NOC, 24 hours SEC1 +1 201 016 2747 NWK1 +1 973 018 5536
Status
21 incidents and notices since our status page started in November 2019. Months with nothing to report are left out.
Resolved. Loss returned to zero at 16:47. The upstream network confirmed a faulty line card at 17:00 and replaced it. We have restored its normal preference.
to 17:00 EDT
Resolved. The road is clear. Gates 6 and 7 have been stood down and the south entrance is open. The water reached 7.1 ft, 9.7 ft below the finished floor of Halls 1 to 3.
to 11:15 EDT
Completed. All five B-side modules were updated. No transfers to bypass mains occurred.
Resolved. Hall 2 has been within its normal range since 15:23. Chiller 3 remains out of service while the condenser tubes are cleaned; the plant is at N+1 until it returns.
to 15:41 EDT
Resolved. Sign-in has worked normally since 11:06. We have added the certificate to the monitoring that should have warned us thirty days ago.
to 11:06 EDT
Resolved. The readers came back at 07:57 after the access-control panel was restarted. Hall readers were not affected.
to 07:57 EST
Resolved. Feeder 2 was restored at 08:51 and the site is back on its normal configuration.
to 09:02 EST
Resolved. Both sites have been fully staffed since 11:30 and the queue is clear.
to 14:00 EST
Completed. All ten enclosures are commissioned and every set is back in service.
Completed. All seven gates were deployed and sealed in 2 hours 44 minutes. Gate 4 needed a replacement seal, which has been fitted. The south entrance is open again.
Post-incident report. Our report on this incident, PIR-2025-03, is published. It covers the cause of the fire, why the chiller shedding sequence let Hall 2 overheat, what we have changed and what we owe the customers affected.
to Jun 25, 2025 · 09:10 EDT
Resolved. Heightened readiness has ended. The utility outage at SEC1 on June 24 is reported separately.
to Jun 26, 2025 · 20:00 EDT
Resolved. Upstream filtering took effect at 21:30 and loss has returned to zero.
to 21:30 EDT
Resolved. The portal and ticketing have worked normally since 11:05. Requests made by telephone in the meantime have been entered as tickets.
to 11:05 EDT
Completed. The barrier passed its acceptance test. All seven gates deployed in 3 hours 52 minutes on this first attempt, and gates 1 and 2 held water to 19.5 ft with seepage inside the allowance.
Completed. Hall 4 passed its integrated systems test and is ready for customers.
Resolved. Air quality has returned to normal and both plants are back in their usual modes.
to Jun 9, 2023 · 12:00 EDT
Resolved. Utility power has been stable since 18:50 and the site transferred back at 19:31.
to 19:44 EDT
Resolved. The roads have reopened and the gates and dock barrier have been stood down. The water at SEC1 reached 8.2 ft, 8.6 ft below the floor of the halls.
to Sep 2, 2021 · 15:00 EDT
Resolved. Utility power returned at 21:36 and NWK1 transferred back at 21:58, after 9 hours and 11 minutes on generators. We used 11,900 gallons of fuel and have reordered it.
to 22:01 EDT
Resolved. The manufacturer replaced a failed power board and the module is back in service.
to Nov 13, 2019 · 16:30 EST