Major Internet Outage

mgarbowski

Registered
Elite Donor
Donor
Seasoned Supporter
Jul 16, 2014
18,225
51,202
353
Queens, NY
mgarbowski.com
Anyone else suffering from this? My home internet has been off more than on for the last 24+ hours. Spectrum seems to be having trouble nationwide and other ISPs are also affected. I'm not sure why this isn't an enormous story.
68F9D3C4-9D3F-4216-8A12-94CB91F9879F.png
 
Spectrum outages, ConEd explosions, 911 down in WA last night...

Cyberattacks.

I'm semi joking, but let's not forget that just last March the city of Atlanta endured a pretty disruptive ransomware cyberattack.
https://en.m.wikipedia.org/wiki/2018_Atlanta_cyberattack

ETA: and let's not forget the natural gas explosions in northeastern MA this past fall.
 
Last edited:
In Mass, we are having sporadic 911 outages today. Yes, 911. In an emergency, if 911 is down, you have to hope you have the 10 digit number for the local fire or police department.
I'm not familiar with the infrastructure behind 911, so I don't know if its related to internet issues or not, probably not, but a big deal nonetheless.

Not sure if we should be blaming China, Russia, North Korea, or ourselves, but this is quite disruptive
 
  • Like
Reactions: mgarbowski
this reminds me of that one incident many years ago (2003) where the northeast had a huge blackout bc of some bug in the alarm software.
 
Spectrum outages, ConEd explosions, 911 down in WA last night...

Cyberattacks.

I'm semi joking, but let's not forget that just last March the city of Atlanta endured a pretty disruptive ransomware cyberattack.
https://en.m.wikipedia.org/wiki/2018_Atlanta_cyberattack

ETA: and let's not forget the natural gas explosions in northeastern MA this past fall.

Resident Con Edison employee! Turns out it wasn't a transformer explosion after all, but one massive, sustained arc flash. Which, I think, is even more terrifying, but *shrug* Regardless, we take cybersecurity very seriously from both technical and physical standpoints. We have never been breached and work hard to keep it that way. I work down the hall from some of our cybersecurity team. They're pretty great.

https://pix11.com/2018/12/27/new-yorkers-report-strange-light-over-the-city-power-outages/

Also, I don't believe the gas explosions in MA were related to cyber attacks either. While I can't speak for every utility, many do not have remotely accessible/controllable devices on their gas systems (at least not yet). Some, though, have invested in various automation tools, which is what happened in MA.

From my understanding, the MA explosions were caused by incomplete work procedures/human error. According to the NTSB, the utility's procedure for replacing old distribution mains did not mention anything about also replacing the automated pressure sensors/regulator lines (I do not believe these regulators are remotely accessible, but if someone knows more please share it). So when the new low pressure mains were installed without a new pressure sensor line, the automated regulators in the high pressure main registered zero pressure and starting pushing in more gas which over pressurized those low pressure mains leading to leaks and explosions.

https://www.ntsb.gov/investigations/AccidentReports/Pages/PLD18MR003-preliminary-report.aspx

None of this is to diminish the very real threat of cyber attacks on utilities. Just look at what happened in Ukraine a few years ago (https://en.wikipedia.org/wiki/December_2015_Ukraine_power_grid_cyberattack). This isn't the apocalyptic catastrophe that some people envision, but instead a little taste of what is possible. However, that does not mean that it is a conclusion to be jumped to after every event.

Just wanted to provide some context and hopefully stem the flow of paranoia/disinformation.
 
Resident Con Edison employee! Turns out it wasn't a transformer explosion after all, but one massive, sustained arc flash. Which, I think, is even more terrifying, but *shrug* Regardless, we take cybersecurity very seriously from both technical and physical standpoints. We have never been breached and work hard to keep it that way. I work down the hall from some of our cybersecurity team. They're pretty great.

https://pix11.com/2018/12/27/new-yorkers-report-strange-light-over-the-city-power-outages/

Also, I don't believe the gas explosions in MA were related to cyber attacks either. While I can't speak for every utility, many do not have remotely accessible/controllable devices on their gas systems (at least not yet). Some, though, have invested in various automation tools, which is what happened in MA.

From my understanding, the MA explosions were caused by incomplete work procedures/human error. According to the NTSB, the utility's procedure for replacing old distribution mains did not mention anything about also replacing the automated pressure sensors/regulator lines (I do not believe these regulators are remotely accessible, but if someone knows more please share it). So when the new low pressure mains were installed without a new pressure sensor line, the automated regulators in the high pressure main registered zero pressure and starting pushing in more gas which over pressurized those low pressure mains leading to leaks and explosions.

https://www.ntsb.gov/investigations/AccidentReports/Pages/PLD18MR003-preliminary-report.aspx

None of this is to diminish the very real threat of cyber attacks on utilities. Just look at what happened in Ukraine a few years ago (https://en.wikipedia.org/wiki/December_2015_Ukraine_power_grid_cyberattack). This isn't the apocalyptic catastrophe that some people envision, but instead a little taste of what is possible. However, that does not mean that it is a conclusion to be jumped to after every event.

Just wanted to provide some context and hopefully stem the flow of paranoia/disinformation.
Exactly the type of answer a company would spin after a cyber attack!


Just kidding :) an arc flash..... that’s crazy that it could go on that long without the fail-safe breakers tripping and terminating the circuit. Electricity is no joke. Thanks for the explanation!
 
Exactly the type of answer a company would spin after a cyber attack!


Just kidding :) an arc flash..... that’s crazy that it could go on that long without the fail-safe breakers tripping and terminating the circuit. Electricity is no joke. Thanks for the explanation!

My official title is Senior Media Spin Analyst =P

And yeah, I'm definitely curious to find out more technical details. I have some friends on the electric operations side (I work in IT) who might know more. I'm not back in the office until after the new year, but I'll try to find out more and share it if I can.
 
My official title is Senior Media Spin Analyst =P

And yeah, I'm definitely curious to find out more technical details. I have some friends on the electric operations side (I work in IT) who might know more. I'm not back in the office until after the new year, but I'll try to find out more and share it if I can.
They sent out a “Senior” level and not “Director” for the plausible deniability ..... it’s worse than we thought.... oh dear......
 
  • Like
Reactions: QuazyPat
According to a WSJ article, the 911 and internet outages were likely linked. CenturyLink (screw you Seattle! ) had an outage which they say was not related to any cyberattack.
 
  • Like
Reactions: mgarbowski
My official title is Senior Media Spin Analyst =P

And yeah, I'm definitely curious to find out more technical details. I have some friends on the electric operations side (I work in IT) who might know more. I'm not back in the office until after the new year, but I'll try to find out more and share it if I can.
Thanks for the info! I'd love to hear any more that you have about the arcing incident. My family and I evacuated our neighborhood (four blocks south of the Astoria ConEd plant) during it.