Page 1 of 1

Downtime Report: Feb 6th, 2015

Posted: 07 Feb 2015, 07:33
by MadCamel
As you may have noticed we have had several hours of downtime today. Things are now back up and everything looks nominal.

For those that are curious, here's a rundown of tonight's events.

Whenever a TMWC member with access to the TMW server comes or goes, I make it a habit to take a backup snapshot of the TMW machine. Unfortunately today this did not work so well. The virtual machine management software glitched and appeared to have completely wiped TMW's disks. Ouch!

Thankfully after a considerable amount of high-stress work I was able to recover the data. It wasn't actually wiped from the physical disks, the logical volume manager's references to it had just been corrupted due to a kernel bug. Extracted the data from the storage array manually and cloned it back in to a new (unglitched) volume. Rock on.

I'd like to thank Nicotine and Caffeine for helping me through these tough times.

Re: Downtime Report: Feb 6th, 2015

Posted: 07 Feb 2015, 08:03
by deepthought
Woa!! Dodged a bullet there. Nice work Mad Cammel.

Re: Downtime Report: Feb 6th, 2015

Posted: 07 Feb 2015, 08:33
by o11c
MadCamel wrote:I'd like to thank Nicotine and Caffeine for helping me through these tough times.
I've never heard of anyone with those nicks.

Re: Downtime Report: Feb 6th, 2015

Posted: 07 Feb 2015, 15:33
by Meicchi
o11c wrote:
MadCamel wrote:I'd like to thank Nicotine and Caffeine for helping me through these tough times.
I've never heard of anyone with those nicks.
Maybe this old guy.

Re: Downtime Report: Feb 6th, 2015

Posted: 07 Feb 2015, 16:36
by Crush
o11c wrote:
MadCamel wrote:I'd like to thank Nicotine and Caffeine for helping me through these tough times.
I've never heard of anyone with those nicks.
I think he is referring to this guy and this guy. I don't like the first, but I couldn't get much work done without the help of the latter.