Truck Forum was down

Admin · 03-27-2006, 06:48 AM

For approx 26hrs TruckForum.org and related websites were down due to a catastrophic failure at the datacenter that houses our servers. This is one of the largest (if not the largest) datacenters in the US (and a very large company).

It was basically due to the error of a low level tech at the datacenter (not an employee of mine, but of the datacenter). They were supposed to be adding a hard drive to the server and accidentally formatted/wiped the WRONG drive. It took getting the CEO of the company directly involved, calling in top recovery experts and my staying up most of the 48 surrounding hours (2.5 hrs sleep).

We are back and with 0 data loss (keeping fingers crossed!).

Going to sleep now.

-Victor/Admin

Admin · 03-27-2006, 03:58 PM

(Woke up, will probably nap again in a bit)

FYI even if the entire server was destroyed in a fire/flood/earthquake/etc we have regular off-site backups of everything at a location hundreds of miles away. This occurs at least every 3 days (soon to change to every day). So worst case scenario ever, we could lose a few days of posts and be down for a day or two switching to another server. We actually have redundant hardware on the server so even if a drive fails the server keeps running fine. This was a freakish human error (and yes they are in trouuuubbbblllle).

-Victor/Admin

03-27-2006, 06:48 AM	#1
Admin Administrator Join Date: Jul 2004 Posts: 189	Truck Forum was down For approx 26hrs TruckForum.org and related websites were down due to a catastrophic failure at the datacenter that houses our servers. This is one of the largest (if not the largest) datacenters in the US (and a very large company). It was basically due to the error of a low level tech at the datacenter (not an employee of mine, but of the datacenter). They were supposed to be adding a hard drive to the server and accidentally formatted/wiped the WRONG drive. It took getting the CEO of the company directly involved, calling in top recovery experts and my staying up most of the 48 surrounding hours (2.5 hrs sleep). We are back and with 0 data loss (keeping fingers crossed!). Going to sleep now. -Victor/Admin

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Help a Student..Take a Truck Survey	ClassSurvey	General Truck Forum	2	12-08-2007 06:51 PM
» FORUM Killer Truck Rolls through Downtown Sofia (Novinite.com)	trucknews	Truck News (headlines)	0	02-02-2005 03:30 AM
» FORUM Driver of Knyazhevo Killer Truck Out on Bail (Novinite.com)	trucknews	Truck News (headlines)	0	11-15-2004 01:30 PM
TEN Magazines(TM) Announces Ford Truck World, Dodge Truck World, and Chevy Truck World Now Published Quarterly With (The Auto Channel)	trucknews	Truck News (headlines)	0	10-29-2004 05:30 PM

03-27-2006, 03:58 PM	#2
Admin Administrator Join Date: Jul 2004 Posts: 189	(Woke up, will probably nap again in a bit) FYI even if the entire server was destroyed in a fire/flood/earthquake/etc we have regular off-site backups of everything at a location hundreds of miles away. This occurs at least every 3 days (soon to change to every day). So worst case scenario ever, we could lose a few days of posts and be down for a day or two switching to another server. We actually have redundant hardware on the server so even if a drive fails the server keeps running fine. This was a freakish human error (and yes they are in trouuuubbbblllle). -Victor/Admin