Black Friday

From Hattrick

Friday November 28, 2003 goes down in Hattrick history as "The Black Friday". The site was down for 28 hours and the Hattrick Team eventually were forced to do what had never been done in six years of operations: roll back the time of the Hattrick Universe to a backup taken previous morning.

The reason was two consecutive crashes of the hard disk cabinet that contained the main database of Hattrick. Further similar crashes in December 2003 made Hattrick change hard disk manufacturer from Dell to Hitachi. Even though Dell initally denied the existence of a problem, they later confirmed it and corrected the bug in February 2004.

On Tuesday, July 11, 2006, during a storage upgrade, the master Hattrick database became corrupted. After limping along well into Wednesday, the Hattrick Team was forced to take the site down for more than two days to do recovery. When the damage proved too great to fix, for the second time in the history of the game, the team was forced to restore from a backup.

On Monday, August 23, 2010, Hattrick was taken down due to a buggy firmware upgrade introduced by Hattrick's disk cabinet supplier during routine maintenance forcing a full restore from back up. (Incident in progress)