Datastore 10 Issue

Last Friday afternoon around 2pm, the VMware storage volume Datastore10 was inadvertently reformatted while trying to attach the new BlackBoard Oracle server to the SAN. This datastore was one of several volumes created for housing virtual machines, and was hosting 15 virtual machines at the time. These machines and their description are:

UISMailFE1 – WebMail2003 load-balancing frontend server
UISSoftricity2 – Softricity server used by Educational Technology
OTEL11 – WebBoard5 server
UISFLASH1 – Adobe Flash server user by Educational Technology
USIBBOramanage1 – Management server to manage BlackBoard’s Oracle servers
UISMoodle16 – Moodle server
UISLimbo1 – Server used for security violations on campus
UISCRD1 – Call Detail Records Server (under construction)
UISASALog1 – Server to collect firewall logs (under construction)
UISMTDC1 – Domain controller for Test Domain
UISMTMail1 – Mail server in the Test Domain
UISMTDC2 – Secondary Domain controller for Test Domain
UISMTMAIL4 – Secondard mail server in the Test Domain
UISADTDC1 – Domain Controller  for the UISAD Test Domain

Thus far, we have been able to successfully restore OTEL11, UISMoodle16, and UISLimbo1 from backups. The Exchange 2003 frontend server and the test servers are not critical and do not negatively impact operations, so they be recreated if and when they are needed again. Servers UISBBoramanage1, and UISFLASH1 are in the process of being recovered from backups, and we are still investigating the recovery options for the Softricity server.

The root cause of the problem has been determined and steps have been taken to prevent similar problems in the future.





Leave a Reply