This morning, Three (3) VMWare hosts (UISESX2, UISESX6, and UISESX8) had the following warning message:

Virtual machine creation may fail because agent is unable to retrieve VM creation options from the host (vmodl.fault.HostCommunication).

While investigating this error, UISESX2 disconnected from the vCenter server preventing us from managing the host and the 18 VM’s on this host. Research pointed to restarting the Management Agents from the console but this process hung at “running usbarbitrator restart” preventing this from being a viable option. Upon further investigating both of these issues, it was evident that the host would need to be rebooted before it could be reattached to the vCenter server. Once UISESX2 was rebooted, the host was reattached to vCenter and each of the VM’s were started.

On the morning of December 28, 2012, the EqualLogic SAN will be upgraded to the to latest firmware version (6.0.2) as part of EqualLogic’s quarterly maintenance release cycle. This release includes maintenance fixes for enhanced stability and performance of the EqualLogic SAN.

This upgrade should not effect users or systems hosted on the SAN.

The software repository on UISDATA1 now includes the latest two (2) versions of McAfee VirusScan for Mac. Either version is able to be installed on any university provided workstations and servers. This same software is also available for download by all  UIS faculty, staff and students though the university’s Software WebStore.

Updates to the 5 hosts within the UISVS1 datacenter have completed successfully. These hosts include UISESX1, UISESX2, UISESX3, UISESX6, and UISESX8.

The University of Illinois Springfield is a National Cyber Security Awareness Month (NCSAM) Champion.

http://www.staysafeonline.org/ncsam/champions/all-champions/

On Wednesday, February 15, 2012, at 10AM, I will be expanding the scope of managed systems on the “updated” ePolicy Orchestrator server (UISEPO1) to include the workstations used by ITS staff. In order to achieve this, a new McAfee agent will be pushed to these machines enabling this new server to monitor the antivirus software. This change should not be noticeable nor require a system reboot; however, if you do notice anything and feel that it is related to this update, please contact me immediately.

Thank you,
Clayton

The DoList service which controls the email distribution lists (ENS-L, CINRC-L, etc.) stopped on it’s on after running out of space. Once space was freed up, the service was allowed to start.

Last Friday afternoon around 2pm, the VMware storage volume Datastore10 was inadvertently reformatted while trying to attach the new BlackBoard Oracle server to the SAN. This datastore was one of several volumes created for housing virtual machines, and was hosting 15 virtual machines at the time. These machines and their description are:

UISMailFE1 – WebMail2003 load-balancing frontend server
UISSoftricity2 – Softricity server used by Educational Technology
OTEL11 – WebBoard5 server
UISFLASH1 – Adobe Flash server user by Educational Technology
USIBBOramanage1 – Management server to manage BlackBoard’s Oracle servers
UISMoodle16 – Moodle server
UISLimbo1 – Server used for security violations on campus
UISCRD1 – Call Detail Records Server (under construction)
UISASALog1 – Server to collect firewall logs (under construction)
UISMTDC1 – Domain controller for Test Domain
UISMTMail1 – Mail server in the Test Domain
UISMTDC2 – Secondary Domain controller for Test Domain
UISMTMAIL4 – Secondard mail server in the Test Domain
UISADTDC1 – Domain Controller  for the UISAD Test Domain

Thus far, we have been able to successfully restore OTEL11, UISMoodle16, and UISLimbo1 from backups. The Exchange 2003 frontend server and the test servers are not critical and do not negatively impact operations, so they be recreated if and when they are needed again. Servers UISBBoramanage1, and UISFLASH1 are in the process of being recovered from backups, and we are still investigating the recovery options for the Softricity server.

The root cause of the problem has been determined and steps have been taken to prevent similar problems in the future.

The last of the virtual machines that resided on the 2.5 ESX hosts have been archived and the hosts themselves have been taken offline. These hosts will be removed and scrubbed before being surplussed.

Thank you for your patience while the various VM’s where migrated off these hosts and into the new NGDC.

Kevin Zepp is December 2011’s employee of the month.

Congratulations!