WSO Server Restoration January 2007

For up-to-date information on the restoration efforts and server situation, please see <http://wso.williams.edu/wso/services/restorejan07>.

What happened?

WSO's user and mail server experienced a severe crash on December 31, 2006. After installing a system security update, the server did not finish rebooting successfully. The server has been down or nonfunctional since then. Upon arriving on campus, we discovered that the RAID (the software system designed to prevent data loss due to the physical failure of one of the machine's hard disks) had been corrupted. After numerous troubleshooting attempts and support calls, we concluded that it was unrecoverable and restored from a backup. Unfortunately, and contrary to what the WSO staff thought, our most recent full backup dates to September 9, 2006. We are currently running with all normal services but with data from this backup. This means that all changes since September 9th have been lost permanently.

After further conversations with Apple support, WSO determined that the Xserve (the user server) had hardware issues with either the CPU or the logic board.

What was affected?

The following were affected:

user accounts (including passwords, email forwarding, and files)
organization websites
email lists
A very limited amount of email may have been lost on December 31. Since then mail to any address @wso.williams.edu may have bounced (the sender would receive an undeliverable mail message) due to Ursula's downtime. Mail sent between December 31 and January 3 may be delayed.

The following services were NOT affected:

WSO homepage and web services (e.g. Willipedia, Discussions, Announcements, etc.)
Databases

What does this mean for users?

All data on the server after the crash and restoration exist as they did on September 9. Any changes made after September have been lost permanently.

Any accounts created after September 9 no longer exist.
Mailing list subscriptions now stand as they did on September 9.
Passwords and email forwarding settings stand as they did on September 9.
Organization websites will look as they did on September 9. (Databases have not been affected).