Difference between revisions of "WSO Server Restoration January 2007"

m (WSO & history)
 
Line 1: Line 1:
 
[[Category:WSO]][[Category:History]]
 
[[Category:WSO]][[Category:History]]
−
<div style="margin: 10px 0px; padding: 0px 8px 8px 8px; border: 1px solid #495; background: #9da; float: left;">
 
−
'''Server Status: ''Working'' '''
 
−
</div>
 
−
<div style="clear: both;"></div>
 
−
As of 10:20 PM Wednesday January 3, 2007.
 
−
 
−
We are currently running from a copy of the backup, with Irene (our iMac) standing in for Ursula (our Xserve). All services should be functioning "normally" from the backup (in their September 9th state), but they may be slower than usual, though Irene hasn't had much trouble with the load yet.
 
−
 
−
 
  
 +
For up-to-date information on the restoration efforts and server situation, please see <http://wso.williams.edu/wso/services/restorejan07>.
  
 
== What happened? ==
 
== What happened? ==
  
 
WSO's user and mail server experienced a severe crash on December 31, 2006. After installing a system security update, the server did not finish rebooting successfully. The server has been down or nonfunctional since then. Upon arriving on campus, we discovered that the RAID (the software system designed to prevent data loss due to the physical failure of one of the machine's hard disks) had been corrupted. After numerous troubleshooting attempts and support calls, we concluded that it was unrecoverable and restored from a backup. Unfortunately, and contrary to what the WSO staff thought, our most recent full backup dates to September 9, 2006. We are currently running with all normal services but with data from this backup. ''This means that all changes since September 9th have been lost permanently.''
 
WSO's user and mail server experienced a severe crash on December 31, 2006. After installing a system security update, the server did not finish rebooting successfully. The server has been down or nonfunctional since then. Upon arriving on campus, we discovered that the RAID (the software system designed to prevent data loss due to the physical failure of one of the machine's hard disks) had been corrupted. After numerous troubleshooting attempts and support calls, we concluded that it was unrecoverable and restored from a backup. Unfortunately, and contrary to what the WSO staff thought, our most recent full backup dates to September 9, 2006. We are currently running with all normal services but with data from this backup. ''This means that all changes since September 9th have been lost permanently.''
 +
 +
After further conversations with Apple support, WSO determined that the Xserve (the user server) had hardware issues with either the CPU or the logic board.
  
 
== What was affected? ==
 
== What was affected? ==
  
−
The following are affected:
+
The following were affected:
  
 
* user accounts (including passwords, email forwarding, and files)
 
* user accounts (including passwords, email forwarding, and files)
Line 25: Line 19:
  
  
−
The following services have NOT been affected:
+
The following services were NOT affected:
  
 
* WSO homepage and web services (e.g. Willipedia, Discussions, Announcements, etc.)
 
* WSO homepage and web services (e.g. Willipedia, Discussions, Announcements, etc.)
Line 32: Line 26:
 
== What does this mean for users? ==
 
== What does this mean for users? ==
  
−
All data on the server now exist as they did on September 9. Any changes made after September have been lost permanently.
+
All data on the server after the crash and restoration exist as they did on September 9. Any changes made after September have been lost permanently.
  
 
* Any accounts created after September 9 no longer exist.
 
* Any accounts created after September 9 no longer exist.
Line 39: Line 33:
 
* Organization websites will look as they did on September 9. (Databases have not been affected).
 
* Organization websites will look as they did on September 9. (Databases have not been affected).
  
−
== How can I get my files and settings back? ==
+
== More Info ==
−
 
 
−
'''It is now fine to proceed here.'''
 
−
 
 
−
=== Files ===
 
−
 
 
−
Restore any personal backups. We apologize for the inconvenience that the outdatedness of our backup may cause for users.
 
−
 
 
−
=== Accounts ===
 
−
 
 
−
==== New Accounts ====
 
−
For accounts created after September 9, 2006, please visit the accounts page at <http://wso.williams.edu/accounts/> to sign up for an account (again). '''We have not had the opportunity to test the functionality of this after the restoration of our backup.'''
 
−
 
 
−
==== Passwords ====
 
−
 
 
−
If you need to change the password on your WSO account, use the tools described here.
 
−
 
 
−
* Change your password: http://wso.williams.edu/password
 
−
* Get a temporary password mailed to you: http://wso.williams.edu/reset_password
 
−
 
 
−
==== Email Forwarding ====
 
−
 
 
−
Change your email forwarding settings at https://wso.williams.edu/updateinfo.html
 
−
 
 
−
=== Mailing Lists ===
 
−
 
 
−
If you need a mailing list owner password reset or a mailing list recreated, please email <email>restoration@wso.williams.edu</email>. If you need to resubscribe to (or unsubscribe from) a list, please visit http://wso.williams.edu/mailman/listinfo/listname, where ''listname'' is the name of the list. If this page does not exist, please wait until the list owner for that list has recreated the list.
 
−
 
 
−
== Reporting problems ==
 
−
 
 
−
Please report any other problems, updates, or changes you need WSO to make by sending email to <email>restoration@wso.williams.edu</email>. Also feel free to offer your help.
 
−
 
 
−
We apologize for any inconvenience caused by data loss etc. We ask for your patience as we work to restore our services as best as possible.
 
−
 
 
−
== What is WSO doing? ==
 
−
 
 
−
We are currently running a different machine (Irene the iMac) as a replacement for our user server (Ursula). It has files from the September 9, 2006 backup. Changes by users will be retained here (barring any further disasters), but '''we strongly encourage users to make and keep backups of their personal data or organization website data. We do not have backups set up for the temporary user machine yet.'''
 
−
 
 
−
Now that we have a working replacement for Ursula, we will continue working on issues on Ursula. In the next few days we will be working on rebuilding Ursula from scratch to have a machine that we know well at our disposal. When this process is complete, we will switch to the new Ursula. At this point, a short downtime for copying the most recent user data will be needed.
 
  
−
Thank you,<br />
+
For up-to-date information on the restoration efforts and server situation, please see <http://wso.williams.edu/wso/services/restorejan07>.
−
WSO Staff
 

Latest revision as of 10:34, January 29, 2007


For up-to-date information on the restoration efforts and server situation, please see <http://wso.williams.edu/wso/services/restorejan07>.

What happened?

WSO's user and mail server experienced a severe crash on December 31, 2006. After installing a system security update, the server did not finish rebooting successfully. The server has been down or nonfunctional since then. Upon arriving on campus, we discovered that the RAID (the software system designed to prevent data loss due to the physical failure of one of the machine's hard disks) had been corrupted. After numerous troubleshooting attempts and support calls, we concluded that it was unrecoverable and restored from a backup. Unfortunately, and contrary to what the WSO staff thought, our most recent full backup dates to September 9, 2006. We are currently running with all normal services but with data from this backup. This means that all changes since September 9th have been lost permanently.

After further conversations with Apple support, WSO determined that the Xserve (the user server) had hardware issues with either the CPU or the logic board.

What was affected?

The following were affected:

  • user accounts (including passwords, email forwarding, and files)
  • organization websites
  • email lists
  • A very limited amount of email may have been lost on December 31. Since then mail to any address @wso.williams.edu may have bounced (the sender would receive an undeliverable mail message) due to Ursula's downtime. Mail sent between December 31 and January 3 may be delayed.


The following services were NOT affected:

  • WSO homepage and web services (e.g. Willipedia, Discussions, Announcements, etc.)
  • Databases

What does this mean for users?

All data on the server after the crash and restoration exist as they did on September 9. Any changes made after September have been lost permanently.

  • Any accounts created after September 9 no longer exist.
  • Mailing list subscriptions now stand as they did on September 9.
  • Passwords and email forwarding settings stand as they did on September 9.
  • Organization websites will look as they did on September 9. (Databases have not been affected).

More Info

For up-to-date information on the restoration efforts and server situation, please see <http://wso.williams.edu/wso/services/restorejan07>.

Retrieved from "https://wso.williams.edu/wiki/index.php?title=WSO_Server_Restoration_January_2007&oldid=14203"