03300 885 250

Technical Infrastructure Status

We believe in full transparency, everything you see here is 100% live.
RESOLVED
This announcement has been resolved, no further updates are expected.
Data migration
As many on this server will know we have recently been suffering some stability issues on this machine. Despite best efforts to resolve the problem it has continued and looks like it is a case of hardware getting old so we are planning to migrate all data to a newer, more powerful machine this week.

We had planned to do this within our weekly maintenance sessions on Friday but due to the urgency of this situation we are bringing it forward with the work scheduled for the night of Wednesday 5th May (tomorrow night).

This procedure is something we are very familiar with and involves syncing all data to the new machine while everything remains online from the old server. After the first sync (which can take some time) another, much quicker sync will be done to catch any new or changed files with the final stage being to simply turn the old machine off and bring up the same IP's on the new machine. All IP numbers, user accounts, files, databases and everything else will remain exactly the same and no clients need to make any changes or do anything for this work schedule.

This announcement will remain updated.
Updated by Chris J. on 4th May 2010 @ 13:58pm
Data migration
The first data sync pass has been happening since mid-day today and it is approx 75% of the way through at this time. This is the longest part of the process as it involves the first copy of all data from the old machine while everything remains online and functional from the old server.

When this part is complete a second sync will be done straight after to catch any new files. This stage will be much, much quicker.
Updated by Chris J. on 5th May 2010 @ 18:11pm
Data migration
The first data sync is now complete and the second will start shortly. Some services will be shut down (eg email) during this quick sync so that no data gets left on the old machine.
Updated by Chris J. on 5th May 2010 @ 20:45pm
Data migration
Note that those that have your main email address (to which these updates are sent) set to an address hosted on Ace, this message may be delayed while email is disabled on the server.

In a slight change of plan we are in the process of doing the second sync to all data EXCEPT user home directories. Once this is complete (approx 5 mins) we will be swapping the IP's over so that all traffic is then served by the new machine. Immediately after this we will do the second sync of user home directory data from the old machine to the new to update any new/changed files since the first sync.

We are doing it this way because MySQL needs to be down on the old server after the second sync in order to keep databases consistent. When traffic is shifted to the new machine all websites should be fully online and operational while the user home directories are sync'd in full.

To safeguard email data integrity we will be keeping email services disabled on the new server until all data sync's are complete though due to our email redundancy all incoming email will still be received and queued on the email systems until the new server if fully operational.
Updated by Chris J. on 5th May 2010 @ 21:03pm
Data migration
All traffic is now getting served from the new machine and the second sync of user home directories is approx 80% complete. During this time all websites and services should be online except email services which will be enabled after the second user home data is finished.
Updated by Chris J. on 5th May 2010 @ 21:21pm
Data migration
The data sync is now fully complete and all services (including email) are back online and being served from the new machine.

A few final bits need to be taken care of now such as fantastico and rvskin installations which are being done next.
Updated by Chris J. on 5th May 2010 @ 21:24pm