Some customers have reported that they are still seeing the maintenance page when visiting their website. This can be corrected by simply refreshing or reloading the browser while holding down the CTRL or Apple key depending on whether you are on a PC or MAC respectively.
The reboot on part of our storage segment has been completed and all services, (web, email, ftp, and etc), are back to normal. We will continue to closely monitor all of the services on (gs) Grid.Cluster.1 for the next few days to ensure that the issues have been resolved. Please remember also that incoming emails during today’s issues have been queued and are currently being delivered. As well as due to the high volume of incoming email, we have placed load limitations on the servers as not to overwhelm the systems. So when attempting to send, you may randomly get deferral messages mentioning “too much load” or “too many connections”. This will be resolved once the SMTP load has decreased.
We also have some scheduled maintenance coming this Friday evening:
This maintenance being performed on the storage segment this Friday directly addresses the issues we have seen today as well as some other issues we have experienced with the storage segment.
We sincerely apologize for the huge inconvenience this has been to some of our customers. We do understand that email is an integral part of your daily business functions and we appreciate your patience and understanding.
Our systems engineers have been able to isolate the issues on the storage segment and the vendor has recommended that a portion of it be rebooted. It is expected that this should resolve the issues we have been seeing this morning and we will continue to monitor the storage closely with our vendor to ensure that the issue is resolved.
During this restart, all services on (gs) Grid.Cluster.1 will be unavailable for 20 to 30 minutes. Although we regret the timing of this maintenance downtime, we will take every measure to ensure that the systems are back online in a timely manner.
Thank you once again for your understanding.
Our Systems Engineers have managed to improve the web latency issue and are still working to resolve the mail connectivity issues. We will have further updates as we get closer to fully resolving the issues.
Our systems engineers have found that the latency and connectivity issues are due to issues with the storage segment and are currently communicating with our storage vendors to investigate the issues.
Thank you for your patience.
Our systems engineers are currently aware of the web latency and SMTP issues on (gs) Grid.Cluster.1 and are investigating the issue. We will post further updates as they are are reported by our systems engineers.
Thank you for your patience and understanding.
»