Resolved- Olga "has" had a bad hard drive

[Matt]

One of the main drives on the Olga server needs to be replaced.

The good news for you folks on the Olga server is that it is running hotswapable raid drives which means that we should be able to yank out the old one and put in a new one without any disruption in service.

We are going to go ahead and schedule this for Tuesday, February 26th at 2AM EST just in case there are any unforeseen problems.

We anticipate this to be uneventful but wanted to post it here just in case.

[Matt]

Well, I guess timing is everything. This machine decided to convert itself to "read only" mode before we could even start the drive replacement. This is not usually a good thing it typically means there are critical drive errors and Linux enters into a protective mode so that data can be copied before a possible entire disk failure.

Still not sure why this would happen on a RAID machine but in any event, the system is now running fsck which is a disk check which attempts to repair any bad sectors on the good disks, which usually, but not always, returns the system back to an operational stage.

fsck can take several hours on a typical machine, however Olga is one of the Budget shared hosting machines, (more disk space for a lower price) so there are large amounts of data on this piece of hardware, which means that fsck is likely going to take several hours.

Is a very large machine with lots of data that will need analyzed so we have no ETA for you at the moment when fsck will complete.

We will update you as we have more information for you.

[kengele]

Hi, I sent in a help ticket about not being able to access cpanel, email or anything.... Later I found your post about Olga's bad hard drive. Not much I can do but wait, and it is good to know that there is a problem and you are working on it. The tough thing for me is I am in Africa so this is happening during working hours. Can you send an email to me next time beforehand so I am more prepared?

[cornnfedd]

so is this going to go for much longer?

this is the second crash for Olga in a few months, so is this issue going to be resolved this time or do we throw 99.9% uptime out the window?

??? very frustrating indeed but appreciate you letting us know what is happening..

[andreadoria]

Hey Matt, it's been an hour or so, mind looking at the progress indicator? Bated breath over here, I've just started a forum couple weeks ago and it's going real crazy, 2k+ uniques yesterday alone.

The outage's not that terribly bad, being 3am on the east coast atm, but I'm still climbing the walls.

[Matt]

Originally Posted by kengele

Can you send an email to me next time beforehand so I am more prepared?

Please see this thread:

How to receive instant downtime notifications

[cornnfedd]

Originally Posted by andreadoria

Hey Matt, it's been an hour or so, mind looking at the progress indicator? Bated breath over here, I've just started a forum couple weeks ago and it's going real crazy, 2k+ uniques yesterday alone.

The outage's not that terribly bad, being 3am on the east coast atm, but I'm still climbing the walls.

hey whats your website address so i can check it when its back up and running?

[andreadoria]

hehehe doingfine.org it is.

threadjacker

[Matt]

Originally Posted by cornnfedd

so is this going to go for much longer?

Unfortunately, I do not have an answer for you because Linux does not give us a reasonable ETA on when it will be done analyzing the filesystem.

Originally Posted by cornnfedd

this is the second crash for Olga in a few months

The last failure on this machine was in December. That was an OS issue. Sorry, we don't make Linux, Apache, or any of the other software bits that can fail.

This time it is hardware, very expensive hardware I might add and we are not happy about this either. The hardware we have in place for this particular unit is supposed to protect everyone for exactly this type of scenario, but apparent it is not doing what it was advertised to do.

Originally Posted by cornnfedd

so is this issue going to be resolved this time or do we throw 99.9% uptime out the window?

Was it not resolved the last time?

I believe the 99.9% uptime guarantee applies to Professional shared and managed dedicated hosting. If you host on the Olga server, your hosting package should not be a Professional shared hosting package, and is definately not a dedicated server.

If you would like a higher SLA from GlowHost, I encourage an upgrade.

If we fall below the uptime guarantee that is promised to you on your hosting package, which is no less than 99% on any plan, please let us know by opening a ticket at the helpdesk and we will be happy to issue you a credit for your hosting service.

Again accept our apologies and please know we are doing everything in our power to get the machine back online.

[cornnfedd]

Hi Matt,

appreciate the response, professional as always and happy you can explain it clearly, lets hope it gets sorted soon.

cheers.