frequent maintenance ... arg!
Permalink
So if you're active here you may have noticed we've pulled concrete5.org down a couple of times in just as many weeks - which for us is an awful lot of down time...
What's going on is we've moved servers and the new box hasn't been all that co-operative. We've got a few different types of hard drives on the new server, one very fast one for mysql and the OS, and one huge one to store all the files you guys upload, which is a shockingly large pile.
The problem has been that storage drive keeps going into "read only mode" which we're told happens when the drive has an error of some type. The first time we took it down was for the actual move to the new machine. Then we had to take it down when the drive went into read mode and we ran a bunch of drive checks and everything said it was okay. It then happened again so we took it down and had the SCSI card replaced. THEN yesterday it went down because it happened YET again and this time we replaced the entire hard drive as well as the mother board. We deeply deeply hope that solves it, because its obviously super annoying, stressful, and slightly embarrassing to be yoinking our site up and down like this.
All I can tell you is:
A) It's not concrete5 causing the issues, its just bad hardware on a new server.
B) Eventually these things typically settle out on machines, we never had any technical issues like this on our last box.
C) If I had a bigger pile of money to throw at this problem I can architect any number of redundant setups that would help us keep concrete5.org up so our performance matched facebook instead of twitter. It just doesn't make sense for us to manage this type of mirrored setup with the resources we have today. If you've got a big enterprise site and the need to keep it up 99.9999% of the time, you absolutely can do that with concrete5.
In other hosting news we also had a DNS server die on our trial setup box, as well as a mailman process go rouge and consume all the resources on one of our cpanel setups yesterday... Generally I'm just deeply deeply hoping to have a better day today than we did yesterday. (knocks on wood)
What's going on is we've moved servers and the new box hasn't been all that co-operative. We've got a few different types of hard drives on the new server, one very fast one for mysql and the OS, and one huge one to store all the files you guys upload, which is a shockingly large pile.
The problem has been that storage drive keeps going into "read only mode" which we're told happens when the drive has an error of some type. The first time we took it down was for the actual move to the new machine. Then we had to take it down when the drive went into read mode and we ran a bunch of drive checks and everything said it was okay. It then happened again so we took it down and had the SCSI card replaced. THEN yesterday it went down because it happened YET again and this time we replaced the entire hard drive as well as the mother board. We deeply deeply hope that solves it, because its obviously super annoying, stressful, and slightly embarrassing to be yoinking our site up and down like this.
All I can tell you is:
A) It's not concrete5 causing the issues, its just bad hardware on a new server.
B) Eventually these things typically settle out on machines, we never had any technical issues like this on our last box.
C) If I had a bigger pile of money to throw at this problem I can architect any number of redundant setups that would help us keep concrete5.org up so our performance matched facebook instead of twitter. It just doesn't make sense for us to manage this type of mirrored setup with the resources we have today. If you've got a big enterprise site and the need to keep it up 99.9999% of the time, you absolutely can do that with concrete5.
In other hosting news we also had a DNS server die on our trial setup box, as well as a mailman process go rouge and consume all the resources on one of our cpanel setups yesterday... Generally I'm just deeply deeply hoping to have a better day today than we did yesterday. (knocks on wood)
blah donations aren't going to cover mirrored server setups.. what you mean is where's IBM's open source grant? ;-P
haha ;) fine i mean that.
what about a sponsors program?
also the renovations thing, it uses the old c5 logo.
Mike
what about a sponsors program?
also the renovations thing, it uses the old c5 logo.
Mike
now, wheres that discreet donate button... :P
Mike