# Server Updates Aug 14



## Drew

After adjusting the date of the changes twice, I am going to be doing it one last time. I apologize for not being able to stick with the original date and time.

We will be doing work on the server around ~2am PST Sunday morning (tomorrow morning or August 14th).

The following changes will be made (and some should have been made a while ago):
 *Mirrored hard drives (RAID 1)*: the site will be hosted from two hard drives that have the exact same data on them. While this will slightly slow down the write speed (e.g. you making a new post), it will increase the read speed and one of the hard drives can fail and the site will not go down. We can replace that failed drive without any downtime.
 *Add 4GB RAM*: this will allow for more simultaneous database connections as the site grows. You'll see less "Too many connections" errors.
 *Memcached*: there is a relatively small amount of vBulletin data that is accessed very often, but isn't changed very often. This isn't the best use of the databases and puts an unnecessary strain on the server. Memcached will store this data directly in the RAM which is much faster.
 *eAccelerator*: the forum is coded in PHP, which is stored in an uncompiled state. eAccelerator caches the compiled code, which runs much faster and puts less of a strain on the server. I thought this was something that I already thought was installed. Anyone reading this who has experience with PHP will be laughing that we don't already have this set up. Whoops.

It's hard to say exactly what the downtime will be, but we hope that it will be around 30 minutes.

Thanks for you patience as we get SAS back up to speed with the increased traffic.


----------



## millenniumman75

Thanks for the bulletin :yes.
The downtimes have freaked me out, but at least they were for good reasons.


----------



## Drew

We did all the updates except the mirrored hard drives. That would have required more downtime than I expected. Given how much downtime we've had recently, I expect to do this update in the next few weeks. I'll let you know ahead of time!


----------



## millenniumman75

Heh, I missed the last update :lol.


----------



## Duke of Prunes

Drew said:


> one of the hard drives can fail and the site will not go down. We can replace that failed drive without any downtime.


Not if the failure causes undetected corruption, in which case it would just mirror the corruption to the other disk.

I just had a few PHP OOM errors btw :/ Maybe your caching is a bit aggressive?


----------



## Drew

Duke of Prunes said:


> Not if the failure causes undetected corruption, in which case it would just mirror the corruption to the other disk.
> 
> I just had a few PHP OOM errors btw :/ Maybe your caching is a bit aggressive?


Of course that's possible. In that case we'd revert to our daily incremental backup.

We are still ironing the issues out with eAccelerator. Please report any errors you get here. It would be much appreciated.

Thanks!


----------



## Duke of Prunes

I'd have copy-pasted them but it didn't occur to me until they stopped happening. I'll post any future ones.


----------



## mind_games

Not sure if related, but I just experienced a '503 Service Unavailable' error. Probably lasted 20mins or so. I don't remember getting these before.


----------



## purplefruit

i was also getting 503'd for a while there.


----------



## Drew

A mistake was made configuring our webserver when increasing the RAM available for the PHP accelerator.

Sorry about that guys.


----------



## DeeperUnderstanding

I am glad that I stayed off SAS for the day. I missed all the fun, apparently!


----------



## GunnyHighway

Mmmmmm, RAID1. I likes me some good old fashioned RAID fun. (RAID0 is where it's at)

But as always Drew, thanks for keeping us updated. Nothing worse than a site with downtime that doesn't tell anybody what's going on.


----------



## Duke of Prunes

Plain RAID 0 is just dangerous unless it's in some kind of nested setup that provides some redundancy, though keep in the mind that the point of redundancy is to make failures less catastrophic to service uptime (when combined with snapshots/backups), not provide any protection against corruption. RAID controllers will detect some corruption and prevent it from affecting service, but there's no protection against any corruption above the controller, e.g. volume manager, encryption, FS.


----------



## Mr Self Destruct

Duke of Prunes said:


> Plain RAID 0 is just dangerous unless it's in some kind of nested setup that provides some redundancy, though keep in the mind that the point of redundancy is to make failures less catastrophic to service uptime (when combined with snapshots/backups), not provide any protection against corruption. RAID controllers will detect some corruption and prevent it from affecting service, but there's no protection against any corruption above the controller, e.g. volume manager, encryption, FS.


^ That, Raid 0 is just plain stupid for anything important, even just a gaming machine. That's just asking for complete data loss. Go with RAID 5 or 10 instead.

If you wanted a RAID setup for sheer performance, just buy a SSD instead.


----------



## Drew

I never expected so much discussion about RAID :lol It's much better than people getting mad about me for the downtime


----------



## GunnyHighway

I guess some people missed my posts (or are just dicks) in the geek section about being a computer enthusiast and working as a computer technician, so joking about RAID failed. I run RAID0, but all my important stuff is on other drives, which are then backed up onto another 2TB drive. I've gotta remember this isn't my computer enthusiast forum. :|

Oh yeah, and Drew. *shakes fist*

Don't let it happen again....OR ELSE. :blank


----------



## Perfectionist

woop woop posting in a thread with techy words I don't understand woop woop


----------



## Duke of Prunes

Mr Self Destruct said:


> ^ That, Raid 0 is just plain stupid for anything important, even just a gaming machine. That's just asking for complete data loss. Go with RAID 5 or 10 instead.
> 
> If you wanted a RAID setup for sheer performance, just buy a SSD instead.


Just one SSD?

What this site needs is an octet of those 10-core Xeon E7s, a fat old chunk of RAM and a gigantic RAID 10 of fibre channel SLC SSDs.

The read speeds would be so quick and you'd have such a huge amount of memory that you could cache the whole host OS and some of the content at boot and write the disk images of the Xen guests (or whatever hypervisor you like) to ramdisks, then use union mounts to save changes to the disks.

With that kind of CPU power, you could probably get away with using compressed ramdisks for better capacity and cache even more content without any serious overhead. The boot times might suck a bit, but as long as it's looked after, the thing should stay up forever.


----------



## Mr Self Destruct

Duke of Prunes said:


> Just one SSD?
> 
> What this site needs is an octet of those 10-core Xeon E7s, a fat old chunk of RAM and a gigantic RAID 10 of fibre channel SLC SSDs.
> 
> The read speeds would be so quick and you'd have such a huge amount of memory that you could cache the whole host OS and some of the content at boot and write the disk images of the Xen guests (or whatever hypervisor you like) to ramdisks, then use union mounts to save changes to the disks.
> 
> With that kind of CPU power, you could probably get away with using compressed ramdisks for better capacity and cache even more content without any serious overhead. The boot times might suck a bit, but as long as it's looked after, the thing should stay up forever.


yeah definitely not for i website, i meant as a better alternative to RAID 0 for a home/gaming computer.


----------



## Duke of Prunes

Was joking, would definitely be overkill


----------

