Preparring for a High traffic surge (Twitter/Digg/Slashdott style)
Permalink
Hey guys,
One of the websites I manage is going to be linked to from a very high profile Twitterer later this week. The twitterer was keen to make us aware his previous link posts have sent websites crumbling (50k visitors in an hour).
Over the next couple of days I am scaling up the server to handle this load by increasing our VPS memory limits and changing from apache to an optimised webserver such as lighttpd or nginx.
The key to preventing the server overloading will be caching.
So firstly I am going to replace the homepage with a static html page using a apache redirect, to not effect the way concrete uses index.php to load all pages. Depending on how much time I have I may convert other high traffic pages to static.
I know concrete caches content by default which is great. Reducing db calls is obviously key. I have been looking into caching uses software like memcached but think this may be too time consuming to implement. Is there anything else I can quickly do to optimise concrete5?
Has anyone had any experience with high traffic concrete sites?
I will report back after the event to let you guys know how everything went. Maybe this will be of interest.
Many thanks,
One of the websites I manage is going to be linked to from a very high profile Twitterer later this week. The twitterer was keen to make us aware his previous link posts have sent websites crumbling (50k visitors in an hour).
Over the next couple of days I am scaling up the server to handle this load by increasing our VPS memory limits and changing from apache to an optimised webserver such as lighttpd or nginx.
The key to preventing the server overloading will be caching.
So firstly I am going to replace the homepage with a static html page using a apache redirect, to not effect the way concrete uses index.php to load all pages. Depending on how much time I have I may convert other high traffic pages to static.
I know concrete caches content by default which is great. Reducing db calls is obviously key. I have been looking into caching uses software like memcached but think this may be too time consuming to implement. Is there anything else I can quickly do to optimise concrete5?
Has anyone had any experience with high traffic concrete sites?
I will report back after the event to let you guys know how everything went. Maybe this will be of interest.
Many thanks,
My concrete5 Japan site...
1. Installed eAccelarator
2. httpd.conf
My server setting
Key point is KeepAlive parameter.
KeepAliveTimeout should be "1" so that it will die immediately, but at the same time, it can handle large number of users.
3.my.conf
is really depends on how big your MySQL database is... so cannot really tell you... I suggest to keep looking at your phpMyAdmin statistic to adjust each parameter.
This handled 10-20 simultaneous users/sec access
Good luck.
1. Installed eAccelarator
2. httpd.conf
My server setting
StartServers 20 MinSpareServers 2 MaxSpareServers 5 ServerLimit 200 MaxClients 200 MaxRequestsPerChild 4000 KeepAlive On MaxKeepAliveRequests 4000 KeepAliveTimeout 1
Key point is KeepAlive parameter.
KeepAliveTimeout should be "1" so that it will die immediately, but at the same time, it can handle large number of users.
3.my.conf
is really depends on how big your MySQL database is... so cannot really tell you... I suggest to keep looking at your phpMyAdmin statistic to adjust each parameter.
This handled 10-20 simultaneous users/sec access
Good luck.
If you have a .htaccess file you should put its content into the virtual host settings of apache and remove the file. Because .htaccess gets loaded on every request.
1. Definitely eAccelerator.
2. If you increase your server's limits, make sure you enable more connections in your database, as well as increasing MySQL's memory limit.
3. Further caching is ideal, including full page caching. Unfortunately concrete5 doesn't do this yet, but we have hacked solutions in the past. Check out PEAR CacheLite.
For something a little bit more robust and a site that is less dynamic, a reverse proxy is another good solution.
2. If you increase your server's limits, make sure you enable more connections in your database, as well as increasing MySQL's memory limit.
3. Further caching is ideal, including full page caching. Unfortunately concrete5 doesn't do this yet, but we have hacked solutions in the past. Check out PEAR CacheLite.
For something a little bit more robust and a site that is less dynamic, a reverse proxy is another good solution.
We'd love to hear how it goes after the linking!
1 Page cache. C5 currently caches objects, in case a page is almost static you could also cache the complete page. I added a page attribute for this which I'm checking before the rendering process (ugly dispatcher hack) to display a simple html file. I never published it because you have to know where you can activate it.. I would get a lot of silly questions about it for sure
2. Reverse proxy. A lot of sites don't need a states. Meaning you could set up 5 web server which don't have a connection. Mirror the site from a master server to the slaves every hours.. Ugly but allows you to scale beyond everything. In combination with nginx as a reverse proxy, this should work fine..
3. memcachedb. I tried to replace eAccelerator using memcache but unfortunatley I stopped because we realized that we don't have a lot of content in that project where I needed a bit more power. We're working with CodeIgniter which is fine if you don't have content to manage..
4. IO. Usually you'll run into IO bottlenecks... VPS is not the best solution if you have lots of I/Os...