I struggled to find out why my server broke down frequently these days. And I think I found it.
My Apache HTTP server was configured with MaxClients at 50, the default AJP connector’s connections number was about 15, and the Tomcat’s maxThreads was 150. So the bottleneck would be AJP connector, especially all my connection-heavy Comet application was serving about 20 Gtalk/MSNLive simultaneous connections. It is obviously it will break down the servers. And after switching my Gtalk/MSNLive into query modes, it still broke down the server. The reason may be the bottleneck of Apache HTTP server’s 50 maximum connections.
I updated all the configuration to 256 for maximum simultaneous connections. Hope server won’t break down for Comet application from now on.
For more about extreme maximum simultaneous connections, please read articles:
<!-- Define an AJP 1.3 Connector on port 8009 -->
maxThreads="256" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" protocol="AJP/1.3" />
Here is an example to replace string with a different delimiter in Windows’ *.bat file.
for /f "tokens=1-15 delims=/" %%i in ("%1") do (
if not "%_X2_%" == "" set _XPATH_=%_XPATH_%\%_X2_%
if not "%_X3_%" == "" set _XPATH_=%_XPATH_%\%_X3_%
if not "%_X4_%" == "" set _XPATH_=%_XPATH_%\%_X4_%
if not "%_X5_%" == "" set _XPATH_=%_XPATH_%\%_X5_%
if not "%_X6_%" == "" set _XPATH_=%_XPATH_%\%_X6_%
if not "%_X7_%" == "" set _XPATH_=%_XPATH_%\%_X7_%
if not "%_X8_%" == "" set _XPATH_=%_XPATH_%\%_X8_%
if not "%_X9_%" == "" set _XPATH_=%_XPATH_%\%_X9_%
if not "%_XA_%" == "" set _XPATH_=%_XPATH_%\%_XA_%
if not "%_XB_%" == "" set _XPATH_=%_XPATH_%\%_XB_%
if not "%_XC_%" == "" set _XPATH_=%_XPATH_%\%_XC_%
if not "%_XD_%" == "" set _XPATH_=%_XPATH_%\%_XD_%
if not "%_XE_%" == "" set _XPATH_=%_XPATH_%\%_XE_%
if not "%_XF_%" == "" set _XPATH_=%_XPATH_%\%_XF_%
For more information, please key in “for /?” in command line.
If you ever visit Eclipse homepage http://www.eclipse.org/, you may notice that Eclipse homepage embeds plugin updates news from Eclipse Plugin Central (http://www.eclipseplugincentral.org/). If you try to visit the news by the given link, you may find that the page loads very very very SLOW!
Why? Open the the page source, you will see that it generates all 900+ plugins in the source! And the *.html file size is up to 200+k. And worse thing is that every page in Eclipse Plugin Central is the same with 200+k HTML file! Every page!
Besides it was once hacked by someone and its functions were buggy, such 200+k HTML pages design in Eclipse Plugin Central must be listed as #1 of aweful designs that I have ever seen.
Recently, my Apache HTTP server was broken down twice. The scenario was when visiting a page, it was just waiting there, blank. And I checked the server through SSH and saw that there was a lot (about 50) of inner 8009 connections. After restarting Apache HTTP Server and Tomcat Server, these connections decreased to less than 10 connections.
Client <-Kept Connection-> Apache HTTP Server <-AJP Connector (8009 Connection)-> Tomcat
When a user is connected, an AJP Connector is setup for him/her. The connection will not be tore down until the user disconnects from Gtalk or MSNLive service. So if there are more than 50 users online, there are no doubts that the server crashes.
To solve this problem, one solution would be not using Apache HTTP Server in the middle of Tomcat server and browser client. So there is no limitation on AJP connections.
And another solution would be monitoring user connections on Tomcat server side, and if there are more than 50 connections, try to switch application’s Comet-mode into Query-mode smartly.
Providing Comet applications need to solve a lot of unknown problems.