linux apache+tomcat集群+负载均衡

1 所需软件及版本：

apache:httpd-2.4.17.tar.gz(apr-1.5.2.tar.gz 、apr-util-1.5.4.tar.gz 、pcre-8.37.tar.gz)

tomcat:apache-tomcat-7.0.63.tar.gz

jk:tomcat-connectors-1.2.41-src.tar.gz

 

2 安装顺序（apr,apr-util,pcre,apache;tomcat、jk）

2.1 apr

./configure --prefix=/usr/local/apr
make && make install

 

2.2 apr-util

./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr
make && make install

 

2.3 pcre

./configure --prefix=/usr/local/pcre --with-apr=/usr/local/apr
make && make install

 

2.4 apache

./configure --prefix=/usr/local/apache2 --enable-so --enable-rewrite --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util/bin --with-pcre=/usr/local/pcre

make && make install

2.5 tomcat

解压至/usr/local/。复制两份tomcat。一个配置成tomcat1，另一个配置成tomcat2。

 

2.6 jk

./configure --with-apxs=/usr/local/apache2/bin/apxs

make && make install

如果成功结束，可以在/usr/local/apache2/modules/下找到mod_jk.so文件

 

3 集群及负载均衡配置

3.1 配置tomcat

对于tomcat配置主要集中在conf/server.xml文件上。首先，鉴于是在同一机器上运行两个tomcat实例，因此，要对 server.xml中配置的所有端口进行修改，避免端口被占用，一种简单而稳妥的修改方法是将该文件中出现的所有端口号按一种简单的规则统一进行改动， 比如在原端口号基础上统一加1000或减1000。

3.2 开启tomcat集群支持

在conf/server.xml文件里，进行两处改动：

改动1：为<Engine>设置jvmRoute

改动2：在<Engine/>中添加关于集群的配置。这份默认配置可以满足大多数应用场景，因此，只需要将这份配置复制到<Engine/>中，tomcat的配置就全部完成了。

修改后，conf/server.xml的内容如下：

 

<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->
    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 8080
    -->
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL HTTP/1.1 Connector on port 8443
         This connector uses the BIO implementation that requires the JSSE
         style configuration. When using the APR/native implementation, the
         OpenSSL style configuration is required as described in the APR/native
         documentation -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol"
               maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :-->
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">
    
   <!-- <Engine name="Catalina" defaultHost="localhost"> -->
      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"  
         channelSendOptions="8">            
          <Manager className="org.apache.catalina.ha.session.DeltaManager"  
             expireSessionsOnShutdown="false"  
                        notifyListenersOnReplication="true"/>                            
          <Channel className="org.apache.catalina.tribes.group.GroupChannel">  
          <Membership className="org.apache.catalina.tribes.membership.McastService"  
                address="228.0.0.4"  port="45564"  frequency="500"  dropTime="3000"/>  
          <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"  
                address="auto" port="4000" selectorTimeout="5000"   maxThreads="6"/>  
                                                                                                                                                  
    <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">  
       <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>  
    </Sender>  
    <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>  
    <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>  
  </Channel>  
                          
  <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>  
  <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>  
               
  <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"  
              tempDir="/tmp/war-temp/"  
              deployDir="/tmp/war-deploy/"  
              watchDir="/tmp/war-listen/"  
              watchEnabled="false"/>  
                                                    
  <ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>  
    <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>  
    </Cluster> 

      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

 3.2 配置apache

 

打开apache安装目录下的conf/httpd.conf文件

1 修改：

 

ServerName localhost:80

 

2 在文件最后追加以下内容：

 

# Load mod_jk module  
LoadModule jk_module modules/mod_jk.so
# Specify jk log file.  
JkLogFile /var/log/mod_jk.log
# Specify jk log level [debug/error/info]  
JkLogLevel info
# Specify workers.properties, this file tell jk:  
# how many nodes and where they are.  
JkWorkersFile conf/workers.properties
# Specify which requests should handled by which node. 
#指定哪些请求由tomcat处理,controller为workers.properties文件里指定的负载控制器. 
JkMount /* controller

 

 

3 在conf下创建 workers.properties文件，并添加一下内容：

#所有节点列表，其中controller是一个逻辑结点，负责负载均衡控制，  
#如果JkMount中的URL指定给了controller就表示这个请求会被自动散列到某个物理节点上。  
#注意：真正负责处理请求的tomcat的名称（这里就是tomcat1,tomcat2)必须于它们在conf/server.xml  
#文件中配置的jvmRout的属性值是一致的！  
worker.list = controller,tomcat1,tomcat2  
   
#========tomcat1========  
worker.tomcat1.port=8009       #ajp13 端口号，在tomcat下server.xml配置,默认8009  
worker.tomcat1.host=localhost  #tomcat的主机地址，如不为本机，请填写ip地址  
worker.tomcat1.type=ajp13  
worker.tomcat1.lbfactor = 1    #server的加权比重，值越高，分得的请求越多  
#========tomcat2========  
worker.tomcat2.port=9009       #ajp13 端口号，在tomcat下server.xml配置,默认8009  
worker.tomcat2.host=localhost  #tomcat的主机地址，如不为本机，请填写ip地址  
worker.tomcat2.type=ajp13  
worker.tomcat2.lbfactor = 1    #server的加权比重，值越高，分得的请求越多  
  
#========controller,负载均衡控制器========  
worker.controller.type=lb  
worker.controller.balance_workers=tomcat1,tomcat2   #指定分担请求的tomcat，旧版本中的balanced_workers,已不再推荐使用！  
worker.controller.sticky_session=1 #sticky_session为1表示，  
#当某一 client的session创建之后，后续由该客户端发起的请求，也就是这个session的所有请求都始终由第一次处理该请求的结点  
#负责处理（除非该结点挂掉）

 4 测试

4.1 可分别在tomcat1、tomcat2的webapps下创建mytest文件夹，在mytest里面放入测试页面index.jsp，其内容为：

<%@ page contentType="text/html; charset=utf-8" %>
<%@ page import="java.util.*" %>
<html><head><title>Cluster App Test</title></head>
<body>
Server Info:
<%
out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%>
<%
  out.println("<br> ID " + session.getId()+"<br>");
  // 如果有新的 Session 属性设置
   String dataName = request.getParameter("dataName");
     if (dataName != null && dataName.length() > 0) {
          String dataValue = request.getParameter("dataValue");
          session.setAttribute(dataName, dataValue);
     }
     out.print("<b>Session 列表</b>");
     Enumeration e = session.getAttributeNames();
     while (e.hasMoreElements()) {
           String name = (String)e.nextElement();
           String value = session.getAttribute(name).toString();
           out.println( name + " = " + value+"<br>");
           System.out.println( name + " = " + value);
     }
%>
     <form action="index.jsp" method="POST">
            名称:<input type=text size=20 name="dataName">
             <br>
            值:<input type=text size=20 name="dataValue">
             <br>
            <input type=submit>
     </form>
  </body>
</html>

 先启动apache服务器，再分别启动两台tomcat。通过apache访问这个应用（假设apache服务器所在IP为192.168.6.121，那么访问地址为：http://192.168.6.121/mytest/index.jsp），观察tomcat后台打出的日志会发现，请求被随机分配给了两个tomcat中的一台。

4.2 以上实验不能体现session复制，可用以下步骤证明：

将以上index.jsp页面拷贝到tomcat自带的webapps下的examples文件夹下，并且修改webapps/examples/WEB-INF/web.xml文件，在里面添加元素：

<distributable/>

当关闭tomcat2（ctrl+c）后继续刷新url(http://192.168.6.121/examples/index.jsp)访问。 tomcat1控制台中输出的session 依然与之前tomcat2中的一致，并且在session中保存的属性值仍然有效。即：tomcat1接手了停掉的tomcat2中的session，继续为用户提供服务，而用户感觉不到背后的变化。

 

5 其他

session复制配置
tomcat集群中的 session管理，主要有两种方式：
1).粘性session
表示从同一窗口发来的请求都将有集群中的同一个tomcat进行处理。配 置方式是在上面workers.properties文件中
worker.lbcontroller.sticky_session=True   
粘 性session的好处在不会在不同的tomcat上来回跳动处理请求，但是坏处是如果处理该session的tomcat崩溃，那么之后 的请求将由其他tomcat处理，原有session失效而重新新建一个新的session，这样如果继续从session取值，会抛出 nullpointer的访问异常。
2).session复制
session复制是指tomcat彼此之间通过组播方式将session 发到各个tomcat实例上，如果其中一个访问出错，则另外tomcat仍然具有有效的session内容，从而能正常接管其session。坏处是当 tomcat实例很多，或者用户在session中有大量操作时，组播发送的信息量十分惊人。session复制配置则是在发布的web应用程序中的 web.xml中添加
<distributable/>
此外,session复制所需的JDK必须是JDK 5.0及其以上版本