Loadbalancing with Payara cluster and Apache2

I have tested the configuration on two operating systems - CentOS 7 and Kubuntu 15.10. These instructions are for CentOS, because there it is a bit more complicated.

All application server instances run on Payara 4.1.1.154 - two on CentOS 7 and one on Solaris5.10. Note that for correct cluster replication you need to have configured the multicast routing - or to have all instances on the same network ;)

1) You need to do this as root:

# aka apache2 and modules in debian linuxes
yum install httpd

2) You need time synchronization - one minute error is fatal. You can select another server for time sychronization - check also if it is accessible from the server.

yum install ntp ntpdate

chkconfig ntpd on

ntpdate pool.ntp.org

3) Edit the httpd.conf file and add two lines (use correct hostname, valid from the outside world):

vim /etc/httpd/conf/httpd.conf

ServerName myhost.mydomain.org

LoadModule rewrite_module modules/mod_rewrite.so

4) Edit httpd configuration file and put the loadbalancer settings in it. 

vim /etc/httpd/conf.d/00-default.conf

• VirtualHost will run on port 80
• you need some time to hold the session on one instance, where the user logs in - in this time the new session will be replicated to other payara instances in cluster. If the next request would be faster, the session would be invalidated. This is the reason why you need also the ROUTEID cookie - the stickysession changes it's value because a new session is created.
• third internal host has status +H - that means "hot standby" - it will be routed only if other hosts would be inaccessible.

<VirtualHost *:80>

    ProxyRequests Off

    RewriteEngine Off
    Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/; Max-Age=60;" env=BALANCER_ROUTE_CHANGED

    ServerName myhost.mydomain.org

    LogLevel debug

    ProxyPass /balancer-manager !

    ProxyPass / balancer://myhost.mydomain.org/ stickysession=ROUTEID

    ProxyPassReverse /  http://myinternal1.mydomain:8300/

    ProxyPassReverse /  http://myinternal2.mydomain:8300/

    ProxyPassReverse /  http://myinternal3.mydomain:8300/

    ProxyPassReverseCookieDomain myinternal1.mydomain myhost.mydomain.org

    ProxyPassReverseCookieDomain myinternal2.mydomain myhost.mydomain.org

    ProxyPassReverseCookieDomain myinternal3.mydomain myhost.mydomain.org

    ProxyPassReverseCookiePath / /

    <Proxy balancer://cisas1.lab.i.cz>

        BalancerMember http://myinternal1.mydomain:8300 loadfactor=1 route=int1

        BalancerMember http://myinternal2.mydomain:8300 loadfactor=1 route=int2

        BalancerMember http://myinternal3.mydomain:8300 status=+H route=int3

        Order Deny,Allow

        Deny from none

        Allow from all

        ProxySet stickysession=ROUTEID

        ProxySet lbmethod=bytraffic

    </Proxy>

</VirtualHost>

5) Allow the httpd service to access the outside world via TCP - without that you will get only HTTP 503 and some message in error_log that the action was declined.

/usr/sbin/setsebool httpd_can_network_connect 1

6) Run the httpd service (and loadbalancing) and watch logs

systemctl restart httpd.service

journalctl -xn

tail -1000f /var/log/httpd/error_log