Overview

Architecture

JLupin Platform provides distributed software load balancers to ensure scale-out architecture of the whole environment (including platform components). At least one instance of JLupin Software Load Balancer is present on Main Server and every servlet and native microservice that is configured to communicate with other microservices on the platform.

Figure 1. JLupin Software Load Balancers in the environment.

Notice that every request between microservices on JLupin Platform goes through two instance of JLupin Software Balancer:

  • the first one, located on the microservice that creates a request (1)
  • the second one, included in Main Server, chosen by the previous balancer (2).

Figure 2. JLupin Software Load Balancers in the default cascade configuration.

This cascade configuration of load balancers gives opportunity to provide the highest reliability of the request's processing. Even after sending the request from the first balancer and the sudden failure of microservice on the target node, the second balancer, located on that node, is still able to redirect the traffic to another node. This protects the communication against microservice failures between service discovery iterations.

All instances of JLupin Software Balancers performs periodic service discovery to update their service repository used in the load balancing process (to get to know more about the service repository & discovery go to this chapter). The algorithms used by load balancers in the process of choosing a target node is descried in next chapters. After the target node is chosen, the load balancer tries to establish a connection and sent the request. If it is not succeeded it tries singleRequestRepeatsAmount (the parameter in main.yml, discussed below) times to find another node from NODE PEERS list (in main.yml, discussed below) to handle the request. The singleRequestRepeatsAmount is set to 3 by default and could be overwritten by repeatsAmount while JLupin Software Load Balancer is created (see details in JavaDocs - JLupinClientUtil specification).

The instance located on Main Server performs an additional task - it periodically checks the health of controlled microservices, basic on the results it determines their status and update its internal service repository. More detailed description can be found in the next chapter.

Health checking

The health checking process is performed by Main Server by periodically connecting to microservices' TRANSMISSION PORT and getting information about their runtime parameters (cpu, memory, threads etc.). The following picture shows this process on the example of WebApp1 and Microservice A microservice (it's done for all microservices controlled by Main Server).

Figure 3. JLupin Software Load Balancers health checking process .

The data collected during health check is then processed by availability script (in JavaScript) defined by checkAvailableScript parameter, located in microservice's configuration file (to get to know more see this chapter)

The default script is as follows:

function isAvailable(checkResponseTimeInMillis, jrmcActiveThreads, jrmcMaxThreads,
                      queueActiveThreads, queueMaxThreads, servletActiveThreads, servletMaxThreads,
                      jvmMaxMemoryInBytes, jvmTotalMemoryInBytes, jvmFreeMemoryInBytes,
                      jvmProcessCpuLoadInPercentage, userAvailableFlag) {

  var isAvailableByUser = Boolean(userAvailableFlag);

  if(checkResponseTimeInMillis > 20000 || !isAvailableByUser) {
    return false;
  }
  return true;
}

If it returns true the microservice is set available, in case of false the load balancer sets the microservice as unavailable. Note that this mechanism can be individually adopted to the characteristic of each microservice.

What's more, you can implement you own service performing more advanced and customized health checking tasks (for example, involving database), which aggregated result can be provided to the availability script by boolean value userAvailableFlag parameter. The only thing you should do is set userAvailableFlag in your service accordingly through System.setProperty command in JAVA code.

Load balancing

There are two built-in load balancing algorithms:

  • PRIORITY - the default algorithm for inner microservice communication (including Main Server to microservices relation).
  • ROUND_ROBIN - the default algorithm for external clients that uses jlupin-client-assembly library to communicate with JLupin environment.

You can change the default algorithm during load balancer initialization (in code) by dividing this task into two steps:

  • create JLupinLoadBalancer using generateLoadBalancer method in JLupinClientUtil class, where jLupinLoadBalancerAlgorithmType input parameter can be set accordingly. Then
  • create JLupinDelegator using previousle created JLupinLoadBalancer as one of the input parameter.

NOTE that algorithm for Main Server to microservices communication is hardcoded as PRIORITY type due to architectural assumptions and cannot be changed.

If you would like to get to know more about custom load balancers configuration please refer to Java Docs.

PRIORITY balancing

The priority balancer always takes the shortest connection path to determine the target node. In practice, it means that priority balancer always first looks for the target microservice on the same node. This algorithm is presented on the following figure on the example of communication between WebApp1 and Microservice A microservice.

The communication schema in normal operation:

Figure 4. Priority load balancer in normal operation.

The communication schema in case of Microservice A failure:

Figure 5. Priority load balancer in case of microservice failure.

ROUND_ROBIN balancing

The round robin balancer tries to equally distribute traffic across available nodes. It takes the first node from the list and after changeServerIntervalInMillis milliseconds (input parameter of JLupinDelegator available in JLupinClientUtil class) switches traffic to the next one, as show on the following pictures.

The communication schema the initial state of balancing:

Figure 6. Round robin load balancer in the initial state.

The communication schema after changeServerIntervalInMillis milliseconds:

Figure 7. Round robin load balancer after switch to the next node.


Configuration

Overview

JLupin Load Balancer is configured by parameters located in main.yml file as well as some sections provided by microservice configuration files (servlet_configuration.yml / configuration.yml) as shown on the following picture.

Figure 8. Round robin load balancer after switch to the next node.

Detailed description of those parameters can be found below (main.yml) and in chapters about native microservices and servlet microservices

Configuration files (main.yml)

Section 'MICROSERVICES_LOAD_BALANCER_SETTINGS'

Parameter Description
singleRequestRepeatsAmount The number of attempts that load balancer will take to process the request to microservice
howOftenCheckingMicroservicesInMillis The health checking period and service discovery process for Main Server's load balancer, expressed in milliseconds
externalHealthCheckPeriod This parameter SHOULD be set to health checking period (expressed in milliseconds) of the external balancer (if it is used) to keep servlet microservices' restart online.
externalNodePeersStartCheckDelayInMillis The delay expressed in milliseconds aftre wich load balancer starts checking peers and collecting infromation from them (zone).

Subsection 'EXTERNAL_CONNECTIONS'

This subsection refers to load balancers located on the microservices and Main Server, it describes connections' parameters initialized by this balancers to other jBricks:

Parameter Description
connectionSocketTimeoutInMillis The maximum time in milliseconds that a client's socket is waiting for connection is established, after this timeout the socket is closed. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
readTimeoutInMillis The maximum time in milliseconds that a client's socket is waiting for the response, after this timeout the socket is closed. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
isKeepAlive One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
isOOBInline One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
isTcpNoDelay One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
isReuseAddress One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
sendBufferSize One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
receiveBufferSize One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
soLinger One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !
trafficClass One of the advanced socket parameter. If you interested in reading more about it, please refer to Oracle's official documentation. USE WITH CAUTION !

Section 'NODE PEERS'

This section is the list of other Main Servers (jBricks) that the Main Server and its microservices will connect to. This section could be managed manually by editing main.yml, but more effective is doing it by local console (node peer add|remove) or JLupin Control Center, where jBricks are managed in zones.

Each entry is described by the following set of parameters.

Parameter Description
ip IP address of Main Server.
jlrmcPort The port number of Main Server's JLRMC entry port.
queuePort The port number of Main Server's QUEUE entry port.
transmissionPort The port number of Main Server's TRANSMISSION entry port.
informationPort The port number of Main Server's INFORMATION entry port.

This list is automatically inherited by load balancers of microservices managed by the Main Server.

The default entry in this section is 'localhost' (127.0.0.1) which enables communication within local jBrick.