Table of Contents

Configuring IPFIX and the QoE Stor Cluster

Each Cubro DPI node includes a local instance of IPFIXCol2, which acts as an IPFIX load balancer.
The DPI service sends IPFIX statistics locally to the balancer. The balancer then distributes the traffic among the QoE servers based on the configuration file.
If one of the QoE nodes in the QoE Stor cluster becomes unavailable, the IPFIX statistics will automatically be redirected to another available server.

Load Balancing Algorithm:
The system uses the Round-Robin algorithm, which routes each new request to the next server in the list.
This ensures even load distribution across all available QoE Stor servers.

Load-balancer configuration file is located at: /var/fast_ipfix_balancer/backend/.env
Load-balancer configuration can also be managed via GUI at: Administrator → IPFIX-balancer configuration

Parameter Descriptions (for all statistics types):

  1. IPFIX_<TYPE>_PORT_TYPE[0] — Protocol used for incoming IPFIX stream (TCP or UDP)
  2. IPFIX_<TYPE>_PORT[0] — Port number for receiving the incoming IPFIX stream
  3. IPFIX_<TYPE>_BALANCER[0] — Load balancing method
  4. IPFIX_<TYPE>_BALANCER_SUB[0] — IP addresses and ports of the QoE servers among which the statistics will be distributed. They are listed comma-separated in the format x.x.x.x/y, where x.x.x.x is the IP address of the receiver for this type of data stream, and y is the corresponding receiver port.
  5. IPFIX_<TYPE>_BALANCER_SUB_PROTO[0] — Protocol used for outgoing IPFIX stream (TCP or UDP)
  6. IPFIX_<TYPE>_BALANCER_TASKSET[0] — CPU core number the balancer process is pinned to (for performance optimization)

Example Configurations by Statistic Type:

  1. FullFlow Statistics — Distributed across five QoE servers: 
    IPFIX_FULLFLOW_PORT_TYPE[0]=tcp
IPFIX_FULLFLOW_PORT[0]=1500
IPFIX_FULLFLOW_BALANCER[0]=RoundRobin
IPFIX_FULLFLOW_BALANCER_SUB[0]=10.0.15.41/10401,10.0.15.42/10401,10.0.15.43/10401,10.0.15.44/10401,10.0.15.45/10401
IPFIX_FULLFLOW_BALANCER_SUB_PROTO[0]=tcp
IPFIX_FULLFLOW_BALANCER_TASKSET[0]=81
  2. Clickstream Statistics — Distributed across three QoE servers: 
    IPFIX_CLICKSTREAM_PORT_TYPE[0]=tcp
IPFIX_CLICKSTREAM_PORT[0]=1501
IPFIX_CLICKSTREAM_BALANCER[0]=RoundRobin
IPFIX_CLICKSTREAM_BALANCER_SUB[0]=10.0.15.41/10402,10.0.15.42/10402,10.0.15.43/10402
IPFIX_CLICKSTREAM_BALANCER_SUB_PROTO[0]=tcp
IPFIX_CLICKSTREAM_BALANCER_TASKSET[0]=82
  3. DNS Flow Statistics — Distributed across three QoE servers: 
    IPFIX_DNSFLOW_PORT_TYPE[0]=tcp
IPFIX_DNSFLOW_PORT[0]=1502
IPFIX_DNSFLOW_BALANCER[0]=RoundRobin
IPFIX_DNSFLOW_BALANCER_SUB[0]=10.0.15.41/10403,10.0.15.42/10403,10.0.15.43/10403
IPFIX_DNSFLOW_BALANCER_SUB_PROTO[0]=tcp
IPFIX_DNSFLOW_BALANCER_TASKSET[0]=83

Managing QoE Stor Cluster Configuration via GUI

Removing a Failed QoE Server:

To maintain the ability to generate reports when a QoE server fails, the cluster must be reconfigured: remove the failed QoE server from qoe_1_main. In GUI navigate: QoE Stor Configuration → qoe_1_main → Cluster Settings, remove the corresponding QoE server, and save the changes.

Once the failed QoE server is operational again, it must be added back in the cluster settings.

Appointing a Master Server

If the master server goes down, another available server must be designated as the new master. The master server is the one to which the GUI is connected.

To assign a new master server, proceed to the following section of GUI: Administrator → GUI Configuration → QoE Stor: DB (Clickhouse) connection:

  1. In the QoE Stor host field, specify the host's IP address
  2. In the QoE Stor cluster mode field, set the value to "Cluster mode enabled"
  3. In the QoE Stor sql optimizer for cluster mod field, set the value to "Enabled"

Scheme of Distributing DPI Statistics Flows Among QoE Servers

DPI DPI ID IPFIX type QoE Receiver's port QoE IP
tec_cub_dpi_01 101 fullflow tec_cub_qoe_01 10101 10.0.15.41
tec_cub_qoe_02 10101 10.0.15.42
tec_cub_qoe_03 10101 10.0.15.43
tec_cub_qoe_04 10101 10.0.15.44
tec_cub_qoe_05 10101 10.0.15.45
clickstream tec_cub_qoe_01 10102 10.0.15.41
tec_cub_qoe_02 10102 10.0.15.42
tec_cub_qoe_03 10102 10.0.15.43
dnsflow tec_cub_qoe_01 10103 10.0.15.41
tec_cub_qoe_02 10103 10.0.15.42
tec_cub_qoe_03 10103 10.0.15.43
tec_cub_dpi_02 102 fullflow tec_cub_qoe_01 10201 10.0.15.41
tec_cub_qoe_02 10201 10.0.15.42
tec_cub_qoe_03 10201 10.0.15.43
tec_cub_qoe_04 10201 10.0.15.44
tec_cub_qoe_05 10201 10.0.15.45
clickstream tec_cub_qoe_01 10202 10.0.15.41
tec_cub_qoe_02 10202 10.0.15.42
tec_cub_qoe_03 10202 10.0.15.43
dnsflow tec_cub_qoe_01 10203 10.0.15.41
tec_cub_qoe_02 10203 10.0.15.42
tec_cub_qoe_03 10203 10.0.15.43
tec_cub_dpi_03 103 fullflow tec_cub_qoe_01 10301 10.0.15.41
tec_cub_qoe_02 10301 10.0.15.42
tec_cub_qoe_03 10301 10.0.15.43
tec_cub_qoe_04 10301 10.0.15.44
tec_cub_qoe_05 10301 10.0.15.45
clickstream tec_cub_qoe_01 10302 10.0.15.41
tec_cub_qoe_02 10302 10.0.15.42
tec_cub_qoe_03 10302 10.0.15.43
dnsflow tec_cub_qoe_01 10303 10.0.15.41
tec_cub_qoe_02 10303 10.0.15.42
tec_cub_qoe_03 10303 10.0.15.43
tec_cub_dpi_04 104 fullflow tec_cub_qoe_01 10401 10.0.15.41
tec_cub_qoe_02 10401 10.0.15.42
tec_cub_qoe_03 10401 10.0.15.43
tec_cub_qoe_04 10401 10.0.15.44
tec_cub_qoe_05 10401 10.0.15.45
clickstream tec_cub_qoe_01 10402 10.0.15.41
tec_cub_qoe_02 10402 10.0.15.42
tec_cub_qoe_03 10402 10.0.15.43
dnsflow tec_cub_qoe_01 10403 10.0.15.41
tec_cub_qoe_02 10403 10.0.15.42
tec_cub_qoe_03 10403 10.0.15.43
tec_cub_dpi_05 105 fullflow tec_cub_qoe_01 10501 10.0.15.41
tec_cub_qoe_02 10501 10.0.15.42
tec_cub_qoe_03 10501 10.0.15.43
tec_cub_qoe_04 10501 10.0.15.44
tec_cub_qoe_05 10501 10.0.15.45
clickstream tec_cub_qoe_01 10502 10.0.15.44
tec_cub_qoe_02 10502 10.0.15.45
dnsflow tec_cub_qoe_01 10503 10.0.15.44
tec_cub_qoe_02 10503 10.0.15.45
tec_cub_dpi_06 106 fullflow tec_cub_qoe_01 10601 10.0.15.41
tec_cub_qoe_02 10601 10.0.15.42
tec_cub_qoe_03 10601 10.0.15.43
tec_cub_qoe_04 10601 10.0.15.44
tec_cub_qoe_05 10601 10.0.15.45
clickstream tec_cub_qoe_04 10602 10.0.15.44
tec_cub_qoe_05 10602 10.0.15.45
dnsflow tec_cub_qoe_04 10603 10.0.15.44
tec_cub_qoe_05 10603 10.0.15.45
tec_cub_dpi_07 107 fullflow tec_cub_qoe_01 10701 10.0.15.41
tec_cub_qoe_02 10701 10.0.15.42
tec_cub_qoe_03 10701 10.0.15.43
tec_cub_qoe_04 10701 10.0.15.44
tec_cub_qoe_05 10701 10.0.15.45
clickstream tec_cub_qoe_04 10702 10.0.15.44
tec_cub_qoe_05 10702 10.0.15.45
dnsflow tec_cub_qoe_04 10703 10.0.15.44
tec_cub_qoe_05 10703 10.0.15.45
tec_cub_dpi_08 108 fullflow tec_cub_qoe_01 10801 10.0.15.41
tec_cub_qoe_02 10801 10.0.15.42
tec_cub_qoe_03 10801 10.0.15.43
tec_cub_qoe_04 10801 10.0.15.44
tec_cub_qoe_05 10801 10.0.15.45
clickstream tec_cub_qoe_04 10802 10.0.15.44
tec_cub_qoe_05 10802 10.0.15.45
dnsflow tec_cub_qoe_04 10803 10.0.15.44
tec_cub_qoe_05 10803 10.0.15.45
tec_cub_dpi_09 109 fullflow tec_cub_qoe_01 10901 10.0.15.41
tec_cub_qoe_02 10901 10.0.15.42
tec_cub_qoe_03 10901 10.0.15.43
tec_cub_qoe_04 10901 10.0.15.44
tec_cub_qoe_05 10901 10.0.15.45
clickstream tec_cub_qoe_04 10902 10.0.15.44
tec_cub_qoe_05 10902 10.0.15.45
dnsflow tec_cub_qoe_04 10903 10.0.15.44
tec_cub_qoe_05 10903 10.0.15.45
rsl_cub_dpi_01 201 fullflow tec_cub_qoe_01 20101 10.0.15.41
tec_cub_qoe_02 20101 10.0.15.42
tec_cub_qoe_03 20101 10.0.15.43
tec_cub_qoe_04 20101 10.0.15.44
tec_cub_qoe_05 20101 10.0.15.45
clickstream tec_cub_qoe_01 20102 10.0.15.41
tec_cub_qoe_02 20102 10.0.15.42
tec_cub_qoe_03 20102 10.0.15.43
dnsflow tec_cub_qoe_01 20103 10.0.15.41
tec_cub_qoe_02 20103 10.0.15.42
tec_cub_qoe_03 20103 10.0.15.43
rsl_cub_dpi_02 202 fullflow tec_cub_qoe_01 20201 10.0.15.41
tec_cub_qoe_02 20201 10.0.15.42
tec_cub_qoe_03 20201 10.0.15.43
tec_cub_qoe_04 20201 10.0.15.44
tec_cub_qoe_05 20201 10.0.15.45
clickstream tec_cub_qoe_01 20202 10.0.15.41
tec_cub_qoe_02 20202 10.0.15.42
tec_cub_qoe_03 20202 10.0.15.43
dnsflow tec_cub_qoe_01 20203 10.0.15.41
tec_cub_qoe_02 20203 10.0.15.42
tec_cub_qoe_03 20203 10.0.15.43
rsl_cub_dpi_03 203 fullflow tec_cub_qoe_01 20301 10.0.15.41
tec_cub_qoe_02 20301 10.0.15.42
tec_cub_qoe_03 20301 10.0.15.43
tec_cub_qoe_04 20301 10.0.15.44
tec_cub_qoe_05 20301 10.0.15.45
clickstream tec_cub_qoe_01 20302 10.0.15.41
tec_cub_qoe_02 20302 10.0.15.42
tec_cub_qoe_03 20302 10.0.15.43
dnsflow tec_cub_qoe_01 20303 10.0.15.41
tec_cub_qoe_02 20303 10.0.15.42
tec_cub_qoe_03 20303 10.0.15.43
rsl_cub_dpi_04 204 fullflow tec_cub_qoe_01 20401 10.0.15.41
tec_cub_qoe_02 20401 10.0.15.42
tec_cub_qoe_03 20401 10.0.15.43
tec_cub_qoe_04 20401 10.0.15.44
tec_cub_qoe_05 20401 10.0.15.45
clickstream tec_cub_qoe_01 20402 10.0.15.41
tec_cub_qoe_02 20402 10.0.15.42
dnsflow tec_cub_qoe_01 20403 10.0.15.41
tec_cub_qoe_02 20403 10.0.15.42
rsl_cub_dpi_05 205 fullflow tec_cub_qoe_01 20501 10.0.15.41
tec_cub_qoe_02 20501 10.0.15.42
tec_cub_qoe_03 20501 10.0.15.43
tec_cub_qoe_04 20501 10.0.15.44
tec_cub_qoe_05 20501 10.0.15.45
clickstream tec_cub_qoe_04 20502 10.0.15.41
tec_cub_qoe_05 20502 10.0.15.42
dnsflow tec_cub_qoe_04 20503 10.0.15.41
tec_cub_qoe_05 20503 10.0.15.42
uhn_cub_dpi_01 231 fullflow tec_cub_qoe_01 30101 10.0.15.41
tec_cub_qoe_02 30101 10.0.15.42
tec_cub_qoe_03 30101 10.0.15.43
tec_cub_qoe_04 30101 10.0.15.44
tec_cub_qoe_05 30101 10.0.15.45
clickstream tec_cub_qoe_01 30102 10.0.15.41
tec_cub_qoe_02 30102 10.0.15.42
tec_cub_qoe_03 30102 10.0.15.43
dnsflow tec_cub_qoe_01 30103 10.0.15.41
tec_cub_qoe_02 30103 10.0.15.42
tec_cub_qoe_03 30103 10.0.15.43