.. _Managing Network Accounting and Bandwidth:

Managing Network Accounting and Bandwidth
-----------------------------------------

This section explains how to perform the following tasks in Virtuozzo:

-  configuring network classes

-  viewing network traffic statistics

-  turning on and off network bandwidth management

-  configuring bandwidth limits

.. _Network Traffic Parameters:

Network Traffic Parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~

The table below summarizes the network traffic parameters that you can control in Virtuozzo.

+--------------------+--------------------------------------------------------+
| Parameter          | Description                                            |
+====================+========================================================+
| ``traffic_shaping``| If set to ``yes``, traffic limitations for outgoing    |
|                    | traffic are set for virtual machines and containers.   |
|                    | The default is ``no``.                                 |
+--------------------+--------------------------------------------------------+
| ``bandwidth``      | This parameter lists all network adapters installed on |
|                    | the hardware node and their bandwidth.                 |
+--------------------+--------------------------------------------------------+
| ``totalrate``      | This parameter defines the bandwidth to allocate for   |
|                    | each network class. It is active if traffic shaping is |
|                    | turned on.                                             |
+--------------------+--------------------------------------------------------+
| ``rate``           | If traffic shaping is turned on, this parameter        |
|                    | specifies the bandwidth guarantee for virtual machines |
|                    | and containers.                                        |
+--------------------+--------------------------------------------------------+
| ``ratebound``      | If this parameter is set to ``yes``, the bandwidth     |
|                    | guarantee (the global rate parameter) is also the      |
|                    | limit for the virtual machine or container, and the    |
|                    | virtual machine or container cannot borrow the         |
|                    | bandwidth from the ``totalrate`` bandwidth pool.       |
+--------------------+--------------------------------------------------------+

.. _Configuring Network Classes:

Configuring Network Classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Virtuozzo allows you to track the inbound and outbound network traffic as well as to shape the outgoing traffic for virtual machines and containers. To provide the ability to distinguish between types of traffic, e.g., domestic and international, a concept of network classes is introduced. A network class is a range of IP addresses for which Virtuozzo accounts and shapes the traffic.

Classes are specified in the ``/etc/vz/conf/networks_classes`` file. The file is in the ASCII format, and all empty lines and lines starting with the ``#`` sign are ignored. Other lines have the following format:

::

    <class_id> <IP_address>/<prefix_length>

where ``<class_id>`` defines the network class ID, and the ``<IP_address>/<prefix_length>`` pair defines the range of IP addresses for this class. There may be several lines for each class.

Classes 0 and 1 have special meanings:

-  Class 0 defines the IP address range for which no accounting is performed. Usually, it corresponds to the hardware node subnet (the node itself and its virtual machines and containers). Setting up class 0 is not required; however, its correct setup improves performance.

-  Class 1 is defined by Virtuozzo to match any IP address. It must be always present in the network classes definition file. Therefore, it is suggested not to change the default line in the ``networks_classes`` file.

   ::

       1 0.0.0.0/0

   If your virtual machines and containers are using IPv6 addresses, you can also add the following line to this file:

   ::

       1 ::/0

Other classes should be defined after class 1. They represent exceptions from the "matching-everything" rule of class 1. The example below illustrates a possible configuration of the network classes definition file containing rules for both IPv4 and IPv6 addresses:

::

    # Hardware node networks
    0 192.168.0.0/16
    0 fe80::/64
    # any IP address (all traffic)
    1 0.0.0.0/0
    1 ::/0
    # class 2 - addresses for the "foreign" traffic
    2 10.0.0.0/8
    2 2001:db88::/64
    # inside "foreign" network there
    # is a hole belonging to "local" traffic
    1 10.10.16.0/24
    1 2001:db88:3333::/64

In this example, IPv4 addresses in the range of ``192.168.0.0`` to ``192.168.255.255`` and IPv6 addresses in the range of ``fe80::`` to ``fe80::ffff:ffff:ffff:ffff`` are treated as class 0 addresses and no accounting is done for the traffic from virtual machines and containers destined to these addresses.

Class 2 matches the following IP addresses:

-  IPv4 addresses from ``10.0.0.0`` to ``10.255.255.255`` with the exception of addresses in the sub-range of ``10.10.16.0`` to ``10.10.16.255``, which are treated as class 1.

-  IPv6 addresses from ``2001:db88::`` to ``2001:db88::ffff:ffff:ffff:ffff`` with the exception of addresses in the sub-range of ``2001:db88:3333::`` to ``2001:db88:3333::ffff:ffff:ffff:ffff``, which are also treated as class 1.

All other IP addresses (both IPv4 and IPv6) belong to class 1.

To apply changes after editing the ``/etc/vz/conf/networks_classes`` file, restart either the virtual machine(s) or/and container(s) for which changes have been made or the hardware node itself if the changes are global.

.. _Viewing Network Traffic Statistics:

Viewing Network Traffic Statistics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In Virtuozzo, you can view the current network traffic statistics for virtual machines and containers using the ``vznetstat`` utility. For example:

::

    # vznetstat
    UUID         Net.Class  Input(bytes)  Input(pkts)  Output(bytes)  Output(pkts)
    0            0          566093064     2800575      3120481        41736
    47406484...  0          67489         155          8033           110
    fbb30afa-... 0          9369          78           12692          71

By default, ``vznetstat`` shows network statistics for both virtual machines and containers. Keep in mind that the ``vznetstat`` utility displays statistics only about virtual machines and containers that were started at least once.

The ``vznetstat`` utility displays the following information:

+--------------------+--------------------------------------------------------+
| Column             | Description                                            |
+====================+========================================================+
| UUID               | UUID assigned to virtual machine or container.         |
+--------------------+--------------------------------------------------------+
| Net.Class          | ID of the network class for which network statistics   |
|                    | is calculated.                                         |
+--------------------+--------------------------------------------------------+
| Input(bytes)       | Amount of incoming traffic, in bytes.                  |
+--------------------+--------------------------------------------------------+
| Input(pkts)        | Amount of incoming traffic, in packets.                |
+--------------------+--------------------------------------------------------+
| Output(bytes)      | Amount of outgoing traffic, in bytes.                  |
+--------------------+--------------------------------------------------------+
| Output(pkts)       | Amount of outgoing traffic, in packets.                |
+--------------------+--------------------------------------------------------+

For example, from the command output above, you can see that around 9 MB of data were uploaded to the container ``MyCT``, (2) about 12 MB were downloaded from it, and all the traffic was exchanged with servers from class 0 networks.

If necessary, you can view network traffic statistics separately for virtual machine or container by passing the ``-t`` option to ``vznetstat``:

-  For containers only:

   ::

       # vznetstat -t ct
       CTID         Net.Class  Input(bytes)  Input(pkts)  Output(bytes)  Output(pkts)
       0            0          566093064     2800575      3120481        41736
       fbb30afa-... 0          9369          78           12692          71

-  For virtual machines only:

   ::

       # vznetstat -t vm
       UUID         Net.Class  Input(bytes)  Input(pkts)  Output(bytes)  Output(pkts)
       0            0          566093064     2800575      3120481        41736
       47406484...  0          67489         155          8033           110

You can also view network statistics for a particular virtual machine or container by specifying its ID after the ``-v`` option, for example:

::

    # vznetstat -v fbb30afa-e770-4081-9d9e-6b9c262eb091
    UUID         Net.Class  Input(bytes)  Input(pkts)  Output(bytes)  Output(pkts)
    fbb30afa-... 0          9369          78           12692          71

This command displays statistics only for the container ``MyCT``.

.. _Configuring Traffic Shaping:

Configuring Traffic Shaping
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Traffic shaping (also known as network bandwidth management) allows you to control what network bandwidth a virtual machine or container may use for outgoing traffic. This feature is disabled by default.

.. note::

    1. Traffic within a host cannot be shaped in the current version of Virtuozzo. This includes traffic between virtual machines and containers on the same host and between those and the host itself.

    2. Incoming traffic cannot be shaped for virtual machines and containers in the current version of Virtuozzo.

The following parameters control traffic shaping in Virtuozzo:

-  ``TRAFFIC_SHAPING``, enables and disables traffic shaping.

-  ``BANDWIDTH``, sets bandwidth for specific network adapters.

-  ``TOTALRATE``, sets the size of a bandwidth pool divided between virtual machines and containers on the host.

-  ``RATEMPU``, limits packet rate in addition to byte rate.

-  ``RATE``, sets a bandwidth guarantee for virtual machines and containers.

-  ``RATEBOUND``, forces ``RATE`` as a limit.

Traffic shaping in Virtuozzo works as follows. The bandwidth pool for a given network class (set by ``TOTALRATE``) is divided among the virtual machines and containers transmitting data proportionally to their ``RATE`` settings. If the sum of ``RATE`` values of all virtual machines and containers transmitting data does not exceed ``TOTALRATE``, each virtual machine or container gets the bandwidth equal to or greater than its ``RATE`` value (unless ``RATEBOUND`` is enabled for said virtual machine or container). If the sum of ``RATE`` values of all virtual machines and containers transmitting data exceeds the ``TOTALRATE`` value, each virtual machine or container may get less than its ``RATE`` value.

To enable and configure traffic shaping, do the following:

1. Set the value of ``TRAFFIC_SHAPING`` to ``yes`` in the global configuration file ``/etc/vz/vz.conf``.

2. Set the parameters ``BANDWIDTH``, ``TOTALRATE`` in ``/etc/vz/vz.conf``.

3. If required, set the optional parameters ``RATEMPU``, ``RATE``, ``RATEBOUND`` in ``/etc/vz/vz.conf``.

4. If required, set ``RATE`` and ``RATEBOUND`` for specific virtual machines and containers with ``prlctl set --rate`` and ``prlctl set --ratebound`` commands.

5. To apply changes, restart either the virtual machines and containers for which changes have been made or the hardware node itself if the changes are global.

The following sections provide more details on and explain how to set traffic shaping parameters listed above.

.. _Setting BANDWIDTH Parameter:

Setting BANDWIDTH Parameter
^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``BANDWIDTH`` parameter is used for shaping traffic of specific network adapters. For example, for two Fast Ethernet cards, a typical setting may look like ``enp0s5 enp0s6:100000`` where ``enp0s5`` and ``enp0s6`` are network adapter names. By default, the parameter is set to ``100000`` which corresponds to a 100 Mbps Fast Ethernet card.

.. _Setting TOTALRATE Parameter:

Setting TOTALRATE Parameter
^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``TOTALRATE`` parameter specifies the size of a bandwidth pool for specific network classes on the host. Virtual machines and containers can borrow bandwidth from the pool for communicating with hosts from the corresponding network class. The parameter thus limits the total available outgoing traffic for a network class that virtual machines and containers can consume.

The parameter is set as ``<NIC>:<network_class>:<bandwidth_in_Kbps>``. For example, to set the pool size to 4 Mbps for network class 1 on the Ethernet adapter ``enp0s5``, set ``TOTALRATE`` to ``enp0s5:1:4000``. Multiple entries can be separated by spaces, e.g., ``enp0s5:1:4000 enp0s6:2:8000``.

.. _Setting RATEMPU Parameter:

Setting RATEMPU Parameter
^^^^^^^^^^^^^^^^^^^^^^^^^

The optional ``RATEMPU`` parameter (where "MPU" stands for "minimum packet unit") limits the packet rate by making packets smaller than MPU in size consume HTB tokens. With it, small packets can be accounted as larger ones and limited by ``TOTALRATE`` and ``RATE`` parameters. Approximately, the maximum packets per second rate can be calculated as ``TOTALRATE`` / ``RATEMPU``.

This parameter has the following syntax: ``<NIC>:<network_class>[:<MPU_in_bytes_per_packet>]``. If the part ``<MPU_in_bytes_per_packet>`` is omitted, the default value of 1000 bytes is used. Multiple entries can be separated by spaces, e.g., ``enp0s5:1:2000 enp0s6:2:4000``. To set the ``RATEMPU`` parameter for all known Ethernet devices set ``<NIC>`` to an asterisk (``*``). For example, to set the minimal packet size to 2 Kb for network class 1 on all the Ethernet adapters on the node, change the value to ``*:1:2000``.

.. _Setting RATE and RATEBOUND Parameters:

Setting RATE and RATEBOUND Parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The optional ``RATE`` parameter allows you to guarantee virtual machines and containers outgoing bandwidth to destinations in a specific network class on a specific Ethernet device. The guaranteed bandwidth is not a limit (unless the ``RATEBOUND`` parameter is also set to ``on``, see below). A virtual machine or container can additionally obtain unused bandwidth from the bandwidth pool defined by ``TOTALRATE``.

You can set the guaranteed bandwidth in two ways:

1. For all virtual machines and containers on the host by setting ``RATE`` in the global configuration file ``/etc/vz/vz.conf``.

   The parameter is set as ``<NIC>:<network_class>:<bandwidth_in_Kbps>``. For example, to guarantee all virtual machines and containers on the host the bandwidth of at least 8 Kbps for outgoing traffic in network class 1 on the Ethernet device ``enp0s5``, set the ``RATE`` parameter to ``enp0s5:1:8``.

2. For specific virtual machines or containers by means of the ``prlctl set --rate`` command.

   For example, to guarantee the container ``MyCT`` the bandwidth of at least 16 Kbps for outgoing traffic in network class 1, run

   ::

       # prlctl set MyCT --rate 1:16

   This command sets the bandwidth for the default network adapter only. If you need to set bandwidth for other network adapters, set ``RATE`` in ``/etc/vz/vz.conf``.

   .. note:: It is recommended to increase ``RATE`` value in 8 Kbps increments and set it to at least 8 Kbps.

The optional ``RATEBOUND`` parameter specifies whether the network bandwidth guaranteed by ``RATE`` is also a limit. By default, this feature is disabled for all newly created virtual machines and containers so they may additionally obtain unused bandwidth from the pool set by ``TOTALRATE``.

You can limit bandwidth of virtual machines and containers to the guaranteed value as follows:

1. For all virtual machines and containers on the host by setting ``RATEBOUND`` in the global configuration file ``/etc/vz/vz.conf`` (omitted by default).

2. For specific virtual machines or containers by means of the ``prlctl set --ratebound`` command. For example:

   ::

       # prlctl set MyCT --ratebound yes

If set, values of ``RATE`` and ``RATEBOUND`` provided for specific virtual machines and containers are chosen over global values in ``/etc/vz/vz.conf``.

.. _Traffic Shaping Example:

Traffic Shaping Example
^^^^^^^^^^^^^^^^^^^^^^^

The example below illustrates a scenario when the containers ``MyCT1`` and ``MyCT2`` have ``RATEBOUND`` set to ``no``, and the virtual machine ``MyVM`` has ``RATEBOUND`` set to ``yes``. With the default ``TOTALRATE`` of 4096 Kbps and ``RATE`` of 8 Kbps, the bandwidth pool will be distributed as follows:

+----------------+----------------+----------------+------------------------------+
| MyCT1          | MyCT2          | MyVM           | Consumed Bandwidth           |
+================+================+================+==============================+
| transmits      | idle           | idle           | MyCT1: 4096 Kbps             |
+----------------+----------------+----------------+------------------------------+
| idle           | idle           | transmits      | MyVM: 8 Kbps                 |
+----------------+----------------+----------------+------------------------------+
| transmits      | transmits      | idle           | MyCT1: 2048 Kbps             |
|                |                |                | MyCT2: 2048 Kbps             |
+----------------+----------------+----------------+------------------------------+
| transmits      | idle           | transmits      | MyCT1: 4032 Kbps             |
|                |                |                | MyVM: 8 Kbps                 |
+----------------+----------------+----------------+------------------------------+
| transmits      | transmits      | transmits      | MyCT1: 2016 Kbps             |
|                |                |                | MyCT2: 2016 Kbps             |
|                |                |                | MyVM: 8 Kbps                 |
+----------------+----------------+----------------+------------------------------+

