.. _Exporting Cluster Statistics in XML Format:

Exporting Cluster Statistics in XML Format
==========================================

You can use the ``vstorage stat -xml`` command to export current cluster statistics in XML format. The statistics includes information about all storage components: cluster, MDSes, CSes, clients, and nodes.

The resulting XML file has the following main elements:

- Cluster identification tags:

  - ``cluster_id``, the cluster identifier generated automatically during cluster creation.
  - ``cluster_name``, the string value with the cluster name specified during cluster creation.
  - ``status``, the current cluster status that may be one of the following:

    - ``healthy``, all CSes are active.
    - ``unknown``, insufficient information about the cluster state (e.g., because the master MDS server was elected a while ago).
    - ``degraded``, some CSes are inactive.
    - ``failure``, the cluster has too many inactive CSes; automatic replication is disabled.
    - ``SMART warning``, one or more physical disks attached to cluster nodes reported a S.M.A.R.T. error.

- Nested tags containing information about the cluster:
  - :ref:`space`, information on total and logical storage space.
  - :ref:`rjournal`, ???
  - :ref:`license`, license status.
  - :ref:`repl`, redundancy settings.
  - :ref:`chunks`, the number of chunks in every chunk state.

    - replicated, redundancy type.

  - :ref:`fs_stat`, statistics on files, inodes, file maps, chunks, and chunk replicas.
  - :ref:`io_stat`, statistics on IO performance.

- Nested tags containing information on MDSes, CSes, clients, and nodes:

  - :ref:`mds_list`, a list of tags containing statistics on MDS.
  - :ref:`cs_list`, a list of tags containing statistics on CS.
  - :ref:`clients_list`, a list of tags containing information on clients.
  - :ref:`host_list`, a list of tags containing information on hosts.

Additionally, some of the tags may contain statistics on the values of various counters at different time frames under the :ref:`rate` tag.

The following sections describe the nested tags and their values in detail.

.. _space:

space
-----

The ``space`` tag contains information on total and logical storage space in the following tags:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``allocatable``                     | Amount of logical disk space available to clients. Allocatable disk space is calculated on the basis of    |
|                                     | the current replication parameters and free disk space on chunk servers. It may also be limited by license.|
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``effective_total``                 | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``allocatable_raw``                 | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``total_raw``                       | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``total``                           | Total physical space on all disks, in bytes.                                                               |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``free``                            | Unused physical space, in bytes.                                                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``tiers``                           | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _rjournal:

rjournal
--------

.

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``epoch``                           | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``epoch_uptime``                    | Time elapsed since the MDS master server election.                                                         |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``round``                           | Number of the voting round.                                                                                |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``master_id``                       | Identifier of the master MDS.                                                                              |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _license:

license
-------

The ``license`` tag contains the ``status`` value that can be one of the following:

- trial license.
- active.
- .

.. _repl:

repl
----

The ``repl`` tag contains replication settings and nests the following tags: 

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``norm``                            | Normal number of chunk replicas.                                                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``limit``                           | Limit after which a chunk gets blocked until recovered.                                                    |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``max``                             | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _chunks:

chunks
------

The ``chunks`` tag contains percentages of chunks with the statuses described below and a nested ``replicated`` tag.

.. include:: ../../virtuozzo_infrastructure_platform_administrators_command_line_guide/source/monitoring-clusters/monitoring-chunk-servers.rst
   :start-line: 158
   :end-line: 190

The ``replicated`` tag contains statistics on the number of replicated chunks under the :ref:`rate` tag.

.. _fs_stat:

fs_stat
-------

The ``fs_stat`` tag contains statistics on files, inodes, file maps, chunks and chunk replicas. 

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``used_size``                       | Used logical/physical (???) space, in bytes.                                                               |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``files``                           | Number of files in the filesystem.                                                                         |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``inodes``                          | Number of inodes in the filesystem.                                                                        |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``file_maps``                       | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``chunk_maps``                      | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``chunk_nodes``                     | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _io_stat:

io_stat
-------

The ``io_stat`` tag contains :ref:`rate` statistics on cluster IO activity excluding replication, replication IO activity, data synchronization, and the following values:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``reads``                           | Statistics on data reads, in bytes per second.                                                             |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``read_ops``                        | Statistics on data reads, in operations per second.                                                        |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``writes``                          | Statistics on data writes, in bytes per second.                                                            |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``write_ops``                       | Statistics on data writes, in operations per second.                                                       |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``repl_reads``                      | Statistics on data replication reads, in operations per second.                                            |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``repl_writes``                     | Statistics on data replication writes, in operations per second.                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``sync``                            | Statistics on synchronization, in operations per second.                                                   |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``datasync``                        | Statistics on data synchronization, in operations per second.                                              |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``queue_aver``                      | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``queue_max``                       | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``hot_nodes``                       | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``last_balanced``                   | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``last_balance_uptime``             | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _mds_list:

mds_list
--------

The ``mds_list`` tag contains the list of MDSes wrapped in ``mds`` tags with the following structure:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``id``                              | Automatically generated global MDS identifier.                                                             |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``status``                          | Current MDS status:                                                                                        |
|                                     |                                                                                                            |
|                                     | - ``avail``, MDS is running and responds to requests.                                                      |
|                                     | - ``stale``, MDS has not responded to requests for some time.                                              |
|                                     | - ``unavail``, MDS has terminated and broke the connection to master.                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``ctime``                           | :ref:`rate` statistics on time spent writing to the local journal.                                         |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``commits``                         | :ref:`rate` statistics on local journal commits.                                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``cpu_usage``                       | MDS CPU usage :ref:`rate` statistics.                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``mem_usage``                       | Number of pages the MDS has in physical memory.                                                            |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``uptime``                          | Time elapsed since MDS startup.                                                                            |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``host_info``                       | Information on the host where the MDS runs; nests the following tags:                                      |
|                                     |                                                                                                            |
|                                     | - ``host``, IP address and port that the MDS daemon listens on.                                            |
|                                     | - ``host_id``, automatically generated host identifier.                                                    |
|                                     | - ``location``, failure domain of the host in the ``room.line.rack`` format.                               |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``build_version``                   | MDS build version.                                                                                         |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _cs_list:

cs_list
-------

The ``cs_list`` tag contains the list of MDSes wrapped in ``cs`` tags with the following structure:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``id``                              | Automatically generated global CS identifier.                                                              |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``status``                          | Current CS status:                                                                                         |
|                                     |                                                                                                            |
|                                     | - ``active``, CS is running and responds to requests.                                                      |
|                                     | - ``inactive``, CS has not responded to requests for some time, replication is not started.                |
|                                     | - ``offline``, CS has not responded for a long time, chunks are being replicated.                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``space``                           | Amount of physical space on CS in bytes: total, free, and available.                                       |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``replcas``                         | Number of chunk replicas stored on the CS.                                                                 |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``available``                       | Space available on the CS.                                                                                 |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``alloc_cost``                      | Cost of allocating a chunk on this CS.                                                                     |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``tier``                            | Tier assigned to the CS.                                                                                   |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``adm_status``                      | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``act_status``                      | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``err_status``                      | CS error status. If not ``none``, the CS is not used for chunk allocation.                                 |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``last_err``                        | Previous CS error status.                                                                                  |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``last_err_uptime``                 | Time elapsed since the previous CS error.                                                                  |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``last_link_err``                   | Last CS link error status.                                                                                 |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``last_link_err_uptime``            | Time elapsed since the previous CS link error.                                                             |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``rmw``                             | :ref:`rate` statistics on the number of read-modify-write sequences due to unaligned IO.                   |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``jrmw``                            | :ref:`rate` statistics on the number of read-modify-write sequences served from the SSD journal.           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``io_stat``                         | Tag that nests IO statistics.                                                                              |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``chunks``                          | Number of chunks in various states.                                                                        |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``latency``                         | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``net_stat``                        | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``host_info``                       | Information on the host running the CS in the following tags:                                              |
|                                     |                                                                                                            |
|                                     | - ``host``, IP address and port that the CS daemon listens on.                                             |
|                                     | - ``host_id``, automatically generated host identifier.                                                    |
|                                     | - ``location``, failure domain of the host in the ``room.line.rack`` format.                               |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``features``                        | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``build_version``                   | CS build version.                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

The ``io_stat`` tag contains :ref:`rate` statistics on the following counters:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``reads``                           | Data reads, in bytes per second.                                                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``read_ops``                        | Data reads, in operations per second.                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``writes``                          | Data writes, in bytes per second.                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``write_ops``                       | Data writes, in operations per second.                                                                     |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``repl_reads``                      | Data replication reads, in operations per second.                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``repl_writes``                     | Data replication writes, in operations per second.                                                         |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``maps``                            | Number of map, in operations per second.                                                                   |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``sync``                            | Synchronization, in operations per second.                                                                 |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``datasync``                        | Data synchronization, in operations per second.                                                            |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``iowait``                          | Percentage of time spent waiting for IO operations including synchronization.                              |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``syncwait``                        | Percentage of time spent waiting for synchronization.                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``ioqueue``                         | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``jfull``                           | Percentage of SSD journal to be stored on HDD.                                                             |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _clients_list:

clients_list
------------

The ``clients_list`` tag contains the list of ``clients`` tags with the following structure:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``id``                              | Automatically generated global client identifier.                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``period_ms``                       | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``leases``                          | Number of ``shared`` and ``exclusive`` leases belonging to the client.                                     |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``reads``                           | Data reads in bytes per second.                                                                            |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``read_ops``                        | Data reads in operations per second.                                                                       |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``writes``                          | Data writes in bytes per second.                                                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``write_ops``                       | Data writes in operations per second.                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``fsyncs``                          | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``latency``                         | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``host_info``                       | Information on the host running the client in the following tags:                                          |
|                                     |                                                                                                            |
|                                     | - ``host``, IP address and port that the CS daemon listens on.                                             |
|                                     | - ``host_id``, automatically generated host identifier.                                                    |
|                                     | - ``location``, failure domain of the host in the ``room.line.rack`` format.                               |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``build_version``                   | Client build version.                                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _host_list:

host_list
---------

The ``host_list`` tag contains the list of ``host_stat`` tags with the following structure:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``host_roles``                      | Contains tags with the following numbers:                                                                  |
|                                     |                                                                                                            |
|                                     | - ``num_act_cses``, number of active CSes.                                                                 |
|                                     | - ``num_all_cses``, number of all configured CSes.                                                         |
|                                     | - ``num_act_mdses``, number of active MDSes.                                                               |
|                                     | - ``num_act_mdses``, number of all configured MDSes.                                                       |
|                                     | - ``num_clients``, number of all configured clients.                                                       |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``net_stat``                        | .                                                                                                          |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``space``                           | Amount of physical space on the host in bytes: total, free, and available.                                 |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``host_info``                       | Information on the host in the following tags:                                                             |
|                                     |                                                                                                            |
|                                     | - ``host``, host IP address.                                                                               |
|                                     | - ``host_id``, automatically generated host identifier.                                                    |
|                                     | - ``location``, failure domain of the host in the ``room.line.rack`` format.                               |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``host_ip``                         | Host IP address.                                                                                           |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+

.. _rate:

rate
----

The ``rate`` tag can be wrapped in various tags and contains statistics of corresponding counters in the following fields:

+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| Tag                                 | Description                                                                                                |
+=====================================+============================================================================================================+
| ``total``                           | (Optional) Maximum value of the counter in question.                                                       |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``units``                           | (Optional) Unit of measure.                                                                                |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``avg5s``                           | Average value during 5-second period.                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``avg1m``                           | Average value during 1-minute period.                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``avg5m``                           | Average value during 5-minute period.                                                                      |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
| ``avg15m``                          | Average value during 10-minute period.                                                                     |
+-------------------------------------+------------------------------------------------------------------------------------------------------------+
