Monitoring
==========

.. _`monitoring`:

.. _influx database: https://www.influxdata.com/products/influxdb-overview/
.. _clickhouse database: https://clickhouse.com/

Data for monitoring
-------------------


Now we monitor two types of events for monitoring: *request*  and *error*. First type is all requests, second is
failed requests only.

**Comparison of data formats for Clickhouse and InfluxDB:**

**InfluxDB:** Each event is presented as a "point" in a time series. The structure of a point includes:

- series name
- start event time
- tags, indexed data in storage, dictionary: keys - string tag names, values - string, integer, float
- fields, non indexed data in storage, dictionary: keys - string tag names, values - string, integer, float

**Clickhouse:** In Clickhouse, the data structure resembles that of a traditional SQL table.
Each event is represented as a **record**, where:

- The **`time`** field contains the record's creation timestamp;
- The **`data`** field contains a JSON object with all the information that would otherwise be distributed across tags and fields in InfluxDB.

**Important**: In Clickhouse, there is no differentiation between "tags" and "fields"—all data is consolidated into a single JSON object within the `data` field.

The structure and the meaning of each monitoring series remain consistent. However, for **Clickhouse**,
data from tags and fields are merged into a single JSON object under the `data` field.
Below are examples for each series:

* *Requests* series.

    Triggered on every request. Each point contains a data about corresponding request
    (execution time and etc).

    **InfluxDB:**

    - tags

        +--------------+----------------------------------------------------------------------------+
        | tag name     |                                description                                 |
        +==============+============================================================================+
        | service      | always "luna-lambda"                                                       |
        +--------------+----------------------------------------------------------------------------+
        | route        | concatenation of a request method and a request resource (POST:/plugin)    |
        +--------------+----------------------------------------------------------------------------+
        | status_code  | http status code of response                                               |
        +--------------+----------------------------------------------------------------------------+

    - fields

        +----------------+------------------------+
        |     fields     |      description       |
        +================+========================+
        | request_id     | request id             |
        +----------------+------------------------+
        | execution_time | request execution time |
        +----------------+------------------------+

    **ClickHouse JSON `data` field Example:**

    .. code-block:: json

        {
            "service": "luna-lambda",
            "route": "POST:/plugin",
            "status_code": 204,
            "request_id": "1536751345,6a5c2191-3e9b-f5a4-fc45-3abf43625c5f",
            "execution_time": 123.45
        }

* *Errors* series.

    Triggered on failed request. Each point contains *error_code* of luna error.

    **InfluxDB:**

    - tags

        +--------------+----------------------------------------------------------------------------+
        | tag name     |                                description                                 |
        +==============+============================================================================+
        | service      | always "luna-lambda"                                                       |
        +--------------+----------------------------------------------------------------------------+
        | route        | concatenation of a request method and a request resource (POST:/plugins)   |
        +--------------+----------------------------------------------------------------------------+
        | status_code  | http status code of response                                               |
        +--------------+----------------------------------------------------------------------------+
        | error_code   | luna error code                                                            |
        +--------------+----------------------------------------------------------------------------+

    - fields

        +----------------+------------------------+
        |     fields     |      description       |
        +================+========================+
        | request_id     | request id             |
        +----------------+------------------------+

    **ClickHouse JSON `data` field Example:**

    .. code-block:: json

        {
            "service": "luna-lambda",
            "route": "POST:/plugins",
            "status_code": 400,
            "error_code": 13037,
            "request_id": "1536751345,6a5c2191-3e9b-f5a4-fc45-3abf43625c5f",
        }


Database
--------

You can refer to documentation for `influx database`_ and `clickhouse database`_ to compare the databases and choose what benefit your needs more.
Note that clickhouse might be the better choice for aggregation
You can setup your database credentials in `configuration file <config.html>`_ in section *"monitoring"*.

Classes
-------

.. automodule:: luna_lambda.crutches_on_wheels.cow.monitoring.points
   :members:

.. automodule:: luna_lambda.crutches_on_wheels.cow.monitoring.base_monitoring
   :members:

.. automodule:: luna_lambda.crutches_on_wheels.cow.monitoring.influx_adapter
   :members:

.. automodule:: luna_lambda.crutches_on_wheels.cow.monitoring.clickhouse_adapter
   :members: