Monitoring¶
Data for monitoring¶
Now we monitor two types of events for monitoring: request and error. First type is all requests, second is failed requests only. Every event is a point in the time series. The point is represented as union of the following data:
series name (now requests and errors)
start request time
tags, indexed data in storage, dictionary: keys - string tag names, values - string, integer, float
fields, nonindexed data in storage, dictionary: keys - string tag names, values - string, integer, float
‘Requests’ series. Triggered on every request. Each point contains a data about corresponding request (execution time and etc).
tags
tag name
description
service
always “luna-admin”
route
concatenation of a request method and a request resource (GET:/accounts)
status_code
http status code of response
fields
fields
description
request_id
request id
execution_time
request execution time
‘Errors’ series. Triggered on failed request. Each point contains error_code of luna error.
tags
tag name
description
service
always “luna-admin”
route
concatenation of a request method and a request resource (GET:/accounts)
status_code
http status code of response
error_code
luna error code
fields
fields
description
request_id
request id
Every handler can add additional tags or fields.
Database¶
Monitoring is implemented as data sending to an influx database. You can setup your database credentials in configuration file in section “monitoring”.
Aggregated statistics¶
For statistical purposes monitoring data is being aggregated and downsampled. Data includes average response time, total responses count, 95 and 99 percentiles, errors counts, sdk estimators usage. To perform this task influx tasks are used. Each task runs around 1am Moscow time, fetches all points from the previous day and performs required calculations. To change schedule either edit flux scripts in base_scripts/flux before creating tasks or edit tasks in the Influx UI after creating them. Resulting data is stored in the bucket luna_monitoring_aggregated, its created along with the tasks. To create tasks run: python base_scripts/influx2_cli.py create_usage_task –token=”INFLUX TOKEN” –host=localhost –org=luna Additionally, the script supports getting parameters from the configurator: python base_scripts/influx2_cli.py create_usage_task –luna-config http://configurator:5070/1