Monitoring¶
Data for monitoring¶
Now we monitor two types of events for monitoring: request and * error*. First type is all requests, second is a failed requests only. Every event is a point in the time series. The point is represented as union of the following data:
series name (now requests and errors)
start request time
tags, indexed data in storage, dictionary: keys - string tag names, values - string, integer, float
fields, non indexed data in storage, dictionary: keys - string tag names, values - string, integer, float
‘Requests’ series. Triggered on every request. Each point contains a data about corresponding request ( execution time and etc).
tags
tag name
description
service
always “luna-handlers”
route
concatenation of a request method and a request resource (POST:/extractor)
status_code
http status code of response
fields
fields
description
request_id
request id
execution_time
request execution time
‘Errors’ series. Triggered on failed request. Each point contains error_code of luna error.
tags
tag name
description
service
always “luna-handlers”
route
concatenation of a request method and a request resource (POST:/extractor)
status_code
http status code of response
error_code
luna error code
fields
fields
description
request_id
request id
Every handler can add additional tags or fields. For example. Handler of resource /handlers/{handlerId}/events adds tag handler_id.
For a resource “/detector”, “/extractor”, “/handlers/{handler_id}/events” there are additional fields:
fields
fields
description
task_execution_time
sdk task execution time
results_queue_transport_time
transport time by results queue
face_detector_queue_transport_time
transport time by face detector queue
face_extractor_queue_transport_time
transport time by face extractor queue
face_warp_attributes_estimator_queue_transport_time
transport time by face warp attributes queue
human_detector_queue_transport_time
transport time by human body detector queue
human_extractor_queue_transport_time
transport time by body extractor queue
face_detector_stage_time
detector stage time
face_extractor_stage_time
extractor stage time
face_warp_estimator_stage_time
warp estimator stage time
human_detector_stage_time
body detector stage time
human_extractor_stage_time
body extractor stage time
faces_detect_time
face detect time
humans_detect_time
human bodies detect time
face_descriptors_extract_time
face descriptors extract time
human_descriptors_extract_time
human body descriptors extract time
basic_attributes_extract_time
basic attributes extract time
gaze_direction_estimation_time
gaze direction estimation time
head_pose_estimation_time
head pose execution time
eyes_attributes_estimation_time
eyes attributes execution time
ags_estimation_time
ags estimation time
quality_estimation_time
quality estimation time
emotions_estimation_time
emotions estimation time
mouth_attributes_estimation_time
mouth attributes estimation time
mask_estimation_time
mask estimation time
glasses_estimation_time
glasses estimation time
liveness_estimation_time
liveness estimation time
save_samples_time
save samples to image store time
save_face_attributes_time
save face attributes to luna-faces time
save_event_attributes_time
save event attributes to luna-events time
load_face_samples_time
load face samples from image-store time
load_body_samples_time
load body samples from image-store time
load_images_for_processing_time
load images for processing time
face_sample_storage_policy_time
save face samples to image store time
body_sample_storage_policy_time
save body samples to image store time
image_origin_storage_policy_time
save origin image to image store time
face_attribute_storage_policy_time
save attributes to luna-faces time
face_storage_policy_time
save face with avatar to luna-faces time
download_images_time
images download from image store time
save_warps_time
save warp to image store
match_policy_time
match by list processing time
‘Usages_statistic’ series. Triggered on every request involving some SDK estimations. Each point contains data on the number of estimations performed.
tags
tag name
description
service
always “luna-handlers”
fields
fields
description
face_detector_usages
face detector usages count
landmarks68_detector_usages
landmarks68 detector usages count
head_pose_estimator_usages
head pose estimator usages count
liveness_estimator_usages
liveness estimator usages count
mask_estimator_usages
mask estimator usages count
emotion_estimator_usages
emotion estimator usages count
mouth_estimator_usages
mouth estimator usages count
eye_estimator_usages
eyes estimator usages count
gaze_estimator_usages
gaze estimator usages count
glasses_estimator_usages
glasses estimator usages count
face_warp_quality_estimator_usages
face warp quality estimator usages count
face_basic_attributes_extractor_usages
face basic attributes extractor usages count
face_descriptor_extractor_usages
face descriptor extractor usages count
body_detector_usages
body detector estimator usages count
body_descriptor_extractor_usages
body descriptor extractor usages count
iso_estimator_usages
iso estimator usages count
‘Licensing’ series. Triggered on each request with liveness estimation if liveness balance expired. Each point contains license check data.
tags
tag name
description
service
always “luna-handlers”
license_status
license status (“ok”, “warning”, “error”, “exception”)
fields
fields
description
liveness_balance
number of liveness estimations before the license expires
warnings
license warning messages
errors
license error messages
Database¶
Monitoring is implemented as data sending to an influx database. You can setup your database credentials in configuration file in section “monitoring”.
Plugins¶
You can realize your own plugin for sending monitoring data. See plugins
Module request monitoring plugin example
- class luna_handlers.crutches_on_wheels.plugins.plugin_examples.request_monitoring_plugin_example.BaseRequestMonitoringPlugin(app)[source]¶
Base class for requests monitoring.
- class luna_handlers.crutches_on_wheels.plugins.plugin_examples.request_monitoring_plugin_example.RequestMonitoringPlugin(app)[source]¶
Example plugin sends a request data for monitoring to third-party source. Only one instance of this class exist during the program execution.
Classes¶
Module contains points for monitoring’s.
- class luna_handlers.crutches_on_wheels.monitoring.points.BaseMonitoringPoint(eventTime)[source]¶
Abstract class for points
- eventTime¶
event time as timestamp
- Type
float
- abstract property fields: Dict[str, Union[int, float, str]]¶
Get tags from point. We supposed that fields are not indexing data
- Returns
dict with fields.
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- abstract property tags: Dict[str, Union[int, float, str]]¶
Get tags from point. We supposed that tags are indexing data
- Returns
dict with tags.
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- class luna_handlers.crutches_on_wheels.monitoring.points.BaseRequestMonitoringPoint(requestId, resource, method, requestTime, service, statusCode)[source]¶
Base class for point which is associated with requests.
- requestId¶
request id
- Type
str
- route¶
concatenation of a request method and a request resource
- Type
str
- service¶
service name
- Type
str
- requestTime¶
a request processing start timestamp
- Type
float
- statusCode¶
status code of a request response.
- Type
int
- property fields: Dict[str, Union[int, float, str]]¶
Get fields
- Returns
“request_id”
- Return type
dict with following keys
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- property tags: Dict[str, Union[int, float, str]]¶
Get tags
- Returns
“route”, “service”, “status_code”
- Return type
dict with following keys
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- class luna_handlers.crutches_on_wheels.monitoring.points.DataForMonitoring(tags=<factory>, fields=<factory>)[source]¶
Class fo storing an additional data for monitoring.
- class luna_handlers.crutches_on_wheels.monitoring.points.RequestErrorMonitoringPoint(requestId, resource, method, errorCode, service, requestTime, statusCode, additionalTags=None, additionalFields=None)[source]¶
Request monitoring point is suspended for monitoring requests errors (error codes)
- errorCode¶
error code
- Type
int
- additionalTags¶
additional tags which was specified for the request
- Type
dict
- additionalFields¶
additional fields which was specified for the request
- Type
dict
- property fields: Dict[str, Union[int, float, str]]¶
Get fields.
- Returns
dict with base fields and additional tags
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- series: str = 'errors'¶
series “errors”
- property tags: Dict[str, Union[int, float, str]]¶
Get tags.
- Returns
dict with base tags, “error_code” and additional tags
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- class luna_handlers.crutches_on_wheels.monitoring.points.RequestMonitoringPoint(requestId, resource, method, executionTime, requestTime, service, statusCode, additionalTags=None, additionalFields=None)[source]¶
Request monitoring point is suspended for monitoring all requests and measure a request time and etc.
- executionTime¶
execution time
- Type
float
- additionalTags¶
additional tags which was specified for the request
- Type
dict
- additionalFields¶
additional fields which was specified for the request
- Type
dict
- property fields: Dict[str, Union[int, float, str]]¶
Get fields.
- Returns
dict with base fields, “execution_time” and additional tags
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- series: str = 'requests'¶
series “request”
- property tags: Dict[str, Union[int, float, str]]¶
Get tags.
- Returns
dict with base tags and additional tags
- Return type
Dict
[str
,Union
[int
,float
,str
]]
- luna_handlers.crutches_on_wheels.monitoring.points.getRoute(resource, method)[source]¶
Get a request route, concatenation of a request method and a request resource :param resource: resource :param method: method
- Returns
{resource}”
- Return type
“{method}
- Return type
str
- luna_handlers.crutches_on_wheels.monitoring.points.monitorTime(monitoringData, fieldName)[source]¶
Context manager for timing execution time.
- Parameters
monitoringData – container for saving result
fieldName – field name
Module implement base class for monitoring
- class luna_handlers.crutches_on_wheels.monitoring.base_monitoring.BaseLunaMonitoring[source]¶
Base class for monitoring
- class luna_handlers.crutches_on_wheels.monitoring.base_monitoring.LunaRequestInfluxMonitoring(credentials, host='localhost', port=8086, ssl=False, flushingPeriod=1)[source]¶
Class for sending data which is associated with request to influx .. attribute:: settings
influxdb settings
- type
InfluxSettings
- flushingPeriod¶
period of flushing points (in seconds)
- Type
int
Module contains classes for sending a data to an influx monitoring.
- class luna_handlers.crutches_on_wheels.monitoring.influx_adapter.BaseMonitoringAdapter(settings, flushingPeriod)[source]¶
Base monitoring adapter.
- client¶
influx client
- Type
InfluxDBClient
- backgroundScheduler¶
runner for periodic flushing monitoring points
- Type
AsyncIOScheduler
- _buffer¶
list of buffering points which is waiting sending to influx
- Type
- flushingPeriod¶
period of flushing points (in seconds)
- Type
float
- _influxSettings¶
current influx settings
- Type
- _job¶
sending monitoring data job
- Type
Job
- addPointsToBuffer(points)[source]¶
Add points to buffer.
- Parameters
points – points
- Return type
None
- static convertPointToDict(point)[source]¶
Convert point to influx client point :param point: point
- Returns
‘time’ - timestamp in nanoseconds
’measurement’ - time series
’tags’ - dict of tags (values are str)
’fields’ - dict of fields
- Return type
dict
- Return type
dict
- class luna_handlers.crutches_on_wheels.monitoring.influx_adapter.InfluxMonitoringAdapter(settings, flushingPeriod)[source]¶
Influx 2.x adaptor. Suspended to send points to an influxdb
- client¶
influx 2.x client
- Type
InfluxDBClient
- bucket¶
influx bucket name
- Type
str