Lambda monitoring

Data for monitoring

Now we monitor two types of events for monitoring: request and error. First type is all requests, second is failed requests only. Every event is a point in the time series. The point is represented as union of the following data:

• series name (now requests and errors)

• start request time

• tags, indexed data in storage, dictionary: keys - string tag names, values - string, integer, float

• fields, non-indexed data in storage, dictionary: keys - string tag names, values - string, integer, float

‘Requests’ series. Triggered on every request. Each point contains data about corresponding request (execution time and etc).

• tags

  tag name	description
  service	always “lambda-<lambda-id>”
  route	concatenation of a request method and a request resource (POST:/main)
  status_code	http status code of response

• fields

  fields	description
  request_id	request id
  execution_time	request execution time

‘Errors’ series. Triggered on failed request. Each point contains error_code of luna error.

• tags

  tag name	description
  service	always “lambda-<lambda-id>”
  route	concatenation of a request method and a request resource (POST:/main)
  status_code	http status code of response
  error_code	luna error code

• fields

  fields	description
  request_id	request id

Custom Monitoring

For base monitoring usage information see monitoring.

It is possible to create custom monitoring points. For example, there is a need to send data about how long it took to download or process images.

It is possible to specify your own series, tags and fields, but there will always be a mandatory tag “service” with the “lambda-<lambda-id>” value.

For add custom monitoring point follow steps:

1. Add file monitoring_points.py to lambda archive with the following content:

   monitoring_points.py

   from luna_lambda_tools.public.monitoring import CustomMonitoringPoint
   
   
   class TestMonitoringPoint(CustomMonitoringPoint):
       """Test monitoring point"""
   
       series = "test_monitoring"
   

   There are several rules for this file:

   • The number of points can be any.

   • Each point class name may be specify by any unique name.

   • Each point class must be inherited by CustomMonitoringPoint

   • Each point class must contain series attribute to specify monitoring series.

2. Set monitoring points from monitoring_points.py to lambda_main.py and specify tags and fields:

   There are several general rules for this file:

   • Enable monitoring. See INFLUX_MONITORING setting in basic principles of configuration.

   • If monitoring is unavailable, the points will not be sent without any errors.

   • Specify tags by set dictionary with tags to pointTags named argument.

   • Specify fileds by set dictionary with fields to pointFields named argument.

   Warning

   There are diferencies between standalone, handlers and tasks lambda monitoring mechanism. See description below.

   Send points using request.sendToMonitoring function for standalone or handlers lambda:

   lambda_main.py

   import asyncio
   from time import time
   
   from luna_lambda_tools import StandaloneLambdaRequest
   from monitoring_points import TestMonitoringPoint
   
   
   async def main(request: StandaloneLambdaRequest) -> dict:
       # start execution time
       stt = time()
   
       # do some logic
       await asyncio.sleep(1)
   
       # send monitoring point with execution time
       request.sendToMonitoring(
           (TestMonitoringPoint(pointTags={"lambda_type": "standalone"}, pointFields={"execution_time": time() - stt}),)
       )
       return {"result": "lambda result"}
   

   request example

   from luna3.luna_lambda.luna_lambda import LambdaApi
   
   SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
   SERVER_API_VERSION = 1
   lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
   lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
   
   
   def makeRequest():
       reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId)
       return reply
   
   
   if __name__ == "__main__":
       response = makeRequest()
       print(response.json)
   

   Send points using self.sendToMonitoring function for tasks lambda:

   To run lambda tasks example, refer to the task processing description here.

   lambda_main.py

   import asyncio
   from time import time
   
   from luna_lambda_tools.public.tasks import BaseLambdaTask
   from monitoring_points import TestMonitoringPoint
   
   
   class LambdaTask(BaseLambdaTask):
       """Lambda task"""
   
       async def splitTasksContent(self, content: dict) -> list[dict]:
           """Split task content to sub task contents"""
           stt = time()
           # do some logic
           await asyncio.sleep(0.5)
           self.sendToMonitoring(
               (
                   TestMonitoringPoint(
                       pointTags={"lambda_type": "tasks", "target": "split_content"},
                       pointFields={"execution_time": time() - stt},
                   ),
               )
           )
           return [content]
   
       async def executeSubtask(self, subtaskContent: dict) -> dict | list:
           """Execute current sub task processing"""
           stt = time()
           # do some logic
           await asyncio.sleep(0.5)
           self.sendToMonitoring(
               (
                   TestMonitoringPoint(
                       pointTags={"lambda_type": "tasks", "target": "execute_subtask"},
                       pointFields={"execution_time": time() - stt},
                   ),
               )
           )
           return {"result": "Some lambda-tasks result"}
   

   request example

   from luna3.tasks.tasks import TasksApi
   
   TASKS_ORIGIN = "http://tasks_address:tasks_port"  # Replace by your values before start
   TASKS_API_VERSION = 2
   tasksApi = TasksApi(origin=TASKS_ORIGIN, api=TASKS_API_VERSION)
   lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
   
   
   def makeRequest():
       reply = tasksApi.taskLambda(content={"lambda_id": lambdaId}, accountId=accountId)
       return reply
   
   
   if __name__ == "__main__":
       response = makeRequest()
       print(response.json)
   

Database

Monitoring is implemented as data sending to an influx database.