Handlers lambda development

Here is handlers lambda development description.

More information about lambda types and differences of handlers and others available at lambda types description.

Handlers lambda requirements

Handlers lambda has several requirements to addition with basic requirements:

  • Luna Faces available by credentials from Luna-Configurator

  • Luna Events available by credentials from Luna-Configurator; it can be disabled using ADDITIONAL_SERVICES_USAGE setting and it this case lambda should provide for work without Luna-Events usage

  • Luna Python Matcher available by credentials from Luna-Configurator

  • Luna Faces/Bodies/Images Samples Store available by credentials from Luna-Configurator; it can be disabled using ADDITIONAL_SERVICES_USAGE setting and it this case lambda should provide for work without Luna-Image-Store usage

  • Luna Remote SDK available by credentials from Luna-Configurator

Usage and purpose of these services described here.

Handlers lambda configuration

The handlers lambda required several settings from luna-configurator, whose can be separated to several groups:

  • LUNA_LAMBDA_UNIT_LOGGER - lambda logger settings

  • luna-services addresses and timeouts settings (for example, LUNA_FACES_ADDRESS and LUNA_FACES_TIMEOUTS will be used by lambda to make requests to luna-faces service)

  • ADDITIONAL_SERVICES_USAGE setting will be used to determine which luna-services can be used by the lambda (the lambda will not check connection to disabled services and will raise an error if user try to make request to such service)

Handlers lambda request usage

The request to handlers lambda has some additions compared to standalone lambda request.

The HandlersLambdaRequest has the data property (detailed description presented at handlers incoming data)

The simple example:

lambda_main.py
 from luna_lambda_tools import HandlersLambdaRequest, logger

 async def main(request: HandlersLambdaRequest) -> dict:
     logger.info(request.data.sources)    # log count of images in request
     ...

Handlers lambda incoming data

The handlers lambda must be designed to process requests from Luna-Handlers service.

The incoming request structure is presented here

luna_lambda_tools.public.handlers.schemas

Module contains pydantic schemas for handlers lambda

The data property usage assumes that request will be validated using presented schema and content type of this request is application/msgpack.

If request’s body content type will differs or incoming data not pass validation, it will raise an exception and reply with 400 status code will be returned.

It will be available using data property of incoming request as follows:

lambda_main.py
 from luna_lambda_tools import HandlersLambdaRequest, logger


 async def main(request: HandlersLambdaRequest) -> dict:
     logger.info("request images count: ", len(request.data.sources))
     for i, image in enumerate(request.data.sources):
         logger.info(f"face bounding box for image number {i+1}: {image.faceBoundingBoxes[0] if image.faceBoundingBoxes else 'not presented'}")
     ...

Handlers lambda examples

The handlers lambda can use luna-services for different operations.

  • The exception whose caused by requests to luna service processing can be regulated using raiseError flag (True by default). The examples demonstrate raiseError flag usage:

    • if request to luna service causes an error, the exception will be raised and the example will returns response with 500 status code and json like this:

      lambda_main.py
      from uuid import uuid4
      
      from luna_lambda_tools import HandlersLambdaRequest
      
      
      async def main(request: HandlersLambdaRequest) -> dict:
          nonExistFaceId = str(uuid4())
          reply = await request.clients.faces.getFace(faceId=nonExistFaceId, raiseError=True)
          return reply.json
      
      request example
      from luna3.luna_lambda.luna_lambda import LambdaApi
      
      SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
      SERVER_API_VERSION = 1
      lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
      lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
      
      
      def makeRequest():
          data = {
              "aggregate_attributes": 0,
              "sources": [
                  {"body": "", "filename": "empty.jpeg", "source_type": "raw_image"},
              ],
          }
          reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId, body=data)
          return reply
      
      
      if __name__ == "__main__":
          response = makeRequest()
          print(response.json)
      
      {
          "error_code": 42004,
          "desc": "Lambda exception",
          "detail": "Not expected code from service: 404 GET http://127.0.0.1:5030/3/faces/048e1428-c7de-426b-82f8-3971bf00284c, error: {\"error_code\":22002,\"desc\":\"Object not found\",\"detail\":\"Face with id '048e1428-c7de-426b-82f8-3971bf00284c' not found\",\"link\":\"https:\\/\\/docs.visionlabs.ai\\/info\\/luna\\/troubleshooting\\/errors-description\\/code-22002\"}",
          "link": "https://docs.visionlabs.ai/info/luna/troubleshooting/errors-description/code-42005"
      }
      
    • in this case, the request will raise an error and for any request, labmda will return response with 201 status code and json like this:

      lambda_main.py
      from uuid import uuid4
      
      from luna_lambda_tools import HandlersLambdaRequest
      
      
      async def main(request: HandlersLambdaRequest) -> dict:
          nonExistFaceId = str(uuid4())
          reply = await request.clients.faces.getFace(faceId=nonExistFaceId, raiseError=False)
          return reply.json
      
      request example
      from luna3.common.requests import RequestPayload
      from luna3.luna_lambda.luna_lambda import LambdaApi
      
      SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
      SERVER_API_VERSION = 1
      lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
      lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
      
      
      def getImage(pathToImage):
          """
          Make sure pathToImage is valid path to specified image
          """
          with open(pathToImage, "rb") as file:
              return file.read()
      
      
      def makeRequest():
          data = {
              "aggregate_attributes": 0,
              "sources": [
                  {"source": {"body": getImage("empty.jpeg")}, "filename": "empty.jpeg", "source_type": "raw_image"},
              ],
          }
          payload = RequestPayload.buildMsgpack(body=data)
          reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId, body=payload)
          return reply
      
      
      if __name__ == "__main__":
          response = makeRequest()
          print(response.json)
      
      {
          "error_code": 22002,
          "desc": "Object not found",
          "detail": "Face with id 'fece7f31-ea14-40fb-8c57-40230aa17256' not found",
          "link": "https://docs.visionlabs.ai/info/luna/troubleshooting/errors-description/code-22002"
      }
      
  • The example lambda which estimates emotions on each specified image and return received results as reply

    (such lambda cannot be used as canonical luna-platform handler)

    lambda_main.py
    from luna3.public.common import BinaryImage
    from luna_lambda_tools import HandlersLambdaRequest
    
    
    async def main(request: HandlersLambdaRequest) -> dict:
        results = []
        for image in request.data.sources:
            # If you need to determine type of image use https://docs.python.org/3/library/imghdr.html or another library.
            mimetype = "image/jpeg"
            result = (
                await request.clients.sdk.sdk(
                    inputData=BinaryImage(
                        filename=image.filename or "raw_image", body=image.source.body, mimetype=mimetype
                    ),
                    detectFace=1,
                    estimateEmotions=1,
                )
            ).json
            results.append({"image_filename": image.filename, "estimations": result})
        return {"result": results}
    
    request example
    from luna3.common.requests import RequestPayload
    from luna3.luna_lambda.luna_lambda import LambdaApi
    
    SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
    SERVER_API_VERSION = 1
    lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
    lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
    
    
    def getImage(pathToImage):
        """
        Make sure pathToImage is valid path to specified image
        """
        with open(pathToImage, "rb") as file:
            return file.read()
    
    
    def makeRequest():
        data = {
            "aggregate_attributes": 0,
            "sources": [
                {"source": {"body": getImage("empty.jpeg")}, "filename": "empty.jpeg", "source_type": "raw_image"},
            ],
        }
        payload = RequestPayload.buildMsgpack(body=data)
        reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId, body=payload)
        return reply
    
    
    if __name__ == "__main__":
        response = makeRequest()
        print(response.json)
    
  • The lambda example which extracts face descriptors from two images, match them and return result as reply

    (such lambda cannot be used as canonical luna-platform handler)

    lambda_main.py
    from uuid import uuid4
    
    from luna3.public.common import BinaryImage
    from luna3.public.matcher import SDKDescriptorReference
    from luna_lambda_tools import HandlersLambdaRequest, UserException
    
    
    class ImageCountException(UserException):
        statusCode = 400
        errorText = "expected two images in request"
    
    
    class FaceCountException(UserException):
        statusCode = 400
        errorText = "excepted one face on each image"
    
    
    async def main(request: HandlersLambdaRequest) -> dict:
        if len(request.data.sources) != 2:
            raise ImageCountException
    
        images = []
        for i in range(2):
            img = request.data.sources[i]
            # If you need to determine type of image use https://docs.python.org/3/library/imghdr.html or another library.
            mimetype = "image/jpeg"
            images.append(BinaryImage(filename=img.filename, body=img.source.body, mimetype=mimetype))
        imagesEstimations = (await request.clients.sdk.sdk(images, estimateFaceDescriptor=1)).json["images_estimations"]
    
        descriptors = []
        for imageEstimations in imagesEstimations:
            estimations = imageEstimations["estimations"]
            if len(estimations) != 1:
                raise FaceCountException
            descriptors.append(estimations[0]["face"]["detection"]["attributes"]["descriptor"]["sdk_descriptor"])
    
        similarity = (
            await request.clients.matcher.matchRaw(
                candidates=[SDKDescriptorReference(descriptor=descriptors[0], referenceId=str(uuid4()))],
                references=[SDKDescriptorReference(descriptor=descriptors[1], referenceId=str(uuid4()))],
            )
        ).json["matches"][0]["matches"][0]["similarity"]
    
        return {"similarity": similarity}
    
    request example
    from luna3.common.requests import RequestPayload
    from luna3.luna_lambda.luna_lambda import LambdaApi
    
    SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
    SERVER_API_VERSION = 1
    lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
    lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
    
    
    def getImage(pathToImage):
        """
        Make sure pathToImage is valid path to specified image
        """
        with open(pathToImage, "rb") as file:
            return file.read()
    
    
    def makeRequest():
        data = {
            "aggregate_attributes": 0,
            "sources": [
                {"source": {"body": getImage("empty.jpeg")}, "filename": "empty.jpeg", "source_type": "raw_image"},
                {"source": {"body": getImage("empty.jpeg")}, "filename": "empty2.jpeg", "source_type": "raw_image"},
            ],
        }
        payload = RequestPayload.buildMsgpack(body=data)
        reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId, body=payload)
        return reply
    
    
    if __name__ == "__main__":
        response = makeRequest()
        print(response.json)
    
  • The lambda example which extracts face descriptor, basic attributes and saves face descriptor, sample and event with estimated data

    (such lambda can be used as canonical luna-platform handler)

    lambda_main.py
    import asyncio
    from datetime import datetime
    from uuid import uuid4
    
    from cow.errors.errors import Error
    from dateutil.tz import tz
    from luna3.public.common import BinaryImage
    from luna3.public.events import (
        BasicAttributes,
        BasicEthnicities,
        Descriptor,
        DetectionSamples,
        Event,
        EventDetection,
        EventFace,
        EventFaceAttributes,
        FaceSample,
    )
    from luna_lambda_tools import HandlersLambdaRequest, UserException
    from vlutils.descriptors.containers import sdkDescriptorDecode
    
    
    class ImageCountException(UserException):
        statusCode = 400
        errorText = "expected at least one image in request"
    
    
    class FaceCountException(UserException):
        statusCode = 400
        errorText = "excepted at least one face on each image"
    
    
    class DetectionException(UserException):
        statusCode = 400
        errorText = "failed to detect at least one face"
    
    
    async def saveData(request: HandlersLambdaRequest, image, handlerId: str, createTime: str) -> tuple[dict, dict]:
        # If you need to determine type of image use https://docs.python.org/3/library/imghdr.html or another library.
        mimetype = "image/jpeg"
        binaryImage = BinaryImage(filename=image.filename or "raw_image", body=image.source.body, mimetype=mimetype)
        imagesEstimations = (
            await request.clients.sdk.sdk(
                binaryImage, estimateFaceDescriptor=1, estimateBasicAttributes=1, estimateFaceWarp=1
            )
        ).json["images_estimations"]
        if len(imagesEstimations) < 1:
            raise FaceCountException
    
        imageEstimation = imagesEstimations[0]
        imageName = imageEstimation["filename"]
        if len(imageEstimation["estimations"]) < 1:
            raise DetectionException
        sdkDescriptor = imageEstimation["estimations"][0]["face"]["detection"]["attributes"]["descriptor"]["sdk_descriptor"]
        basicAttributes = imageEstimation["estimations"][0]["face"]["detection"]["attributes"]["basic_attributes"]
        faceSample = imageEstimation["estimations"][0]["face"]["detection"]["warp"]
    
        descriptorVersion, descriptor = sdkDescriptorDecode(sdkDescriptor)
    
        sampleId = (await request.clients.faceSamplesStore.postImage(imageInBytes=faceSample, raiseError=True)).json[
            "image_id"
        ]
    
        faceId = (
            await request.clients.faces.createFace(
                descriptors=[sdkDescriptor],
                externalId=imageName,
                descriptorSamples=[sampleId],
                raiseError=True,
            )
        ).json["face_id"]
        eventId = str(uuid4())
        event = Event(
            eventId=eventId,
            createTime=createTime,
            handlerId=handlerId,
            externalId=imageName,
            face=EventFace(faceId=faceId, lists=[]),
            faceAttributes=EventFaceAttributes(
                descriptorData=Descriptor(descriptor=descriptor, descriptorVersion=descriptorVersion),
                basicAttributes=BasicAttributes(
                    age=basicAttributes["age"],
                    gender=basicAttributes["gender"],
                    ethnicities=BasicEthnicities(
                        predominantEthnicity=basicAttributes["ethnicities"]["predominant_ethnicity"]
                    ),
                ),
            ),
            detections=[
                EventDetection(
                    filename=imageName,
                    detectTime=createTime,
                    samples=DetectionSamples(face=FaceSample(sampleId=sampleId)),
                )
            ],
        )
        await request.clients.events.saveEvents([event], waitEventsSaving=True, raiseError=True)
        await request.clients.sender.publish(
            events=[{"value": f"Hello, I'm new event with id {eventId}!"}],
            handlerId="00000000-0000-4000-a000-000003491877",
            eventCreateTime=str(datetime.now()),
            eventEndTime=str(datetime.now()),
        )
        imageResult = {"filename": binaryImage.filename, "status": 1, "error": Error.Success.asDict()}
        eventResult = {
            "face_attributes": {
                "attribute_id": None,
                "url": None,
                "basic_attributes": basicAttributes,
                "samples": [sampleId],
            },
            "body_attributes": None,
            "aggregate_estimations": {
                "face": None,
                "body": None,
            },
            "source": None,
            "tags": [],
            "external_id": "",
            "user_data": "",
            "face": {
                "face_id": faceId,
                "url": f"{request.clients.faces.getAddress()}/faces/{faceId}",
            },
            "event_id": eventId,
            "url": f"{request.clients.events.getAddress()}/events/{eventId}",
            "matches": None,
            "location": {},
            "detections": [
                {
                    "filename": binaryImage.filename,
                    "samples": {
                        "face": {
                            "sample_id": sampleId,
                            "url": f"{request.clients.faceSamplesStore.getAddress()}/images/{sampleId}",
                            "detection": {},
                        },
                        "body": None,
                    },
                    "detect_time": image.detectTime or createTime,
                    "detect_ts": image.detectTs,
                    "image_origin": image.imageOrigin,
                }
            ],
            "track_id": None,
            "meta": None,
        }
        return imageResult, eventResult
    
    
    async def main(request: HandlersLambdaRequest) -> dict:
        if not len(request.data.sources):
            raise ImageCountException
    
        handlerId = str(uuid4())
        createTime = datetime.now(tz=tz.tzlocal()).isoformat()
        kwargs = {"request": request, "handlerId": handlerId, "createTime": createTime}
    
        results = await asyncio.gather(*[saveData(image=image, **kwargs) for image in request.data.sources])
        resultEvents, resultImages = [], []
        for imageResult, eventResult in results:
            resultEvents.append(eventResult)
            resultImages.append(imageResult)
    
        return {"events": resultEvents, "images": resultImages, "filtered_detections": {"face_detections": []}}
    
    request example
    from luna3.common.requests import RequestPayload
    from luna3.luna_lambda.luna_lambda import LambdaApi
    
    SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
    SERVER_API_VERSION = 1
    lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
    lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
    
    
    def getImage(pathToImage):
        """
        Make sure pathToImage is valid path to specified image
        """
        with open(pathToImage, "rb") as file:
            return file.read()
    
    
    def makeRequest():
        data = {
            "aggregate_attributes": 0,
            "sources": [
                {"source": {"body": getImage("empty.jpeg")}, "filename": "empty.jpeg", "source_type": "raw_image"},
            ],
        }
        payload = RequestPayload.buildMsgpack(body=data)
        reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId, body=payload)
        return reply
    
    
    if __name__ == "__main__":
        response = makeRequest()
        print(response.json)
    
  • The lambda example which gets face detection and estimate mask directly using the LUNA SDK (such user lambda cannot be used as canonical luna-platform handler) (available at https://github.com/VisionLabs/lunasdk)

    LUNA python SDK must be added it to requirements.txt (requirements description available here)

    requirements.txt
    https://github.com/VisionLabs/lunasdk/archive/refs/tags/v.1.9.2.tar.gz
    

    Note

    To develop lambda with lunasdk locally it needs LUNA FSDK python bindings to be installed previously.

    Warning

    Such user lambda requires LUNA SDK data available from user lambda (the fsdk/data folder near the lambda_main.py main file) The LUNA SDK is available on VL release portal which must be extracted to fsdk folder (see archive file structure below). The only thing which is needed is data folder with used fsdk plans and config files (faceengine.conf and runtime.conf). It is recommended to not include not using plans from folder in archive to decrease result lambda image size.

    Archive file structure with files required for the example
      ├──lambda_main.py
      ├──requirements.txt
      └──fsdk
         └──data
             ├──faceengine.conf
             ├──runtime.conf
             ├──FaceDet_v3_a5_cpu-avx2.plan
             ├──FaceDet_v3_redetect_v3_cpu-avx2.plan
             ├──LNet_precise_v2_cpu-avx2.plan
             ├──mask_clf_v3_cpu-avx2.plan
             └──slnet_v5_cpu-avx2.plan
    
    lambda_main.py
    from luna_lambda_tools import HandlersLambdaRequest, UserException
    from lunavl.sdk.faceengine.engine import VLFaceEngine
    from lunavl.sdk.faceengine.setting_provider import DetectorType
    from lunavl.sdk.image_utils.image import VLImage
    
    
    class ImageCountException(UserException):
        statusCode = 400
        errorText = "expected one image in request"
    
    
    class FaceDetectionException(UserException):
        statusCode = 400
        errorText = "failed to get face detection from image"
    
    
    async def main(request: HandlersLambdaRequest) -> dict:
        if len(request.data.sources) != 1:
            raise ImageCountException
    
        image = VLImage(body=request.data.sources[0].source.body)
        faceEngine = VLFaceEngine()
        detector = faceEngine.createFaceDetector(DetectorType.FACE_DET_V3)
        detections = detector.detect([image])
        if not detections:
            raise FaceDetectionException
        faceDetections = detections[0]
        warper = faceEngine.createFaceWarper()
        warps = [warper.warp(faceDetection) for faceDetection in faceDetections]
    
        maskEstimator = faceEngine.createMaskEstimator()
        mask = await maskEstimator.estimate(warps[0].warpedImage, asyncEstimate=True)
    
        return {"results": mask.asDict()}
    
    request example
    from luna3.common.requests import RequestPayload
    from luna3.luna_lambda.luna_lambda import LambdaApi
    
    SERVER_ORIGIN = "http://lambda_address:lambda_port"  # Replace by your values before start
    SERVER_API_VERSION = 1
    lambdaApi = LambdaApi(origin=SERVER_ORIGIN, api=SERVER_API_VERSION)
    lambdaId, accountId = "your_lambda_id", "your_account_id"  # Replace by your values before start
    
    
    def getImage(pathToImage):
        """
        Make sure pathToImage is valid path to specified image
        """
        with open(pathToImage, "rb") as file:
            return file.read()
    
    
    def makeRequest():
        data = {
            "aggregate_attributes": 0,
            "sources": [
                {"source": {"body": getImage("empty.jpeg")}, "filename": "empty.jpeg", "source_type": "raw_image"},
            ],
        }
        payload = RequestPayload.buildMsgpack(body=data)
        reply = lambdaApi.proxyLambdaPost(lambdaId=lambdaId, path="main", accountId=accountId, body=payload)
        return reply
    
    
    if __name__ == "__main__":
        response = makeRequest()
        print(response.json)