Interaction of LP Services#
There are separate databases specified for each service in the images below. This is done for illustration purposes.
It is not required to launch several similar databases for each service. You can launch a single database instance (e. g., PostgreSQL) and use it to store data for all the LP services. Each service will have its own table in this database.
See "General concepts" for details about the databases used by LP services.
General services#
API service provides a RESTful interface for faces and bodies detection, extraction and matching procedures.
Requests are sent to LP via HTTP. A common case is when an external service sends requests to LP, receives results and processes the results according to business needs.
You can find all the information about general requests to the API service in the "APIReferenceManual.html" document. The name of the section describing the corresponding request is given in brackets.
API receives requests, processes them and sends them to other services:
- The requests for face detection, face parameters estimation and samples creation (Samples > Detect faces) are sent to the Handler (A1);
- The requests for temporary attributes creation (Attributes > Extract attributes) are sent to the Handler service (A1);
- The requests for descriptors matching (Matcher > Matching) can be sent to the Python Matcher service (A6).
API sends requests to Faces for new faces creation, as well as lists creation and managing (Faces section) (A2).
The API service receives information about the current license terms from the Licenses service (A3) using middleware.
The API service sends requests to the Liveness V1 service (Liveness > Predict liveness) (A4), when Liveness V1 is utilized. Liveness V2 is a part of the Handlers service.
The API service sends samples received from external services to the Image Store service (Samples > Save sample) (A5).
Handlers processes face detection and attributes creation requests. It estimates face attributes and face parameters.
Handler processes detection request from API (A1) and sends resulting samples to the Image Store service (H1).
Handler processes attributes creation request from API (H1), requests existing samples from the Image Store service (H1) and sends the created temporary attributes to the Faces service that stores them in the Redis database (F1).
The Handlers service creates new handlers (Handlers > Create handler) and stores them in the Handlers database (H3).
The request for a new event creation (Events > Generate events) is received by API, sent to Handlers (H1) and processed according to the Handler, which ID is specified in the request.
Image Store receives samples from the Handlers service and stores them on SSD or in the S3 database and provides access to them (Im1).
Licenses. The Licenses service receives information about the number of created faces with attributes from the Faces database (Li1).
Liveness V1. Requests to the Liveness V1 service are sent by the API service (A4). The Liveness V1 service processes images and returns liveness probability and other data.
*\ Note that Liveness V2 is a part of the Handlers service.
Faces is responsible for interaction with Faces database, which stores: faces, descriptors for faces, and lists (F2). It provides access to the stored data for API, Python Matcher, and Tasks services.
Faces also stores the created temporary attributes in the Redis database (F1).
Events service stores and provides information about events (the Events section). After an event is created, the Events service receives the created event from the Handlers service (H4) and stores it in the Events database (Ev2).
The Events service usage can be enabled/disabled in the API service configuration file.
Python Matcher can receive matching requests directly from the API service (A6). Python Matcher sends matching requests to the Faces database (M3) or Events database (M1). The databases matches descriptors and sends results back to the API service.
Python Matcher can also receive the temporary attributes required for matching from the Redis database (M2).
Python Matcher Proxy receives requests from API (A9) and routes requests to Python Matcher (M4), Events DB (M5), Redis DB (M7), Faces DB (M8), Index Manager (Ix1), Indexer and Indexed Matcher.
Python Matcher Proxy service can also route requests to matching plugins, which can significantly speed up the matching requests execution. This interaction is not shown in the diagram. See "Matching plugins" section for details.
Requests to Indexer and Indexed Matcher are sent by means of MQ (Ix9).
Index service allows to index descriptors to provide better matching speed (the Index services section).
Index creation
A request is sent to the API. API sends the request to the Matcher Proxy (A6). Matcher Proxy directs the request to the Index Manager (the request contains the list(-s) to be indexed) (Ix1). Index Manager poses an indexing task to the Message Queue (Ix2). Indexer takes the task from the Message Queue (Ix3). Indexer sends a request to the Faces service to get the face descriptors from the list(-s) (Ix4). Indexer creates the index and saves it in the File Storage (Ix5). Matcher Daemon checks the File Storage and retrieves the created indexes from it (Ix6). Matcher Daemon transmits the index to the Indexed Matcher (Ix7). Index Manager controls the list creation process by checking Matcher Daemon (Ix8).
Matching process
A request is sent to the API. API sends a request to the Matcher Proxy (A6). Matcher Proxy adds a matching task in the Message Queue (Ix9). Indexed Matcher checks the Message Queue and retrieves the task (Ix10). Indexed Matcher performs the matching and sends the results back to API (Ix10, Ix9, A6)
Delta acquisition
Indexed Matcher periodically calls the Faces service to check whether new descriptors have appeared and retrieves them (Ix11). New descriptors are those not included in the index.
General and optional services#
Admin. A user can manage accounts and perform other actions via the user interface of the Admin service (Adm1). The corresponding requests are sent to the Admin service (Adm2).
You can find all the information about requests in the "../ReferenceManuals/AdminReferenceManual.html" document.
All information about user accounts is stored in the Admin database (Adm3).
The Admin service sends requests to Tasks service (Adm4). These tasks are performed for LP databases in general, not for a single account ID.
The Admin service receives information about the current license terms from the Licenses service (Adm5).
The Admin service checks the current number of created faces with linked attributes in the Faces database (Adm6) and calculates the percentage of the database of fullness.
The Admin service also makes requests to all LP services, receiving system information (see "/luna_sys_info" request). This interaction is not shown in the diagram.
Sender. All generated events are received by the Redis DB from the Handlers service (H6). External third-party party software subscribes to receive events according to the specified filters (Se2). Sender service checks Redis channel, receives the required information and sends it to external software (Se1).
Sender service usage is enabled/disabled in the API service configuration file.
Configurator. LP services receive configuration parameters from the Configurator service (Con1). A user can manage configurations in Configurator by means of the graphical user interface (Con2). The changes are sent to Configurator (Con3). All the changes in configuration files are saved in Configurator DB (Con4).
Configurator is utilized for each Luna service by default.
Tasks. Tasks service receives requests from API service (A8). Then Tasks creates and stores a task in the Tasks database (T1). Then Tasks sends subtasks to its workers (T2), which also have access to the Tasks (T3) database.
Tasks service receives data from Faces service for clustering, linker, and additional extraction tasks (T4).
Tasks service receives data from Events service for clustering and linker tasks (T9, T0).
Tasks service usage is enabled/disabled in the API service configuration file.
The interaction of Tasks workers with other services depends on the task type.
Tasks workers receive data from Faces service for every processed task (T5).
Tasks workers send a request to Python Matcher (T7) for clustering, cross-matching and ROC creation tasks.
Tasks workers store all the reports in the storage of Image Store (T8). Tasks workers also store and receive clusters from Image Store.
Tasks workers send the request for additional extraction task to the Handler service (T6).
Monitoring. Monitoring system receives requests and errors from each service (Mon1). This data stores in the Influx database (Mon2). Then the monitoring data is sent to Grafana to visualize the monitoring data (Mon3). You can find more information about monitoring in the "Monitoring" section.