Task processing

To schedule index building LIM Manager performs two types of background jobs:

  1. planning routine to schedule background indexing tasks (according to the app config)

  2. lookup routine to submit indices for build & check building process

Note

The multi-instance mode of LIM Managers is supported with master auto selection based on redlock. The background planning & indexer lookup is the sole responsibility of the master. Other managers can accept user requests for one-time index building, as well as provide some information about tasks & index storage.

Planning routine

By default, LIM Manager builds indexes at startup and creates a further indexing schedule based on the app config.

There are two ways to schedule indexing: you can either specify the LIM_MANAGER_INDEXING.PLANNING_PERIOD property (which is the default option) in the app config file. The PLANNING_PERIOD is specified in seconds and defines the period between indexing. Alternatively, you may use LIM_MANAGER_INDEXING.PLANNING_SCHEDULE. This parameter is a cron-like string (no letter symbols support at the moment), you can refer to the helper for assistance.

There is no preferred method to schedule indexing, but if both PLANNING_SCHEDULE and PLANNING_PERIOD are specified, LIM Manager prioritizes the use of PLANNING_SCHEDULE.

In both cases, LIM Manager considers existing indexes. Therefore, if there are two app instances running or if the app work was interrupted for some reason, it won’t rebuild a fresh index whose lifespan has not yet expired. It will also consider the last indexing time (but not app startup time) as the base for planning further indexing calculations.

Lookup routine

The indexer lookup routine is carried out with the period specified by LIM_MANAGER_INDEXING.LOOKUP_PERIOD setting from app config. At this stage, the LIM Manager checks the status of all indexers. If some LIM Indexer has finished index building, the LIM Manager updates task information and sends data for monitoring. If some LIM Indexer is ready to accept the task, the LIM Manager pops next task from the pending queue, submits the task to the indexer and updates task information.