Events saving

How the service save events

The following section describes events saving.

When you send the request with method POST on the /events resource, the Events service validates each event and saves it to the queue. Next, it responds to the client indicating either events was rejected due to some some reason, or successfully put in the queue (but not saved yet). After that, in a background process, the service takes COPY_BATCH_SIZE number of events from the queue and tries to save them. During this process two types of errors may arise: recoverable (timeout, connectionerror) and unrecoverable (integrity, out of memory, out of disk space, auth error, etc). In case of recoverable error, the service will put failed batch back to the queue and wait for the database to become healthy, before trying to save them again. In case of unrecoverable error, whole batch will be dropped. For both cases, appropriate messages are written to the log.

Caveats

Athough this feature allows to survive temporary database unavailability, it’s not designed for it. The main intention is to provide better performance while sacrificing durability.

The queue for events objects is not durable and has limits on its size, see option COPY_MAX_QUEUE_SIZE. When the queue size exceeds this amount, new requests for creating events will be rejected with appropriate error response.