Improve handling of termination signals

Previously, upon receiving a termination signal, the process tries to
flush all changes to the database, retrying every 0.1s until it
succeeds. However, if the database is not set up properly, this always
fails, and spams the terminal/logging with the error message 10 times a
second.

If the user sends another termination signal, the handleTermSignal
function is called once again, and we get twice the number of error
messages.

This commit changes the behaviour in various ways.

(1) It lowers the retry rate to 0.5s, and aborts after 30 seconds.

(2) If the write to the database errored, the error message explains
    that this is due to us flushing the final changes.

(3) We replace realtime.maintenance with realtime.state, which is an
    Enum with three possible states --- Starting, Running, and Stopping.
    If a termination signal is received in the starting state, the
    process simply aborts because there is nothing to clean up. This is
    the case when the database is misconfigured, since the application
    starts up only after connecting to the databse. If it is in the
    Stopping state, the handleTermSignal function returns because
    another instance of handleTermSignal is already running.

Fixes #408

Signed-off-by: Dexter Chua <dec41@srcf.net>
This commit is contained in:
Dexter Chua 2020-06-27 11:06:48 +08:00
parent f22a2ad15d
commit c8033f9a3a
3 changed files with 26 additions and 9 deletions

View file

@ -20,6 +20,11 @@ export type SocketWithNoteId = Socket & { noteId: string }
const chance = new Chance()
export enum State {
Starting,
Running,
Stopping
}
/* eslint-disable @typescript-eslint/no-use-before-define */
const realtime: {
onAuthorizeSuccess: (data, accept) => void;
@ -27,7 +32,7 @@ const realtime: {
io: SocketIO.Server; isReady: () => boolean;
connection: (socket: SocketWithNoteId) => void;
secure: (socket: SocketIO.Socket, next: (err?: Error) => void) => void;
getStatus: (callback) => void; maintenance: boolean;
getStatus: (callback) => void; state: State;
} = {
io: SocketIO(),
onAuthorizeSuccess: onAuthorizeSuccess,
@ -36,7 +41,7 @@ const realtime: {
connection: connection,
getStatus: getStatus,
isReady: isReady,
maintenance: true
state: State.Starting
}
/* eslint-enable @typescript-eslint/no-use-before-define */
@ -751,7 +756,7 @@ function updateUserData (socket: Socket, user): void {
}
function connection (socket: SocketWithNoteId): void {
if (realtime.maintenance) return
if (realtime.state !== State.Running) return
parseNoteIdFromSocket(socket, function (err, noteId) {
if (err) {
return failConnection(500, err, socket)