Closed Bug 1441181 (bmo-cloud-jobqueue) Opened 6 years ago Closed 6 years ago

Implement new process model for running multiple email jobqueue daemons

Categories

(bugzilla.mozilla.org :: General, enhancement)

Production
enhancement
Not set
normal

Tracking

()

RESOLVED FIXED

People

(Reporter: glob, Assigned: dylan)

References

()

Details

Attachments

(9 files, 2 obsolete files)

45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
45 bytes, text/x-github-pull-request
dylan
: review+
Details | Review
Implement new process model for running multiple email jobqueue daemons, as per ckolos's recommendations.
This PR removes the subprocess code that was added because of memory leaks.
It's intended to be replaced at the end of the patch series with Process::SizeLimit::Core.
The header in the jobqueue code is weird; this patch makes it more like more recent changes.
While working on this I noticed memory wasn't being freed in a predictable way. This is because the memory is cleared before each job, instead of after.
This re-implements the sub process code in terms of IO::Async. It doesn't yet handle memory problems but I'm going to add that in my next period of coding.
Attached file Step 5 - Add logging (obsolete) —
This is not strictly needed, but it wires up the TheSchwartz logging code into our logging framework.
Attached file Step 6 - Add concurrency (obsolete) —
Attachment #8955706 - Attachment description: Bug 1441181 - Step 6 - Add concurrency → Step 6 - Add concurrency
Attached file Step 5 - Add logging
Attachment #8955706 - Attachment is obsolete: true
Attachment #8958290 - Attachment description: PR → Step 8 - Add jobqueue command to entrypoint
Attachment #8955591 - Flags: review+
Attachment #8955593 - Flags: review+
Attachment #8955594 - Flags: review+
Attachment #8955707 - Flags: review+
Attachment #8955602 - Attachment is obsolete: true
Attachment #8955706 - Attachment is obsolete: false
Attachment #8957333 - Flags: review+
Attachment #8955601 - Flags: review+
Attachment #8955706 - Attachment is obsolete: true
Attachment #8958514 - Flags: review+
Attachment #8958290 - Flags: review+
Attachment #8958513 - Flags: review+
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Attachment #8958514 - Attachment description: Bug 1441181 - Step 6 - Add concurrency → Step 6 - Add concurrency
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: