Add supervised systemd queue worker (phase 8)

Runs php artisan queue:work as a systemd --user service (Restart=always,
hourly recycling) so OSINT lookups process automatically instead of
requiring a manually-run queue:work. All 8 planned phases now done.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
cedry2k1992
2026-07-20 10:40:37 +03:00
co-authored by Claude Sonnet 5
parent 873699e23e
commit 853686f3f3
3 changed files with 70 additions and 7 deletions
+18 -7
View File
@@ -145,9 +145,21 @@ commands).
and by target value, opened the printable report, downloaded and
inspected both CSV and JSON exports. All test data cleaned up
afterward.
8. **Queue worker process management** — `QUEUE_CONNECTION=database`
needs a supervised `queue:work` process (supervisor/systemd) before
this is usable outside manual dev testing (see Environment notes)
8. **Queue worker process management** (done) — `deploy/systemd/
osint-queue-worker.service`, a `systemd --user` unit (no sudo
needed; this box already has lingering enabled so it survives
logout) running `queue:work --timeout=180 --max-time=3600
--tries=1` with `Restart=always`. `--timeout` is set above
`config('osint.run_timeout')` (120s) — the worker's timeout must
exceed a runner's own timeout or the worker kills the job first.
`--max-time` restarts the worker hourly to pick up code/`.env`
changes and avoid long-lived-process memory growth. Installed and
verified live: dispatched a real job via the web app *without*
manually running `queue:work`, confirmed the persistent service
picked it up and completed it (ip-api lookup, ~200ms); killed the
worker process directly and confirmed systemd auto-restarted it
within seconds. See `deploy/systemd/README.md` for install/operate
commands.
## Open questions
- Any legal/ToS constraints on the sources to be queried?
@@ -164,10 +176,9 @@ commands).
`--env=testing` directly against migrate/seed
- No sudo access in the dev sandbox that created this project; SQLite
extension isn't installed for PHP 8.4 here, so MySQL was used instead
- `QUEUE_CONNECTION=database`, so lookups won't actually run unless a
queue worker is running (`php artisan queue:work`). No supervisor/
systemd unit set up for this yet — needed before this is usable
outside of manual dev testing
- `QUEUE_CONNECTION=database`; the queue worker is supervised by
`systemd --user` (see `deploy/systemd/`) and running continuously —
no more need to manually run `php artisan queue:work` in dev
- The two Python venvs are ~390MB combined (`.venv` 166MB,
`.venv-theharvester` 223MB, mostly playwright/censys/shodan/fastapi
pulled in by theHarvester) — worth knowing before assuming this is