Orientation
Orbital-Express is an opinionated Express.js + Sequelize (PostgreSQL) framework for building really good backend APIs — fast to develop on, consistent across a team, and easy to scale. This documentation takes you from zero to building features confidently — read it top to bottom and each section will build on the last.
Orbital-Express is the architecture/framework. Nitra Brain is the product this repo is building on top of Orbital-Express. Everything in these docs describes the Orbital-Express way of building APIs — you'll apply it to ship Nitra Brain's features.
What this is (and the philosophy behind it)
The framework is a hybrid of two battle-tested ideas: Django's feature-based organization and Ruby on Rails' Model-View-Controller flow. Instead of splitting code by type (all models in one folder, all controllers in another), we group everything for a single feature — its model, routes, controller, actions, background tasks, tests, translations, and emails — into one feature folder under app/.
The payoff: when you work on “Orders,” everything about Orders is in one place. No jumping across a dozen top-level folders. As the app grows to hundreds of tables, this is the difference between a codebase you can navigate and one you fight.
A feature ≈ a database table. New table → new feature folder. The folder holds the table's model plus every way the app reads and writes it.
Who this is for
Backend engineers comfortable with JavaScript/ES6, Node.js, and Express. You don't need to know Sequelize, Bull, or Socket.IO going in — we cover how we use them. You do need the basics of how an HTTP API works.
How to read this doc
It's designed to be read sequentially. Concepts are introduced in dependency order, so you never hit a term that hasn't been explained. When something is genuinely needed later, we'll say so explicitly and tell you what to hold in your head for now.
- Getting Started — the philosophy, the architecture at a glance, and getting it running locally.
- Core Concepts — the feature folder, the request lifecycle, and how to write actions and the data layer.
- Advanced — background jobs, real-time sockets, authentication, and testing.
- Doing the Work — a hands-on tutorial, and how to drive all of this with AI agents.
- Reference — endpoint reference, the conventions rulebook, operations, and a glossary.
Skim the Big Picture diagram next so the rest of the doc has a place to land — then read straight through.
The shape of a feature folder
You'll spend most of your time here. We go deep in §4; for now, just absorb the shape:
app/Order/
├── actions/ # real-time API handlers (V1Create, V1Update, …)
├── tasks/ # background jobs (V1ExportTask, …)
├── tests/ # integration + task + unit tests
├── languages/ # i18n strings (en.js, …)
├── mailers/ # email templates
├── controller.js # thin router: request → correct action
├── model.js # Sequelize table definition
├── routes.js # feature routes
├── error.js # feature error codes
├── helper.js # feature-local helpers
└── worker.js # registers which task handles which jobThe Big Picture
An Orbital-Express app runs as three cooperating processes backed by PostgreSQL and Redis. Everything else in this doc is a detail of one of these boxes.
flowchart LR
C["Clients
web and mobile"] -->|HTTP| W["Web server
clustered per CPU"]
C -.->|WebSocket| W
W -->|queries| PG[("PostgreSQL")]
W -->|enqueue jobs| RD[("Redis")]
WK["Worker"] -->|process jobs| RD
WK -->|queries| PG
CR["Cron clock"] -->|enqueue on schedule| RD
classDef proc fill:#c15f3c,stroke:#a84e2f,color:#fff;
class W,WK,CR proc;
The three processes — web (yarn s), worker (yarn w), cron (yarn cron) — share Postgres & Redis.
The three processes
An Orbital-Express deployment isn't one server — it's three, each with a single job. They share the same codebase and the same Postgres + Redis, but run independently.
| Process | File | Run | Responsibility |
|---|---|---|---|
| Web | index.js → server.js | yarn s | Handles HTTP/WebSocket requests. Clustered — one instance per CPU core (via throng). |
| Worker | worker.js | yarn w | Pulls background jobs off Redis queues and runs the matching task. |
| Cron | cronjobs.js | yarn cron | The clock. Enqueues jobs on a schedule. Exactly one instance runs in production. |
Two data stores
PostgreSQL is the source of truth — all your tables live here, accessed through Sequelize (the ORM). Redis is the backbone for background jobs (the Bull queue) and the Socket.IO adapter that lets real-time events fan out across all the clustered web instances.
The two ways work happens
Every piece of work is either a real-time request or a background job — and knowing which is half of designing a feature:
Action (real-time): a client makes an HTTP request, you do the work, you return a response immediately. The default for almost everything.
Task (background job): work that's slow (exporting 1M rows), scheduled (nightly cleanup), or shouldn't block the response. The action enqueues a job and returns right away; the worker does the heavy lifting; the user is notified later (email / push / socket).
You'll see exactly how to write each in §6 (actions) and §9 (tasks). For now just hold the split in your head.
The repo at a glance
Almost everything you build lives in app/ (one folder per feature). The rest of the repo root is the global layer — shared infrastructure, cross-feature code, and the three process entry points. Here's the whole layout, top to bottom:
repo root
├── app/ # ★ ALL features live here — one folder per feature (see §4)
│
├── index.js # web entry point → boots server.js (clustered via throng)
├── server.js # builds the Express app: middleware, routes, socket, error handler
├── worker.js # worker entry point — registers every queue's task processors (§9)
├── cronjobs.js # cron entry point — schedules jobs on a clock (§9)
├── routes.js # GLOBAL route aggregator — mounts every feature's routes.js
├── models.js # GLOBAL model aggregator — scans app/*/model.js into one `models` object
│
├── middleware/ # global Express middleware (id, args, auth, error, exit) — runs on every request (§5)
├── services/ # global SERVICES — third-party wrappers + big shared infra (see below)
├── helpers/ # global HELPERS — small pure functions shared across features (see below)
│
├── database/ # everything about the DB except the per-feature models
│ ├── schema.sql # human-readable documentation of every table (never executed)
│ ├── sequence.js # the ORDER tables are created in (FK deps) — generator updates it, never by hand
│ ├── seed/ # dev seed data (set1/…) loaded by `yarn seed`
│ ├── backups/ # DB dumps from `yarn backup`; `yarn restore` reads these
│ └── index.js # the Sequelize connection
├── migrations/ # the real, ordered schema changes applied by `yarn migrate` (§7)
├── models/ # (essentially empty — models live in app/*/model.js; models.js aggregates them)
│
├── languages/ # GLOBAL i18n source strings (en.js, …) — compiled into locales/ (§8)
├── locales/ # COMPILED i18n output (generated by `yarn lang` — never edit by hand)
├── mailers/ # GLOBAL email templates (feature-specific ones live in app//mailers/)
│
├── config/ # per-environment .env files (gitignored) + config glue (§3)
├── docs/ # all the deep docs (conventions.txt, workflow.md, auth-migration.md, …)
├── scripts/ # standalone scripts you run by hand (e.g. compile-lang, password helpers)
├── redis/ # project-local Redis build (gitignored; local dev only — §3)
│
├── test/ # global test entry: stitches feature tests + holds fixtures, helper/service tests
│ ├── app/ # stitches together each feature folder's tests
│ ├── fixtures/ # test-DB baseline data (fix1/…) — the test analog of database/seed
│ ├── helpers/ # unit tests for GLOBAL helpers
│ └── services/ # unit tests for GLOBAL services
│
├── public/ views/ # static assets & server-rendered views (e.g. email preview)
└── AGENTS.md CLAUDE.md # agent guides (see §15) The global layer, folder by folder
| Path | What goes here |
|---|---|
app/ | Every feature. One singular-PascalCase folder per table (app/Order/), holding its model, routes, controller, actions, tasks, tests, i18n, mailers. This is where ~all your day-to-day work happens (§4). |
routes.js (root) | The global route aggregator. Each feature has its own routes.js; this root file mounts them all into the one Express router. Register a new feature here once. |
models.js (root) | The global model aggregator. The models/ folder is essentially empty on purpose — models live in app/<F>/model.js, and models.js scans them all into a single models object you require everywhere (models.order, models.user). |
cronjobs.js (root) | The cron clock process — defines what gets enqueued on what schedule (§9). Run with yarn cron; exactly one instance in production. |
middleware/ | Global Express middleware that runs on every request — id, args, auth, error, exit (§5). |
services/ | Services = third-party wrappers and bigger-than-a-helper shared infrastructure: queue.js (Bull), redis.js, socket.js, email.js, language.js, passport.js, postgres.js, plus vendor wrappers (google.js, outlook.js, phone.js, …). Reach for a service when something is stateful, wraps an external system, or is too substantial to be a plain helper. |
helpers/ | Global helpers = small, pure utility functions shared across multiple features (constants.js, cruqd.js, logic.js, schemas.js, validate.js). The rule: if a helper is used by only one feature, it goes in that feature's helper.js; the moment it's shared across features, promote it here. |
database/ | Everything about the DB except the per-feature models: schema.sql (documentation of every table), sequence.js (table-creation order — the generator maintains this automatically; never edit it by hand), seed/ (dev data via yarn seed), backups/ (yarn backup/restore), and the connection (index.js). |
migrations/ | The real, ordered schema changes applied to dev/prod by yarn migrate (§7). |
languages/ → locales/ | languages/ holds the global i18n source strings; yarn lang compiles them into locales/*.json (the compiled output — never hand-edit). Feature strings live in app/<F>/languages/ (§8). |
mailers/ | Global email templates; feature-specific emails live in app/<F>/mailers/ instead. |
config/ | Per-environment .env.* files (gitignored, local to each dev) plus the config glue that loads them (§3). |
docs/ | All the deep documentation — conventions.txt (authoritative rulebook), workflow.md, auth-migration.md, etc. |
scripts/ | Standalone scripts you run by hand (one-off or manual maintenance) — not part of the request/worker/cron flow. |
redis/ | The project-local Redis build (gitignored). Local dev only; production uses a managed Redis (§3). |
test/ | The global test entry point. test/app/ stitches together each feature's tests; test/fixtures/ holds test-DB baselines; test/helpers/ and test/services/ hold unit tests for the global helpers and services (§12). |
It's easy to remember tests for features and forget the global layer. Any time you add or change a global helper (helpers/*.js) or a global service (services/*.js), write/update its unit test in test/helpers/ or test/services/ — named after the file it tests. These are pure unit tests (no server, no DB, no lifecycle hooks); if you can't test a helper without booting a DB, its I/O needs extracting (§12). Treat this as part of the build routine, not an afterthought.
services/ + helpers/. Next → §3 Get It Running: get all of this running on your machine.Get It Running
Get the three processes running locally before diving into concepts — it's easier to learn a system you can poke at.
Prerequisites
- Node v22.x and Yarn.
- PostgreSQL running locally (create the dev + test databases — see below).
- Redis — project-local (see the callout).
We don't install Redis system-wide. Each project builds and runs its own copy in a gitignored redis/ folder so the version is pinned per-project (no global conflicts). Download a release, make, rename the folder to redis/, drop a vX.X.X.txt version note inside, then start it with yarn redis (stop: yarn redis:stop). Full steps in docs/redis.txt.
Local development only — in production we use a managed Redis (Heroku Redis / Redis Cloud) via the REDIS_URL config var; Redis is never built or run from the repo on a deploy.
1. Install dependencies
yarn install — this also runs postinstall (which compiles i18n via yarn lang).
When adding a dependency, always pin the exact version: yarn add <pkg> --exact (--dev for dev deps). Never ^/~ ranges — they let a fresh install silently pull a different version and cause "works on my machine" bugs.
2. Environment files
Config lives in config/, one file per environment, all gitignored. Copy config/.env.template into the files you need and fill them in:
config/.env.development
config/.env.test
config/.env.production # only if connecting to prod locally
config/.env.staging # (staging is just a second "production" app)Every config/.env.* file is gitignored. They never get committed, so each developer creates and maintains their own copy locally.
.env.development holds the variables used when you run the app locally (yarn s/yarn w/yarn cron) — your local DB, local Redis, dev API keys. This is not the test environment.
.env.test holds the variables used when you run the test suite (yarn test) — typically a separate test database that the suite wipes and rebuilds. Keeping it separate means tests never touch your dev data.
Every time you add a new variable, add it to .env.template too (that file is committed, so teammates know what to set).
The variables you must set to get the app running (the source of truth is always config/.env.template):
| Variable | What it is |
|---|---|
NODE_ENV | development / test / production — selects which .env.* loads and drives env-specific behavior. |
DATABASE_URL | Postgres connection string for this environment (a separate DB for .env.test). |
REDIS_URL | Redis connection (Bull queues + the Socket.IO adapter). Local project-Redis in dev; a managed add-on in prod. |
ACCESS_TOKEN_SECRET | Signing secret for the short-lived access JWT (§11). Distinct from the refresh secret. |
REFRESH_TOKEN_SECRET | Signing secret for refresh tokens — must differ from the access secret. |
ACCESS_TOKEN_EXPIRES_IN | Access-token lifetime, e.g. 15m. |
REFRESH_TOKEN_EXPIRES_IN | Refresh-token lifetime, e.g. 60d. |
.env.test
The four token vars (ACCESS_TOKEN_SECRET, REFRESH_TOKEN_SECRET, ACCESS_TOKEN_EXPIRES_IN, REFRESH_TOKEN_EXPIRES_IN) must be present in every env file. Miss them in .env.test and the auth tests can't mint tokens — the suite fails in confusing ways. Use the two secrets distinct, and never commit real secrets (the .env.* files are gitignored; only .env.template is committed).
3. Create the database & run migrations
# create the dev and test databases in Postgres, e.g.
createdb nitrabrain_dev
createdb nitrabrain_test
yarn migrate # run all pending migrations (dev)
yarn seed # optional: load dev seed data4. Run the three processes
yarn redis # start project-local Redis (separate terminal)
yarn s # web/API server
yarn w # background worker
yarn cron # clock / cronjobs5. Run the tests
Postgres and Redis must be running. The suite runs serially:
yarn test # full suite (compiles i18n + fixtures first)
npx jest app/Admin/tests --runInBand # a subsetThe Feature Folder
A feature folder is the unit of work. It holds a database table and every way the app reads and writes it — model, routes, controller, actions, tasks, tests, translations, emails — all in one place under app/.
Why feature-based (not type-based)
Most frameworks split code by type: all models in /models, all controllers in /controllers, etc. That means a single feature is smeared across a dozen top-level folders, and working on it is constant scrolling and jumping. Orbital-Express groups by feature instead — everything about Order is in app/Order/. Fewer merge conflicts (devs touch different folders), and the codebase stays navigable at hundreds of tables.
What's inside
app/Order/
├── actions/
│ ├── index.js # auto-managed aggregator (don't edit by hand)
│ ├── V1Create.js # one file per action
│ └── V1Query.js
├── tasks/
│ ├── index.js # auto-managed aggregator
│ └── V1ExportTask.js # one file per background task
├── tests/
│ ├── integration/ # one test per action (real HTTP)
│ ├── tasks/ # one test per task
│ └── helper.test.js # unit tests for helper.js
├── languages/en.js # i18n strings for this feature
├── mailers/ # email templates (index.ejs)
├── model.js # the Sequelize table definition
├── routes.js # this feature's routes
├── controller.js # thin router: request → correct action
├── error.js # this feature's error codes
├── helper.js # feature-local helper functions
└── worker.js # maps job names → task functions| File | Job | Covered in |
|---|---|---|
model.js | Defines the table & its behavior (Sequelize) | §7 |
routes.js | Maps URLs to controller methods | §5 |
controller.js | Thin router — picks the action by role | §5 |
actions/ | The real-time business logic | §6 |
tasks/ + worker.js | Background jobs | §9 |
error.js | Feature error codes | §8 |
languages/ | i18n strings | §8 |
tests/ | Integration / task / unit tests | §12 |
You never hand-create these files
Creating a feature by hand means a dozen files with exact structure, imports, and naming. Instead, use the generator — every file comes pre-filled correctly:
yarn gen Order # whole feature folder (singular PascalCase name)
yarn gen Order -a V1Create # add an action (+ its test, updates actions/index.js)
yarn gen Order -t V1ExportTask # add a task (+ its test, updates tasks/index.js)
yarn gen Order -m Confirmation # add a mailer
yarn del Order [-a|-t|-m ...] # the reverseNever hand-create feature/action/task files. yarn gen keeps actions/index.js, tasks/index.js, and database/sequence.js correct — hand-creation drifts from conventions and misses that wiring.
The code behind yarn gen/yarn del is in app/feature.js — that's where all the scaffolding logic (the file templates, the index/sequence wiring) is defined. If you want to see exactly what gets generated, or change a template, that's the file to read.
Feature names are always singular PascalCase: Order ✅ orders/Orders ❌.
The Request Lifecycle
Every request flows through the same chain: middleware → route → controller → action → response. Understanding this path is how everything else clicks into place.
flowchart TB
REQ["Incoming request"] --> MW
subgraph MW ["middleware runs in order"]
direction TB
M1["id: requestId"] --> M2["cors, helmet, compression"] --> M3["args: builds req.args"] --> M4["auth: sets req.user or req.admin"]
end
MW --> RT["routes.js: match URL to controller"]
RT --> CT["controller.js: pick action by role"]
CT -->|authorized| ACT["action: validate then run logic"]
CT -->|wrong role| E401["401 Unauthorized"]
ACT -->|returns object| RES["controller sends JSON response"]
ACT -.->|throws| ERRMW["error middleware: 500"]
classDef hot fill:#c15f3c,stroke:#a84e2f,color:#fff;
class ACT,CT hot;
A request passes down the middleware chain, into the feature folder (routes → controller → action), and back out as a response — or diverts to the error middleware if something throws.
1. Middleware — and why order matters
Middleware is the chain of functions every request passes through before it reaches your feature code. Each one inspects or augments the req object and calls next() to pass control down the chain. They're registered once in server.js, and the order they're registered is the order they run — which is the whole game: each middleware can rely on what the ones before it already did.
Here's the whole chain at a glance, in registration order — then we go through each one:
id— assigns a uniquerequestIdfor tracing/correlation.cors·helmet·compression— transport & security (origins, protective headers, gzip).args— builds the normalizedreq.args(POST body or GET query).auth— resolves identity ontoreq.user/req.admin(non-blocking).exit— during shutdown, answers new requests with503(graceful drain).- routes → controller → action — your feature code runs here.
error— the last-registered safety net; catches anything that throws →500.
Now each one, in order:
middleware/id.js — runs first
Generates a unique requestId for the request, attaches it to req, and sets it as the X-Request-ID response header. Why first? Because every later stage — logging, the auth layer, and especially the error middleware — wants to reference one stable id for this request. When something blows up in production, that id is how you find this exact request in the logs (it's returned in the body on 500s). Putting it first means everything downstream, including failures, can be correlated to it.
cors · helmet · compression — transport & security
Standard hardening and transport concerns: cors controls which origins may call the API, helmet sets protective HTTP headers, compression gzips responses. They run early because they apply to every response regardless of route, and there's no point doing real work for a request a security policy will reject.
middleware/args.js — the req.args normalizer
This is the one you'll feel most. It builds a single req.args object: the request body for a POST, the query string for a GET. Why it matters: your action code never has to know or care whether it was called via POST or GET — it always reads req.args. That's a real ergonomic win and it's why the convention is "never touch req.body or req.query directly." It also runs parseUrlQueryFilter here, converting bracket-notation filters like createdAt[gte]=… into Sequelize operators, so range/date filtering arrives ready to drop into a where clause (see §7 / the query helpers). It's placed after body parsing (so the body exists) and before routes (so every action sees a normalized req.args).
This API speaks exactly two HTTP methods: GET for reads and POST for everything that changes state (create, update, delete). We deliberately don't use PUT, PATCH, or DELETE.
Why: the verb adds nothing here — the action name already says what's happening (V1Update, V1Delete), so the method is redundant ceremony. Sticking to two methods keeps args.js dead simple (body = POST, query = GET, no per-verb branching), avoids inconsistent client/proxy/CORS handling of the less-common verbs, and means every engineer can predict an endpoint's method without thinking. The router uses router.all(...) and the action does the real work regardless of verb.
Most of all, it's a waste of developer time to sit and debate whether an endpoint should be PUT vs PATCH vs DELETE — many requests legitimately fit several of them, or some of each, and there's rarely a clean answer. We'd rather not burn any thought on that distinction at all: if it changes state, it's a POST. Done.
middleware/auth.js — identifies the caller
Reads the Authorization scheme (jwt-user / jwt-admin), runs the matching Passport JWT strategy, and — on success — attaches the authenticated record to req.user or req.admin (and sets the locale). Why here, before routes? Because the controller's first job is to gate access by role (if (req.admin) …), so identity must already be resolved by the time the controller runs. Crucially it's non-blocking: if there's no recognized token it just calls next() — it does not reject. That lets public endpoints work, and pushes the actual "who's allowed" decision into the controller per-route. Full detail in §11.
Because this middleware has already done the work, every controller and action downstream just reads req.user or req.admin to know exactly who is logged in — no token parsing, no DB lookup, no boilerplate. The authenticated record is simply there on the request. That's what makes the code so clean: gating by role is a one-liner (if (req.admin)), and scoping a query to the caller is just where: { userId: req.user.id }. The hard part (verify token → load the user) is solved once, in one place, for the whole app.
middleware/error.js — runs last
Express error-handling middleware (the kind with four args, (err, req, res, next)) only catches errors from middleware registered before it — so it must be registered last, after the routes. Anything that throws or rejects anywhere in the chain lands here; it logs the error (with the requestId from step 1), notifies in production, and returns a clean 500. This is why your action code never returns a 500 itself — you throw and let this catch it.
Each middleware sets up something the next stage depends on: id gives everyone a correlation id → args gives the action a uniform input → auth gives the controller an identity to gate on → error (last) is the safety net for the whole chain. Reorder them and things break.
middleware/exit.js — graceful shutdown
There's one more middleware, and it exists for a different reason: shutting down cleanly. Every deploy or restart (Heroku, a crash-restart, Ctrl-C) sends the process a SIGTERM. If the process just dies, any in-flight request is dropped and connections (DB/Redis/sockets) are left dangling. exit.js prevents that. It has two halves:
exit.middleware— registered inserver.js(it runs on every request). Normally it just callsnext(). But once shutdown has started it short-circuits with503 SERVICE_UNAVAILABLEand aConnection: closeheader — so the load balancer stops sending new work to this instance and clients don't reuse the dying connection.gracefulExit(server)— the actual drain, called from the signal handlers.
The wiring spans the two entry points:
// index.js — each clustered worker (throng) registers the signal handlers after it starts listening
server.listen(PORT, () => {
process.on('SIGTERM', async () => await gracefulExit(server)); // deploy / restart / kill
process.on('SIGINT', async () => await gracefulExit(server)); // Ctrl-C in dev
});
// middleware/exit.js — drain in order, then exit
async function gracefulExit(server) {
if (isShuttingDown) return; // re-entrant guard (two signals won't double-run it)
isShuttingDown = true; // from now on exit.middleware answers new requests with 503
// safety net: if draining hangs, force-exit after 30s so the platform doesn't SIGKILL mid-write
setTimeout(() => process.exit(1), 30000);
await queue.closeAll(); // stop pulling/queuing background jobs
await socket.close(); // close socket.io connections
await models.db.close(); // close the Sequelize (Postgres) pool
server.close(() => process.exit(0)); // stop accepting connections, then exit clean once in-flight requests finish
}So the full sequence on a deploy: SIGTERM → flip the flag (new requests get 503) → finish in-flight requests → close queue, sockets, DB → server.close() → process.exit(0), with a 30-second hard cap so a stuck connection can't block the deploy forever. index.js runs this per worker (it clusters one worker per CPU via throng), so every process in the cluster drains independently.
2. Routes → 3. Controller
The feature's routes.js maps a URL to a controller method; the controller is a thin router that picks the right action based on who's calling, then returns the result.
API URLs are all lowercase with no dashes, underscores, or camelCase — run the words together. A multi-word action becomes one token: /v1/users/logoutall (not logout_all, logout-all, or logoutAll), /v1/admins/updateemail, /v1/users/smssendcode. The path simply mirrors the action name lowercased (V1LogoutAll → logoutall). Why: every URL is predictable from the action name and there's no hyphen-vs-underscore bikeshedding; a minor side benefit is run-together words read slightly less cleanly to casual crawlers — but that's a side effect, not the point (real protection is auth, not URL spelling). Note this is the opposite of page URLs, which use lowercase-with-dashes.
// app/Order/routes.js — lowercase, no separators; registered into the global routes.js
router.all('/v1/orders/update', controller.V1Update);
// app/Order/controller.js — pick the action by role, send the action's object as the response
async function V1Update(req, res, next) {
let method = null;
// pick the action based on who is calling
if (req.admin) {
method = 'V1UpdateByAdmin';
} else if (req.user) {
method = 'V1UpdateByUser';
} else {
return res.status(401).json(errorResponse(req, ERROR_CODES.UNAUTHORIZED));
}
try {
const result = await actions[method](req, res);
return res.status(result.status).json(result);
} catch (error) {
// hand off to middleware/error.js — never return res.status(500) yourself
return next(error);
}
} // END V1UpdateYou don't only branch on who is calling (the role) — you can also branch on what device they're calling from. The same logical action can have a different implementation on web vs. mobile (different payload shape, different side effects), so the controller picks by device too:
// controller.js — branch on role AND device
async function V1Sync(req, res, next) {
let method = null;
if (req.user) {
// req.device is resolved from a client header (see §11 — explained later)
if (req.device === 'MOBILE') {
method = 'V1SyncByUserOnMobile';
} else {
method = 'V1SyncByUserOnWeb';
}
} else {
return res.status(401).json(errorResponse(req, ERROR_CODES.UNAUTHORIZED));
}
try {
const result = await actions[method](req, res);
return res.status(result.status).json(result);
} catch (error) {
return next(error);
}
} // END V1SyncAction names carry an optional On{Device} suffix (V1SyncByUserOnMobile, V1SyncByUserOnWeb) exactly so the controller can route to the right one. The device is derived from a client header sent by the frontend — we cover how that header works and how the token's audience is scoped per device in §11 Authentication. The takeaway here: the controller is the one place that resolves both the role and the device into a single action to run.
Auth rejection happens in the controller, before the action runs — the wrong role never reaches business logic. And the action returns a plain object ({ status, success, … }); only the controller touches res for the final response.
4. Action → 5. Response
The action does the real work and returns an object. The controller serializes it. On an unexpected throw, next(error) hands off to the error middleware. That's the whole loop.
Writing an Action
Actions are where your real-time business logic lives — one file per action under actions/. They follow a strict shape so any engineer can read any action instantly.
The JS file structure (every .js file)
Every .js file in the codebase follows the same top-to-bottom order. Here it is as a real file — read the comments:
/**
* Header comment — what this file is
*/
'use strict'; // strict mode, always first
// env (only where a file reads process.env directly)
const { NODE_ENV } = process.env;
// built-in node modules
const path = require('path');
const crypto = require('crypto');
// third-party modules
const joi = require('joi');
const moment = require('moment');
// services (app/../services) — note: the queue SERVICE is required here
const queue = require('../../../services/queue');
const { errorResponse, ERROR_CODES } = require('../../../services/error');
// helpers (app/../helpers)
const { getOffset } = require('../../../helpers/cruqd');
// models (the Sequelize models)
const models = require('../../../models');
// queues — grab the queue INSTANCES right after models (queue.js itself lives in services above)
const OrderQueue = queue.get('OrderQueue');
// module-level constants (UPPER_CASE)
const DEFAULT_LIMIT = 25;
// module.exports BEFORE the implementations — a table of contents for the file
module.exports = {
V1ReadByUser
};
/**
* Each method is defined below, in the same order as module.exports.
*/
async function V1ReadByUser(req, res) {
// ...
} // END V1ReadByUserTwo ordering rules inside the imports: they're sorted by increasing line length, and plain requires come before destructured ones. Every function closes with a // END <name> comment so you can see where long functions end.
Don't confuse the two. The queue service (services/queue.js) is required in the services section like any other service. But the actual queue instances you work with — queue.get('OrderQueue') — go in their own queues section right after models. So the full order is: env → built-ins → third-party → services → helpers → models → queues → consts → module.exports → methods.
List every method in module.exports above the implementations (function declarations hoist, so this works). It gives every file a readable table of contents. It doesn't have to be the very top — constants may sit above it.
The anatomy of an action
/**
* Read and return an order
*
* GET/POST /v1/orders/read
* Must be logged in · Roles: ['user']
*
* req.args = { @id - (STRING - REQUIRED): the order id }
* Success: returns the order
* Errors:
* 400: BAD_REQUEST_INVALID_ARGUMENTS
* 404: ORDER_NOT_FOUND_ORDER_DOES_NOT_EXIST
*/
async function V1ReadByUser(req, res) {
// i18n: a locale-aware translator for user-facing strings. In an HTTP action you
// normally use req.__('KEY') (locale auto-set by middleware); getLocalI18n() gives
// you a standalone instance for places without a req (covered more in §8).
const i18n = lang.getLocalI18n();
// 1. validate req.args with Joi
const schema = joi.object({
id: joi.string().uuid().required()
});
const { error, value } = schema.validate(req.args);
if (error) {
return errorResponse(req, ERROR_CODES.BAD_REQUEST_INVALID_ARGUMENTS, joiErrorsMessage(error));
}
req.args = value; // Joi has now coerced types ('5' -> 5, 'true' -> true)
try {
// 2. business logic — scope to the owner; exclude sensitive fields
const order = await models.order.findOne({
where: {
id: req.args.id,
userId: req.user.id
},
attributes: {
exclude: models.order.getSensitiveData()
}
});
if (!order) {
return errorResponse(req, ERROR_CODES.ORDER_NOT_FOUND_ORDER_DOES_NOT_EXIST);
}
// 3. flat success response
return {
status: 200,
success: true,
order: order.dataValues
};
} catch (error) {
// hand off to the controller -> error middleware (never return res.status(500))
throw error;
}
} // END V1ReadByUserThe JSDoc header is your contract — and your test checklist
Every action documents its route, auth, req.args (each typed), success shape, and every error code it can return. That Errors: list is exactly what your tests must cover (see §12).
Validation: always Joi, always req.args
Validate first, every time. On failure return errorResponse(req, ERROR_CODES.BAD_REQUEST_INVALID_ARGUMENTS, joiErrorsMessage(error)), then assign req.args = value to get Joi's coerced types. Never touch req.body/req.query.
helpers/schemas.js
When the same shape shows up across features — an address, a phone number, a date, a { gte, lte } number-comparison filter — don't redefine it in each action. Those reusable Joi pieces live in helpers/schemas.js as small factory functions (e.g. addressSchema(), phoneSchema(), numberComparisonSchema()); import one and drop it into your joi.object({ … }). Add a new one there whenever you find yourself writing the same validation twice.
A write action: transaction → helper → queue → socket
The read above is the simplest shape. A write action usually does more after validation: wrap the DB work in a transaction, run pure logic through a helper (so it's unit-testable without a DB), enqueue a background job for slow follow-up work, and emit a socket event so connected clients update in real time. Each of these is a recurring pattern you'll reach for constantly — the comments flag where each one is covered in depth:
async function V1UpdateByUser(req, res) {
// ...validate req.args with Joi (same as the read above)...
// open a transaction: every write below either all commits or all rolls back
const t = await models.db.transaction();
try {
// load the row, scoped to the caller (req.user — courtesy of the auth middleware)
const order = await models.order.findOne({
where: {
id: req.args.id,
userId: req.user.id
},
transaction: t
});
if (!order) {
// roll back before returning an error from inside a transaction
await t.rollback();
return errorResponse(req, ERROR_CODES.ORDER_NOT_FOUND_ORDER_DOES_NOT_EXIST);
}
// pure logic lives in helper.js — no DB, no req — so it's trivial to unit-test (see §12)
const nextStatus = helper.computeNextStatus(order.status, req.args.action);
// the actual DB write, inside the transaction
await order.update({ status: nextStatus }, { transaction: t });
// commit FIRST — nothing below should observe an uncommitted row
await t.commit();
// enqueue slow follow-up work as a background job (the worker runs it) — see §9
await OrderQueue.add('V1SendReceiptTask', {
orderId: order.id
});
// push a real-time update to connected clients — ALWAYS after commit — see §10
socket.getIO()
.to(`${SOCKET_ROOMS.USER}${socketWrapper(req.user.id)}`)
.emit(SOCKET_EVENTS.ORDER_UPDATED, { id: order.id, status: nextStatus });
// flat success response
return {
status: 200,
success: true,
order: order.dataValues
};
} catch (error) {
// on any throw, roll the transaction back, then let it propagate to the error middleware
await t.rollback();
throw error;
}
} // END V1UpdateByUserValidate → transaction → helper (pure logic) → commit → then enqueue jobs and emit sockets. The queue and socket steps come after commit() on purpose: if you enqueue or emit before committing and the commit then fails, you've told the rest of the system about a change that never happened. Transactions, queues, sockets, and helpers each get their own section (§7/§9/§10/§12) — this is just where they come together.
Actions are just exported functions, so one action can require and call another to reuse its logic instead of duplicating it — and it can queue.add(...) to hand work off to a background task. The whole toolbox is available inside an action: read, transaction, edit, enqueue jobs, emit sockets, and call other actions. Caveat: an action expects a req-shaped first argument (req.args, often req.user/req.admin, the i18n helpers); if the callee reaches deep into req/res, extract the shared logic into a helper that both call rather than shimming a fake req.
An action does three things: load (DB/API) → process (business logic) → save (DB/side effects). Pull the pure process step out into a helper — plain inputs, plain output, no DB, no req — so it can be unit-tested exhaustively without booting a server or database, while the action stays thin and just wires the pieces together. If you can't test a piece of logic without a running DB, that's the signal to extract it. Full detail in §12.
The flat response shape
Return a plain object. status and success are always required; everything else is named after what it contains. No data nesting.
| Status | When |
|---|---|
200 | Default — reads, updates, queries, logins. |
201 | Created a new database record. |
202 | Work handed off to a background job (returns e.g. a jobId). |
// 201 — created a new record
return {
status: 201,
success: true,
order: newOrder
};
// 202 — accepted, work continues in a background job
return {
status: 202,
success: true,
jobId: job.id
};
// 200 — default (e.g. a list/query)
return {
status: 200,
success: true,
orders: rows,
total: count
};One method per role — never if/else on role
If an action behaves differently for admins vs users, write separate methods (V1CreateByAdmin, V1CreateByUser) — those are the ones exported and wired into the controller. The controller picks which to call. This keeps each path independently readable, testable, and safe to change.
Sharing the bulk of the logic: a private V1Create in the same file
Very often the two role methods are ~90% identical. The pattern for that is not to push everything into helper.js. Instead, in the same action file, write a third function — V1Create — that does the shared bulk of the work, and have both V1CreateByUser and V1CreateByAdmin call it. That shared function is not exported and never referenced by the controller — it's a private internal worker that lives right next to the two public methods:
// only the two role methods are exported — the controller calls these
module.exports = {
V1CreateByUser,
V1CreateByAdmin
};
// PUBLIC: user entry point — does the user-specific bit, then delegates
async function V1CreateByUser(req, res) {
// ...user-only validation / defaults...
return V1Create(req, { isAdmin: false });
} // END V1CreateByUser
// PUBLIC: admin entry point — does the admin-specific bit, then delegates
async function V1CreateByAdmin(req, res) {
// ...admin-only validation / extra fields...
return V1Create(req, { isAdmin: true });
} // END V1CreateByAdmin
// PRIVATE: the 90% shared logic. NOT in module.exports, NOT called by the controller.
// This is where the real work lives — kept in the action file, just not exported.
async function V1Create(req, { isAdmin }) {
// ...validate, transaction, write, queue, socket — the bulk of the action...
return {
status: 201,
success: true,
order: newOrder
};
} // END V1Createhelper.js for big shared logic
helper.js is for small, pure, reusable bits (a calculation, a formatter) — things worth unit-testing in isolation. When the shared piece is the majority of the action, don't move it into another file. Keep it as a private V1Create in the action file so the logic stays where you'd look for it. Rule of thumb: small slice → helper.js; big shared body → private function in the action.
V{version}{Action}[By{Role}][On{Device}] — e.g. V1Create, V1UpdateByAdmin, V1SyncByUserOnMobile. The plain V1Create (no role/device) is the conventional name for the shared, non-exported worker. Create actions with yarn gen Order -a V1Create.
Query actions: pagination, filtering, sorting
A Query action returns many records — a list/search/index endpoint. It's the same action shape as everything else, plus three concerns: paginate, filter, and sort. The helpers for all three live in helpers/cruqd.js (CRUQD = Create, Read, Update, Query, Delete).
The three knobs, explained
Pagination — page & limit. You don't return the whole table; you return one page of rows.
limit— how many rows per page (e.g.25). Always give it amaxso a caller can't ask for everything at once.page— which page, 1-based (page: 1is the first page,page: 2the next, …).getOffset(page, limit)turns those into Sequelize'soffsetfor you (page 3 at limit 25 → skip the first 50 rows). You returntotalalongside the rows so the client can compute how many pages exist.
Sorting — sort is a comma-separated list of columns. Pass it to getOrdering(sort):
- Prefix a column with
-for descending; no prefix means ascending. SocreatedAt= oldest first,-createdAt= newest first. - Order in the list = sort priority. The first column is the primary sort; the next only breaks ties within it, and so on.
'-startTime,partySize'means "newest start time first, and when two share the same start time, order those by party size ascending." - It's just a string, so a client sends
sort=-startTime,partySizeand you forward it straight togetOrdering.
Filtering — a filter can also take a list. A single value (status=CONFIRMED) matches one; a comma-separated list (status=PENDING,CONFIRMED) should match any of them. convertStringListToWhereStmt turns that list arg into a Sequelize IN (…) clause for you (and removes the raw arg). For range/date filters, parseUrlQueryFilter has already converted bracket operators like createdAt[gte]=… into Sequelize operators back in middleware/args.js (§5) — so those arrive ready to drop into the where.
async function V1QueryByUser(req, res) {
const schema = joi.object({
page: joi.number().integer().min(1).default(1), // which page (1-based)
limit: joi.number().integer().min(1).max(100).default(25), // rows per page — capped at 100
sort: joi.string().default('-startTime'), // comma-list of columns; '-' = descending
status: joi.string().optional() // a comma-separated LIST, e.g. 'PENDING,CONFIRMED'
});
const { error, value } = schema.validate(req.args);
if (error) {
return errorResponse(req, ERROR_CODES.BAD_REQUEST_INVALID_ARGUMENTS, joiErrorsMessage(error));
}
req.args = value;
try {
// SECURITY: always scope the list to the owner — never return another user's rows.
// The flattened owner FK (§7) makes this a flat, indexed filter.
const where = { userId: req.user.id };
// turn a 'PENDING,CONFIRMED' list arg into status: { [Op.in]: ['PENDING','CONFIRMED'] }
convertStringListToWhereStmt(where, req.args, [
{ name: 'status', col: 'status' }
]);
const { count, rows } = await models.booking.findAndCountAll({
where,
order: getOrdering(req.args.sort), // '-startTime' → [['startTime','DESC']]
limit: req.args.limit,
offset: getOffset(req.args.page, req.args.limit),
attributes: {
exclude: models.booking.getSensitiveData()
}
});
// flat response: name the array after the content, include the paging metadata
return {
status: 200,
success: true,
bookings: rows,
total: count,
page: req.args.page,
limit: req.args.limit
};
} catch (error) {
throw error;
}
} // END V1QueryByUserAlways scope to the owner (where: { userId: req.user.id }) — a list that forgets this leaks other users' data. Always cap limit with a sane max so a caller can't trigger an unbounded table scan. Test pagination boundaries, each filter, the sort order, and that one user can't see another's rows (§12).
A note on req.params vs req.args
You'll occasionally see req.params — those are URL path parameters (the :id in a route like /v1/orders/:id). req.args is the body/query data the args middleware normalizes (§5). In this codebase almost everything travels in req.args (POST body or GET query); path params are rare. Validate and read req.args — never req.body/req.query — and reach for req.params only when a route actually declares a path segment.
The Data Layer
Three pieces define your data: the model (how the ORM sees the table at runtime), the migration (what actually changes the real database), and schema.sql (human-readable documentation). Keep all three in sync.
| File | Role | Executed? |
|---|---|---|
app/<F>/model.js | Runtime table definition for Sequelize (queries, validation, associations). Freely editable. | Yes (runtime). Test DB is built from models via sync. |
migrations/*.js | The real schema changes for dev/prod. Once deployed, never edited. | Yes (on yarn migrate). |
database/schema.sql | Documentation of every table — scan the whole schema in one place. | No (never executed). |
The model — the standard shape
const sensitiveData = ['salt', 'password', 'passwordResetToken']; // never exposed to anyone
const privateData = sensitiveData.concat(['phone', 'birthdate']); // private between users
module.exports = (sequelize, DataTypes) => {
const Order = sequelize.define('order', {
id: {
type: DataTypes.UUID,
allowNull: false,
defaultValue: DataTypes.UUIDV4,
primaryKey: true,
validate: {
isUUID: 4
}
},
// foreign keys are declared in associate(), not here
status: {
type: DataTypes.ENUM(...ORDER_STATUSES),
allowNull: false,
defaultValue: 'PENDING'
},
isPaid: {
type: DataTypes.BOOLEAN,
allowNull: false,
defaultValue: false
}
}, {
timestamps: true, // auto-manage createdAt / updatedAt columns
paranoid: true, // soft delete: destroy() sets deletedAt instead of a real DELETE
// freezeTableName: stop Sequelize from auto-pluralizing the model name into the table name.
// We name the table explicitly via tableName below, so freezing keeps it exactly as written.
freezeTableName: true,
tableName: 'Orders', // the real table name — PascalCase, plural
// sensitive fields excluded from every query automatically (unless you opt back in)
defaultScope: {
attributes: {
exclude: sensitiveData
}
},
// always index foreign keys; named {Table}_{col}_{idx|unique}, mirrored in the migration
indexes: [
{ name: 'Orders_userId_idx', fields: ['userId'] }
]
});
// associate(): this is the FOREIGN KEY section. Sequelize calls it after all models load,
// so you can reference other models here. Declare every relationship + its FK column in here
// (NOT in the attributes block above), each with an explicit onDelete/onUpdate.
Order.associate = models => {
Order.belongsTo(models.user, {
as: 'user',
foreignKey: {
name: 'userId', // creates the userId FK column on Orders
allowNull: false
},
onDelete: 'CASCADE', // delete the user → their orders go too
onUpdate: 'CASCADE'
});
};
// extra model functions: small helpers attached to the model so callers don't hardcode the lists.
// e.g. actions do attributes: { exclude: models.order.getSensitiveData() }. Add getPrivateData()
// here too if the table has fields that are private between users.
Order.getSensitiveData = () => sensitiveData;
return Order;
};- UUID v4 primary keys, generated at the ORM level (you know the id before insert).
paranoid: true= soft deletes. Standard queries hidedeletedAtrows; useModel.scope(null)to see them.- Foreign keys go in
associate()with explicitonDelete/onUpdate, never in the attributes block. - Named indexes
{Table}_{col}_{idx|unique}, mirrored in the migration. Always index every FK. sensitiveData(never exposed) vsprivateData(hidden between users); expose viagetSensitiveData()/getPrivateData().
These are the highlights — the complete model/DB ruleset is in §17 Conventions Quick-Reference.
Constants & enums
The model above uses DataTypes.ENUM(...BOOKING_STATUSES) — that BOOKING_STATUSES is a global constant. All global constants live in one file: helpers/constants.js. Anything that's a fixed set of values (statuses, roles, types, formats) goes there, never hard-coded inline.
The conventions for that file:
- Names are
UPPER_CASE_WITH_UNDERSCORES— that casing is the signal "this is a global constant." - An array is named with a plural word (
LANGUAGES); an object with a singular word (LANGUAGE). When that reads awkwardly, append_ARR/_OBJinstead. - For an enum you almost always want both shapes — the object for referring to a value by name in code, and the array for Sequelize's
ENUM(...)and Joi's.valid(...).
// helpers/constants.js
module.exports = {
// object (singular) — reference a value by name: BOOKING_STATUS.CONFIRMED
BOOKING_STATUS: {
PENDING: 'PENDING',
CONFIRMED: 'CONFIRMED',
CANCELLED: 'CANCELLED'
},
// array (plural) — feed Sequelize ENUM(...) and Joi .valid(...)
BOOKING_STATUSES: ['PENDING', 'CONFIRMED', 'CANCELLED']
};Then the dual export gets used in both places, with no duplicated string literals drifting apart:
// in the model
status: {
type: DataTypes.ENUM(...BOOKING_STATUSES),
defaultValue: BOOKING_STATUS.PENDING
}
// in an action's Joi schema
status: joi.string().valid(...BOOKING_STATUSES)Two different casings, don't mix them up: the ENUM type name in the DB is ALL-CAPS with no underscores/dashes (BOOKINGSTATUS); the values are ALL_CAPS_WITH_UNDERSCORES (GOOGLE_CALENDAR). Keep the ENUM's values in sync with the constant array.
The most important schema rule: carry the owner FK to every descendant
Don't FK only to the immediate parent. When data is nested — and it almost always is — put every ancestor's id on the descendant, not just the row directly above it. In practice that means a deep leaf carries its parent's id, and its parent's parent's id, and so on up to the top-level owner. Keep walking up the hierarchy and add each one.
It looks like duplication, but it's deliberate: it flattens the hierarchy. Because every ancestor's id is sitting right on the row, you can query "all X for any ancestor above it" — all items for a user, all items for an order, all items for a workspace — as a single indexed where on that one column. No joins, no walking the tree. The owner id in particular is also your security scope (where: { userId: req.user.id }), so having it on every table means you can authorize and filter any descendant directly. Joins up a multi-level hierarchy on a hot path are exactly what this avoids.
erDiagram
User ||--o{ UserOrder : owns
User ||--o{ UserOrderItem : "owns (flattened)"
UserOrder ||--o{ UserOrderItem : contains
User { uuid id }
UserOrder { uuid id "FK userId" }
UserOrderItem { uuid id "FK userOrderId + FK userId" }
UserOrderItem carries both userOrderId (its parent) and userId (the owner) — so "all items for this user" is a flat, indexed, join-free query.
The obvious objection: "duplicated data drifts" — what stops an item's userId from disagreeing with its order's userId? The answer, and the thing that makes this pattern safe, is the composite foreign key. Give UserOrders a unique (id, userId), then have UserOrderItems.(userOrderId, userId) reference that pair. Now Postgres itself rejects any item whose userId doesn't match its parent order's userId — the redundancy cannot drift, because the database enforces the agreement at write time. So you get both halves: join-free reads from the flattened ids, and a hard integrity guarantee that the duplicated id is always correct.
Always carry the top-level owner (userId) down to every descendant. Carry intermediate ancestors only when you actually query a deep leaf by that level — don't add every ancestor reflexively.
DB conventions (quick hits)
- Tables PascalCase plural (
Orders); columns camelCase. - Column order: id → FKs → vendor IDs → custom columns →
deletedAt/createdAt/updatedAt. - Booleans start with a verb:
is/has/can/does(isActive,hasFailed). - FK columns
<entity>Id→ PascalCase plural table; multiple FKs to the same table get a role prefix (hostUserId,bookerUserId). - Vendor IDs prefixed (
stripeId); ENUM type names ALL-CAPS-no-underscores, valuesALL_CAPS_WITH_UNDERSCORES; all times UTC.
This is a teaser — the full naming, column-order, and DB rules live in §17 Conventions Quick-Reference (and the authoritative docs/conventions.txt).
Migrations
Generate with yarn model (new table) or yarn migration (alter), rename to the convention (<ts>-create-Order-model.js / <ts>-add-cols-x-to-Orders-tbl.js), and wrap up/down in a transaction. Never drop or rename in place — add a new column, backfill, remove the old one much later (rollback safety). Mirror every change in the model so sync (tests) matches.
Two database connections: the CLI config vs the runtime connection
This trips people up, so it's worth being explicit. There are two separate places the app connects to Postgres, and they're configured differently:
- Runtime —
database/index.js. What the app uses (every model query, at runtime). It builds the Sequelize instance fromDATABASE_URLwith the connection pool, UTC timezone, SSL in prod, etc. This is the one yourequireviamodels. - CLI / migrations —
config/config.js. What the Sequelize CLI uses (yarn migrate,yarn rollback,yarn model,yarn migration). The CLI can't see your runtime file — it looks for its own config, pointed at by.sequelizerc.
.sequelizerc is a tiny shim that tells the CLI where its config lives:
// .sequelizerc
const path = require('path');
module.exports = {
config: path.resolve('config', 'config.js')
};And config/config.js loads the right .env.<env> by NODE_ENV, then exports one block per environment — each just pointing the CLI at the same DATABASE_URL:
// config/config.js — used ONLY by the Sequelize CLI (migrations), not at runtime
const path = require('path');
const { NODE_ENV } = process.env;
// load the matching env file so DATABASE_URL is populated when you run a CLI command
if (NODE_ENV === undefined || NODE_ENV === 'development')
require('dotenv').config({ path: path.join(__dirname, '.env.development') });
else if (NODE_ENV === 'test')
require('dotenv').config({ path: path.join(__dirname, '.env.test') });
module.exports = {
development: { use_env_variable: 'DATABASE_URL', dialect: 'postgres', dialectOptions: { decimalNumbers: true } },
test: { use_env_variable: 'DATABASE_URL', dialect: 'postgres', dialectOptions: { decimalNumbers: true } },
production: { use_env_variable: 'DATABASE_URL', dialect: 'postgres', ssl: true, dialectOptions: { decimalNumbers: true, ssl: { required: true, rejectUnauthorized: false } } }
};The Sequelize CLI runs as its own process — it doesn't boot your app, so it can't use database/index.js. It needs a standalone config it can read directly, selected by NODE_ENV. Both ultimately read the same DATABASE_URL, so they point at the same database — they're just two doors into it. (This is why NODE_ENV matters when you run yarn migrate vs yarn migrate:prod: it picks which block of config/config.js, and which .env file, the CLI uses.)
Errors & i18n
Two intertwined systems: structured, machine-readable error codes, and translated user-facing strings. Every error message is an i18n key.
The error lifecycle: 4xx vs 5xx
4xx (expected): bad input, failed auth, business-rule violations. You handle these explicitly with errorResponse — they never reach the error middleware.
5xx (unexpected): a thrown exception or rejected promise. You never return a 500 yourself — you throw (or let it propagate) and middleware/error.js catches it, logs it, and responds.
Error codes live in each feature's error.js
They're auto-aggregated into a global ERROR_CODES at startup. Each code has three fields and a precise naming scheme:
// app/Admin/error.js (grouped by the action that owns them)
const LOCAL_ERROR_CODES = {
// V1Login
ADMIN_BAD_REQUEST_INVALID_LOGIN_CREDENTIALS: {
error: 'ADMIN.BAD_REQUEST_INVALID_LOGIN_CREDENTIALS', // ← machine code (frontend branches on this)
status: 400,
messages: [ // ← an ARRAY of i18n keys
'ADMIN[invalid_login_credentials]', // index 0 — the default message
'ADMIN[invalid_login_credentials_locked]' // index 1 — pick by index at the call site
]
}
};| Identifier | Format | Example |
|---|---|---|
| JS object key | NAMESPACE_STATUS_DESCRIPTION (underscores) | ADMIN_BAD_REQUEST_INVALID_LOGIN_CREDENTIALS |
.error string | NAMESPACE.STATUS_DESCRIPTION (dot after namespace) | ADMIN.BAD_REQUEST_INVALID_LOGIN_CREDENTIALS |
The messages array lets one code carry multiple phrasings — pick one by index at the call site. Global codes (no namespace) like BAD_REQUEST_INVALID_ARGUMENTS, UNAUTHORIZED, INTERNAL_SERVER_ERROR live in services/error.js.
Returning errors: errorResponse
const { ERROR_CODES, errorResponse, errorResponseRollback, joiErrorsMessage } = require('../../../services/error');
return errorResponse(req, ERROR_CODES.BAD_REQUEST_INVALID_ARGUMENTS, joiErrorsMessage(error)); // custom string (Joi)
return errorResponse(req, ERROR_CODES.ADMIN_BAD_REQUEST_ACCOUNT_INACTIVE); // default message
return errorResponse(req, ERROR_CODES.ADMIN_BAD_REQUEST_EMAIL_CONFLICT, 1); // alternate message (index 1)
// inside an open transaction, roll back first:
return errorResponseRollback(t, req, ERROR_CODES.ORDER_BAD_REQUEST_PAYMENT_FAILED);Pass req as the first arg in HTTP actions (the i18n middleware attaches __() to it); pass a getLocalI18n() instance in tasks/services. In tasks and socket-invoked actions you throw instead of returning errorResponse.
i18n: keys, compilation, safety
Strings live per-feature in app/<F>/languages/en.js (and global languages/en.js), keyed NAMESPACE[snake_case]. They're compiled into locales/*.json by yarn lang — never edit locales/ directly.
// app/Admin/languages/en.js
module.exports = {
'ADMIN[invalid_login_credentials]': 'The email and/or password you entered is incorrect.',
'ADMIN[reset_email_sent]': 'An email has been sent to {{email}}.' // {{var}} interpolation
};- In actions:
req.__('ADMIN[key]', { var })— locale is set automatically. - In tasks/services:
const i18n = lang.getLocalI18n(); i18n.setLocale(locale); i18n.__('…'). - After editing any language file, run
yarn lang. It also validates that every.__('KEY')in the code exists — missing keys throw in test/production (andyarn testrunsyarn langfirst).
Add the i18n key before running tests. A missing key fails the yarn lang step that yarn test depends on, and the whole suite won't run.
Sending email (mailers)
Transactional emails (welcome, password reset, booking confirmation) are mailers. A mailer is an EJS template plus the data to render it; the services/email.js wrapper sends it. Mailers are scaffolded, never hand-created: yarn gen <Feature> -m <Mailer>.
Where mailers live:
- Feature mailers —
app/<Feature>/mailers/<name>/index.ejs— for emails specific to that feature. - Global mailers — the top-level
mailers/— for app-wide emails (e.g. a shared layout/footer). - Like i18n, email copy is translatable;
yarn gulpwatches templates and regenerates apreview.htmlso you can eyeball the rendered email without sending one.
You almost always send the email from a task, not inline in the action — sending is slow and shouldn't block the response. The action enqueues; the worker renders and sends (the §14 worked example does exactly this via its notify service):
// inside a task — render + send through the service wrapper
const email = require('../../../services/email');
await email.send({
to: user.email,
template: 'booking-confirmation', // the EJS mailer
locale: user.locale, // mailers respect i18n, same as actions
data: {
name: user.firstName,
startTime: booking.startTime
}
});In tests, mock services/email.js (your wrapper) with jest.spyOn and assert it was called with the right to/template/data — never hit a real email provider (§12).
Background Jobs
When work is slow, scheduled, or shouldn't block a response, it becomes a task — a background job processed by the worker off a Redis (Bull) queue.
flowchart LR
A["Action
returns 202 + jobId"] -->|queue.add| Q
CR["Cronjob
on a schedule"] -->|queue.add| Q
AGG["Aggregate task
fans out"] -->|queue.add per record| Q
Q[("Redis queue")] --> WK["worker.js
maps job name to task"]
WK --> T["task
does the work"]
T -->|success| OK["return true"]
T -->|throws| F["Bull marks failed, calls queueError"]
classDef hot fill:#c15f3c,stroke:#a84e2f,color:#fff;
class T,WK hot;
Jobs are enqueued from an action, a cronjob, or another task; the worker routes each job name to its task function.
Anatomy of a task
A task is basically the same as an action — same file structure, same conventions, same kinds of work. It validates its input with Joi, reads and writes the database, opens transactions, edits records, can enqueue further jobs, and can even call actions (see below). The only real differences: it receives a job (args come from job.data instead of req.args), it has no res, and on failure it throws instead of returning errorResponse (success is return true). If you know how to write an action, you know how to write a task.
/**
* Exports admin data to a CSV and emails it to the requesting admin.
*
* @job = {
* @id - (INTEGER - REQUIRED): ID of the background job (Bull assigns this)
* @data = {
* @adminId - (STRING - REQUIRED): ID of the admin requesting the export
* }
* }
*
* Success: Returns true
*/
async function V1ExportTask(job) {
// 1. validate job.data with Joi — exactly like an action validates req.args
const schema = joi.object({
adminId: joi.string().uuid().required()
});
const { error, value } = schema.validate(job.data);
if (error) {
throw new Error(joiErrorsMessage(error)); // tasks THROW on failure (no errorResponse, no res)
}
job.data = value; // Joi has now coerced types
try {
// 2. do the work — anything an action can do:
// read from the DB
const admin = await models.admin.findByPk(job.data.adminId);
// open a transaction for multi-step writes, exactly like an action
const t = await models.db.transaction();
try {
await admin.update({ lastExportedAt: new Date() }, { transaction: t });
await t.commit();
} catch (err) {
await t.rollback();
throw err;
}
// enqueue ANOTHER job (tasks can create jobs too — this is the fan-out pattern)
await AdminQueue.add('V1EmailExportReadyTask', { adminId: admin.id });
// call an ACTION from inside the task: require it and invoke its method.
// (actions are just functions — a task can reuse one instead of duplicating logic)
const result = await V1GenerateReport({ args: { adminId: admin.id } });
// ... the heavy lifting ...
// emit a socket event so the client knows the job finished (same as actions — §10).
// tasks have no req, so grab io via socket.getIO(); emit AFTER any commit.
socket.getIO()
.to(`${SOCKET_ROOMS.ADMIN}${socketWrapper(admin.id)}`)
.emit(SOCKET_EVENTS.ADMIN_EXPORT_READY, { adminId: admin.id });
return true; // tasks return true on success
} catch (error) {
throw error; // Bull marks the job failed → 'failed' handler → queueError
}
} // END V1ExportTaskJust to repeat (same as actions in §6): actions are just exported functions. A task can require a feature's action and call its method directly — handy when the same business logic is needed both in real time (the action) and in the background (the task). Likewise a task can queue.add(...) more jobs. So the toolbox inside a task is the full toolbox: read, transaction, edit, enqueue, call actions — the same as an action, just triggered by the worker instead of HTTP.
Caveat: actions are written for the request lifecycle, so they expect a req-shaped first argument (at minimum req.args, and often req.user/req.admin and the i18n helpers). When you call one from a task you have to hand-build that object ({ args: { … }, user: … }). If an action reaches deep into req/res, that's a sign the reusable logic should be extracted into a shared helper that both the action and the task call — cleaner than shimming a fake req.
Just to repeat (same as actions in §6): tasks follow the same testability rule — pull the pure process logic out into a helper so it can be unit-tested without a DB or a running worker, and keep the task itself thin (load → process → save). See §6 and §12 — the guidance is identical.
Action vs. Task — the differences at a glance
They're ~95% the same; only these four things differ:
| Action | ✅ Task | |
|---|---|---|
| Invoked by | controller (HTTP) | Bull worker (off the Redis queue) |
| Args from | req.args | job.data |
| On failure | return errorResponse(req, …) | throw (no res, no errorResponse) |
| On success | return { status, success, … } | return true |
| Everything else | — identical: Joi validation, DB reads/writes, transactions, enqueueing jobs, calling actions, same file structure & conventions — | |
Register the processor in worker.js
The generator makes the task file; you wire it up (one line per task) plus the three event handlers:
const AdminQueue = queue.get('AdminQueue');
AdminQueue.process('V1ExportTask', tasks.V1ExportTask);
AdminQueue.on('failed', async (job, err) => queueError(err, AdminQueue, job));
AdminQueue.on('stalled', async job => queueError(new Error('Queue Stalled.'), AdminQueue, job));
AdminQueue.on('error', async err => queueError(err, AdminQueue));
// another queue + task, wired the same way (one queue per feature, one process() line per task)
const UserQueue = queue.get('UserQueue');
UserQueue.process('V1CheckAllUsersTask', tasks.V1CheckAllUsersTask);
UserQueue.process('V1CheckUserTask', tasks.V1CheckUserTask); // a queue can host many tasks
UserQueue.on('failed', async (job, err) => queueError(err, UserQueue, job));
UserQueue.on('stalled', async job => queueError(new Error('Queue Stalled.'), UserQueue, job));
UserQueue.on('error', async err => queueError(err, UserQueue));Reliability: retries, backoff & idempotency
Bull is at-least-once: a job can run more than once (a retry after a transient failure, or stall-recovery when a worker dies mid-process). The queue service sets reliability defaults on every queue in services/queue.js, so you don't configure this per job:
// services/queue.js — defaultJobOptions applied to every job
{
attempts: 5, // retry transient failures
backoff: { type: 'exponential', delay: 5000 }, // 5s → 10s → 20s → 40s between attempts
removeOnComplete: 1000, // prune completed jobs (don't bloat Redis)
removeOnFail: 5000 // keep recent failures for inspection / replay
}- Final-failure alerting:
queueErrordistinguishes a retryable failure from a final one (all attempts exhausted) and only escalates the final one — the single place to wire Sentry. Bull keeps final failures in the failed set, so replay isqueue.getFailed()→job.retry(). - Tasks MUST be idempotent. Because retries and stall-recovery re-run a task, running it twice must be harmless: guard on state (
if (booking.isConfirmed) return true;), use upserts / unique constraints, wrap multi-step writes in a transaction, and dedupe the enqueue with a deterministic job id (queue.add(name, data, { jobId: \`export-${adminId}\` })) when you must not double-queue.
The same hazard exists at the edge: a client whose response is lost will retry the POST → a duplicate create/charge. Three defenses, cheapest first: (1) natural idempotency — "set to a value" updates are already safe, and for creates let the client supply the UUID so the PK/unique constraint dedupes (catch the conflict, return the existing row); (2) a unique request-token column for a few sensitive ops; (3) a Stripe-style Idempotency-Key middleware — client sends Idempotency-Key: <uuid>, the middleware caches the response in Redis under idem:<userId>:<key> and replays it on retry (returning 409 if one is already in flight). Make natural idempotency the default; reserve the middleware for genuinely dangerous mutations (payments, signups).
Three ways a job gets created
// 1. From an action (return 202 + jobId)
const job = await AdminQueue.add('V1ExportTask', {
adminId: req.admin.id
});
return {
status: 202,
success: true,
jobId: job.id
};
// 2. From a cronjob (cronjobs.js) — always UTC, enqueue only
new CronJob('0 0 * * * *', () => {
UserQueue.add('V1CheckAllUsersTask', {});
}, null, true, 'UTC');
// 3. From another task (the fan-out pattern, below)Large datasets: the aggregate + singular pattern
Never process a huge table in one task. Split into an aggregate task that only queries + fans out (one job per record), and a singular task that processes exactly one record. This gives parallelism and fault isolation — one record's failure doesn't abort the batch.
async function V1CheckAllUsersTask(job) { // aggregate — no business logic
const users = await models.user.findAll({
where: {
isActive: true
},
attributes: ['id']
});
for (const u of users) {
await UserQueue.add('V1CheckUserTask', {
userId: u.id
});
}
return users.length;
} // END V1CheckAllUsersTaskNo req means no auto-locale. Pass locale in job.data when enqueuing, then i18n.setLocale(job.data.locale) on a getLocalI18n() instance.
Cronjobs: scheduling recurring work
A cronjob is how work gets triggered on a schedule instead of by a request. They all live in the top-level cronjobs.js, which is the clock process — the third of the three processes (§2), run with yarn cron.
How it runs, and the rules that matter:
- It's its own process.
cronjobs.jsdoesn't run inside the web server or the worker — it's a separate dyno whose only job is to fire timers. In production there must be exactly one clock dyno (heroku ps:scale clock=1); two would double-fire every job. - A cron only enqueues — it never does the work. The callback's single job is to
queue.add(...)a task; the worker then runs it. No DB calls, no business logic inline. (So a scheduled job is really cron → queue → worker → task.) - Always
'UTC'(the last argument). All scheduling is in UTC — never server-local time. - Cron expressions are 6-field here:
second minute hour day month weekday(the leading seconds field is the one people forget). - Group by feature with a comment header, and grab that feature's queue once at the top.
// cronjobs.js
const CronJob = require('cron').CronJob;
const queue = require('./services/queue');
/***** USER *****/
const UserQueue = queue.get('UserQueue');
// Nightly cleanup of inactive users. A common trick: fire often in dev so you can
// watch it work, but use the real cadence in production.
const cleanupSchedule = NODE_ENV === 'development'
? '0 */5 * * * *' // every 5 minutes in dev
: '0 0 0 * * *'; // 00:00 UTC daily in prod
new CronJob(cleanupSchedule, () => {
// the callback ONLY enqueues — the task does the actual work
UserQueue.add('V1CleanupInactiveUsersTask', {});
}, null, true, 'UTC'); // start immediately, timezone UTCMost scheduled jobs touch many rows. Combine the two patterns: the cron enqueues an aggregate task on a schedule; the aggregate queries the targets and fans out one singular task per record. The cron stays a one-liner, and the heavy work is parallel and fault-isolated in the worker. To test it, test the task directly (§12) — the cron line itself is just configuration.
Real-Time (Sockets)
Real-time push is built on Socket.IO with a Redis adapter, so an event emitted on one clustered web instance reaches clients connected to any of them.
sequenceDiagram
participant C as Client
participant S as services/socket.js
participant A as Action
participant DB as Postgres
C->>S: connect (access token)
S->>S: authenticate (verify JWT + tokenVersion)
S->>C: join rooms, e.g. the USER room
C->>S: emit MESSAGE_CREATED {data}
S->>A: V1Create(args, {io, ROOMS, EVENTS, socketWrapper})
A->>DB: write (transaction)
DB-->>A: commit ✓
A->>S: io.to(room).emit(...) (AFTER commit)
S-->>C: MESSAGE_CREATED broadcast
Connect → authenticate → join rooms → action does DB work → emits after commit → clients receive it.
Rooms & events
Both are constants in services/socket.js, ALL_CAPS_WITH_UNDERSCORES. Events are named FEATURE_ACTION (MESSAGE_CREATED). Rooms are either broadcast (a fixed name like GLOBAL, ADMIN) or instance rooms built with socketWrapper(id) → USER<uuid>, CONVERSATION<uuid>.
The context-object pattern (avoiding a circular dependency)
services/socket.js imports feature actions, so those actions can't import socket back. Instead, socket passes a context object into the action:
// services/socket.js
socket.on(SOCKET_EVENTS.MESSAGE_CREATED, async (data, callback) => {
try {
const result = await V1MessageCreated(data, {
io,
SOCKET_ROOMS,
SOCKET_EVENTS,
socketWrapper
});
return callback(null, result);
} catch (error) {
// socket actions THROW; the handler returns the error via the callback
return callback(error);
}
});HTTP actions that need to emit use socket.getIO() (never import io directly — it's null at require-time).
The cardinal rule: emit AFTER commit
If you emit before t.commit() and the commit fails, clients get an event for a row that doesn't exist. Emit is the last thing before return.
await t.commit();
io
.to(`${SOCKET_ROOMS.CONVERSATION}${socketWrapper(conversationId)}`)
.emit(SOCKET_EVENTS.MESSAGE_CREATED, data);
return {
status: 200,
success: true,
message
};Auth & testing
Socket connections authenticate with the same access token as HTTP (verified identically, including tokenVersion — see §11). Never guard emits with NODE_ENV; in tests, jest.spyOn(socket, 'getIO') and assert the room + event + payload (see §12).
Authentication
A modern two-token model — a short-lived access token plus a long-lived, revocable refresh token — built on Passport, with one strategy and one session table per user type.
The two tokens
| Token | Lifetime | Form | Stored? |
|---|---|---|---|
| Access | ~15 min | Stateless JWT | No — sent on every request as Authorization: jwt-<type> <token> |
| Refresh | ~60 days | Opaque 256-bit random | Yes — only its SHA-256 hash in UserSessions/AdminSessions |
The access token carries sub, type, and tokenVersion, with exp, iss, and aud all enforced. The refresh token is opaque and stored server-side so it can be revoked — that's the whole reason it isn't a JWT.
The access token is returned in the response body (hold it in memory, send via the header). Only the refresh token is an httpOnly cookie.
sequenceDiagram
participant C as Client
participant API as API
participant DB as Sessions
C->>API: POST /login (email, pw)
API->>DB: create session (store refresh hash)
API-->>C: access token (body) + refresh (cookie)
Note over C,API: ...later, access token expires...
C->>API: POST /refresh (refresh token)
API->>DB: find session by hash → rotate (revoke old, issue new)
API-->>C: new access + new refresh
Note over C,API: if an ALREADY-ROTATED token is replayed →
API->>DB: revoke ALL sessions + bump tokenVersion (theft!)
Login issues both tokens; refresh rotates them; replaying a rotated token triggers full revocation.
Rotation, reuse detection & instant revocation
- Rotation: every refresh revokes the presented token and issues a new one (linked via
replacedBySessionId). - Reuse detection: if an already-rotated token is replayed, that's a theft signal → revoke all the user's sessions and bump
tokenVersion. - Instant revocation:
tokenVersionis embedded in the access token and checked by the JWT strategy — bumping it (logout-all, password change, compromise) kills every outstanding access token immediately. - No raw tokens stored (only SHA-256 hashes); login runs a dummy bcrypt compare even when the account isn't found (timing-safe, prevents email enumeration).
Audience & client kind
The aud claim is per user type and client kind — web (browser/cookie) vs app (native/token, incl. desktop). The client is read from the X-Client header and stored on the session so refresh re-mints the same audience. Platform (ios/android/macos/…) is descriptive metadata on the session (X-Platform) and deliberately not part of the audience.
How req.user / req.admin get set
middleware/auth.js is driven by one registry — AUTH_TYPES = [{ scheme, strategy, reqKey }]. It reads the Authorization scheme (jwt-admin …), runs the matching Passport JWT strategy (verifies signature/exp/iss/aud + tokenVersion, loads the record), and attaches it to req.admin / req.user. No recognized scheme → next() (the controller enforces auth). Adding a new user type = one registry entry + a strategy + a session table.
One table per user type
A fundamentally different kind of user (Admin, User, Partner) with its own login portal → its own table + feature folder. Never a single Users table with a role column. A role is a variation within a type that shares the same login → a column on that type's table (and the By{Role} action suffix).
Security posture (the whole picture)
Auth is the bulk of security, so here's the full posture in one place — what protects the app, and what's still a known gap. Most of it you get for free by following the conventions; a few things are global infrastructure.
| Area | What's in place |
|---|---|
| Auth tokens | Short-lived access JWT + opaque refresh token; refresh is SHA-256-hashed at rest, rotated on every use, with reuse detection and tokenVersion instant revocation. JWT algorithm is pinned to HS256 (signer + verifier) — no algorithm-confusion. |
| Passwords | bcrypt (cost = BCRYPT_ROUNDS, currently 12). Login runs a dummy bcrypt compare when the account isn't found, so response timing can't be used to enumerate emails. Login also rejects inactive/soft-deleted accounts. |
| Session cookie | The refresh cookie is httpOnly + secure (prod) + sameSite: 'strict' — which is the CSRF defense (a cross-site form can't send it). |
| Rate limiting | Enabled in production: a global per-IP limiter on every request, plus a stricter limiter on the credential endpoints (login, refresh, password reset/change, SMS send/verify) to stop brute force and SMS-cost abuse. Redis-backed (shared across dynos) with an in-memory fallback. |
| Transport | helmet() security headers, CORS allowlist (not *) via ALLOWED_ORIGINS, forced SSL redirect in production, trust proxy for correct client IPs behind Heroku. |
| Injection | All queries go through Sequelize (parameterized) — no string-built SQL. Every action validates req.args with Joi before touching the DB. |
| Data exposure | Sensitive columns are excluded from every response by the model's defaultScope / getSensitiveData(). Queries are owner-scoped (where: { userId: req.user.id }), so one user can't read another's rows. |
| Encryption at rest | services/secure.js provides AES-256-GCM (authenticated, per-message IV) for sensitive fields that must be stored reversibly. Access / refresh / encryption use distinct secrets. |
| Request size | Body limit capped (5 MB) so a giant payload can't exhaust memory; raise it only on specific upload routes. |
500s are console.log'd with the requestId (returned in the response body — that's your correlation key) and, in production, trigger an email alert; queue failures go through queueError. A real error-monitoring service (Sentry) is a TODO — until it's wired, you're relying on logs + email, which doesn't scale. Don't treat the current setup as full observability.
The framework gives you the perimeter; each feature still has to be correct. For every action: validate req.args with Joi, scope every query to the owner, exclude sensitive fields, gate by role/type in the controller, and write a "who-cannot" test (logged-out → 401, wrong type → 401/403). Authorization gaps are how security bugs ship — §12 treats those tests as mandatory.
Testing
Tests are not optional — every action and every task has one. We test against a real Postgres + Redis (no DB mocks), with Jest + supertest.
Tests are not optional. Every action and every task has a corresponding test. No exceptions. Every ERROR_CODE in an action's JSDoc must have a test, and every role that cannot do something must have a test proving it's rejected. An action isn't finished until its tests are.
Step 1 — Write testable code first (before any test)
The most common place engineers go wrong is writing an action or task where the business logic is tangled up with database calls and third-party API calls — which makes it nearly impossible to test the logic without standing up the entire system. The fix is one rule: separate your logic from your I/O.
Every action or task does roughly three things:
- Load — fetch from the database and/or call external APIs to gather what you need.
- Process — run the actual business logic on that data.
- Save — write results back to the DB or trigger downstream side effects.
The logic in step 2 should live in its own function — in the feature's helper.js (or a clearly separated function). It takes plain inputs and returns a plain output: no DB, no API, no req. That makes it pure, so you can test it with just objects and assertions.
// ❌ BAD — logic tangled with I/O, nearly impossible to test in isolation
async function V1ProcessOrder(req) {
const order = await models.order.findByPk(req.args.id);
const charge = await stripeService.charge({ amount: order.amount });
// business logic buried inside the action:
const tax = order.amount * 0.08;
const total = order.amount + tax;
const status = charge.status === 'succeeded' ? 'paid' : 'failed';
await models.order.update({ tax, total, status, chargeId: charge.id }, { where: { id: order.id } });
return { status: 200, success: true };
}// ✅ GOOD — pure logic extracted; each piece independently testable
// helper.js — no DB, no req: just inputs → output
function calculateOrderResult(order, charge) {
const tax = order.amount * 0.08;
const total = order.amount + tax;
const status = charge.status === 'succeeded' ? 'paid' : 'failed';
return { tax, total, status, chargeId: charge.id };
}
// V1ProcessOrder.js
async function V1ProcessOrder(req) {
// 1. load
const order = await models.order.findByPk(req.args.id);
const charge = await stripeService.charge({ amount: order.amount });
// 2. process — pure logic lives in the helper, trivial to test
const result = calculateOrderResult(order, charge);
// 3. save
await models.order.update(result, { where: { id: order.id } });
return { status: 200, success: true };
}calculateOrderResult can now be tested in helper.test.js with two plain objects — every edge case (zero amounts, failed charges, rounding) with no database, no Stripe, no server. Then the integration test for V1ProcessOrder runs the happy path once, just to verify the wiring (loaded the right record, called Stripe, wrote the result). The principle: test logic exhaustively at the unit level; test integration minimally at the integration level. Running 20 logic variations through a real DB + HTTP stack is slow and re-tests Sequelize and Stripe — code that isn't yours. Trust the DB works; trust Stripe works; test your logic and verify your wiring. If you can't test a piece of logic without a running DB, that's the signal to extract it.
Step 2 — The three kinds of feature tests
Every feature folder has a tests/ directory with three things:
app/Order/tests/
├── integration/ # one file per ACTION — full HTTP via supertest, real server + DB
│ ├── V1Create.test.js
│ └── V1Query.test.js
├── tasks/ # one file per TASK — call the task fn directly, assert side effects
│ └── V1ExportTask.test.js
└── helper.test.js # pure unit tests for the feature's helper.js — no server, no DB| Location | What | How |
|---|---|---|
tests/integration/ | One per action | Real HTTP via supertest against the booted server + real test DB. |
tests/tasks/ | One per task | Call the task function directly; assert output + side effects. |
tests/helper.test.js | Pure logic | Import & call — no server, no DB, no lifecycle hooks. |
The structure above is for tests that belong to a feature. The global layer (§2) is tested separately: unit tests for global helpers/*.js live in test/helpers/, and tests for global services/*.js live in test/services/ — each named after the file under test. Those are pure unit tests (no server, no DB, no lifecycle hooks). Covered at the end of this section.
Step 3 — The integration test file, top to bottom
Every integration file follows the same strict structure: boot the server in beforeAll, reset state in beforeEach, close everything in afterAll, and group test cases by role. Between the dotenv line and let app = null, put the exact same import block every .js file uses — the standard JS file structure from §6 (built-ins → third-party → services → helpers → models → queues), just with test utilities (supertest, reset/populate, the login helpers) added. A test file is a normal file: same header, same import order.
/**
* TEST ADMIN V1Login METHOD
*/
'use strict';
// load the TEST env (note: .env.test, not .env.development)
require('dotenv').config({ path: path.join(__dirname, '../../../../config/.env.test') });
// ── then the SAME import block every .js file has (§6 order: built-ins → third-party
// → services → helpers → models → queues), e.g.: ──
const path = require('path'); // built-in
const request = require('supertest'); // third-party
const _ = require('lodash');
const queue = require('../../../../services/queue'); // services
const socket = require('../../../../services/socket');
const { errorResponse, ERROR_CODES } = require('../../../../services/error');
const { adminLogin } = require('../../../../helpers/tests'); // helpers
const { reset, populate } = require('../../../../test/fixtures/sql'); // test utilities
const models = require('../../../../models'); // models
let app = null; // server: set in beforeAll because booting is async
let AdminQueue = null; // declare the queue instances this file needs (assigned in beforeEach)
describe('Admin.V1Login', () => {
// wrap the fixture require in a function so each test gets a FRESH deep copy
// (a mutation in one test must never bleed into the next)
const adminFixFn = () => _.cloneDeep(require('../../../../test/fixtures/fix1/admin'));
let adminFix = null;
// describe the route under test
const routeVersion = '/v1';
const routePrefix = '/admins';
const routeMethod = '/login';
const routeUrl = `${routeVersion}${routePrefix}${routeMethod}`;
// boot the server ONCE for the whole file
beforeAll(async () => {
app = await require('../../../../server');
});
// before EACH test: fresh fixtures, empty queue, reset the DB
beforeEach(async () => {
adminFix = adminFixFn();
AdminQueue = queue.get('AdminQueue');
await AdminQueue.obliterate({ force: true }); // always start with an empty queue
await socket.get();
await reset(); // wipe the test database
});
// after ALL tests: close every connection or the process hangs
afterAll(async () => {
await queue.closeAll();
await socket.close();
await models.db.close();
app.close();
});
// group test cases by the ROLE making the request
describe('Role: Logged Out', () => {
beforeEach(async () => {
await populate('fix1'); // load fixture data at the narrowest scope that needs it
});
it('[logged-out] should login successfully', async () => {
const res = await request(app)
.post(routeUrl)
.send({ email: adminFix[0].email, password: adminFix[0].password });
expect(res.statusCode).toBe(201);
expect(res.body.success).toBe(true);
expect(typeof res.body.token).toBe('string');
});
it('[logged-out] should fail if credentials are incorrect', async () => {
const res = await request(app)
.post(routeUrl)
.send({ email: 'wrong@email.com', password: 'wrongpassword' });
expect(res.statusCode).toBe(400);
expect(res.body).toEqual(errorResponse(i18n, ERROR_CODES.ADMIN_BAD_REQUEST_INVALID_LOGIN_CREDENTIALS));
});
}); // END Role: Logged Out
}); // END Admin.V1Loginprocess.env
Unique to test files: you must require('path') and then load the test env with require('dotenv').config({ path: …/.env.test }) before you destructure process.env. dotenv is what populates process.env from config/.env.test — if you read const { NODE_ENV } = process.env first, the values aren't loaded yet and every env variable comes back blank/undefined. Order is always: require('path') → load the env → read the env. (Non-test files don't do this — the env is already loaded by the time they're required.)
- Always fixture as a function. Wrap the
requirein() => _.cloneDeep(require(...))and reassign inbeforeEach. Never share a fixture object between tests — one mutation silently breaks the next. - Always obliterate queues in
beforeEach. Start every test with an empty Redis queue, or leftover jobs cause false positives/negatives. - Reset the DB in the outer
beforeEach; populate at the narrowest scope. (See ordering below.) - If an action enqueues a job, assert it in the integration test — not the task test.
- Close all connections in
afterAll(queues, sockets, DB, app) or the suite hangs after completion. - Group by role with
describe(Role: Logged Out,Role: Admin, …) — makes coverage gaps obvious.
Step 4 — Fixtures & the reset/populate ordering
Fixtures are baselines, not scenarios. A fixture set (fix1, fix2) is a clean, minimal, representative starting state — not one file per test case. Don't create fix1_admin_inactive.js, fix1_admin_deleted.js, etc. — those are scenarios disguised as fixtures. Instead, load the baseline, then mutate it in the test so the reader sees exactly what's different right there in the test code.
it('[logged-out] should fail to login if account is inactive', async () => {
const admin1 = adminFix[0]; // baseline: an active admin
// mutate the baseline to the scenario you need — in the test, not in a fixture file
await models.admin.update({ isActive: false }, { where: { id: admin1.id } });
const res = await request(app).post(routeUrl).send({ email: admin1.email, password: admin1.password });
expect(res.statusCode).toBe(400);
expect(res.body).toEqual(errorResponse(i18n, ERROR_CODES.ADMIN_BAD_REQUEST_ACCOUNT_INACTIVE));
});Only create a new fixture set (fix2) when the baseline structure is genuinely different (e.g. a brand-new user vs. a user with an established history and many relationships). A different value (inactive, a role, a status) is a mutation, not a new set. Every record in a fixture file should carry a comment explaining its role in the baseline.
reset() wipes the test DB; populate('fix1') loads the baseline. Call reset() in the outer beforeEach so it runs before every test. Call populate() at the narrowest scope that needs the data: for integration tests grouped by role, put populate() inside each role's own describe → beforeEach (so a group needing different mutations isn't forced to load the default first). For task tests (no role sub-groups, always need data), a single outer beforeEach populate() is correct. The whole point: each test starts from a clean, known baseline, and mutations never leak across tests.
Fixtures live in test/fixtures/<set>/ (e.g. fix1/), one JS file per table — the test-DB cousin of dev seed data (§18).
Why fixtures load as SQL, not JavaScript (the yarn sql step)
This is the performance trick that makes the suite usable, and it's worth understanding because it runs automatically and you'll see it in the test script.
populate() runs in every test's beforeEach — potentially hundreds of times per run. If each call loaded the JavaScript fixtures and inserted them through Sequelize (the ORM), that's hooks, validations, and per-row round-trips on every single test — agonizingly slow. So instead:
yarn sql fix1runs once up front — the converter scripttest/fixtures/sql.jsreads the JS fixtures (indatabase/sequence.jsorder) and compiles them into a single flat SQL file,test/fixtures/fix1.sql, full of rawINSERTstatements. (That's literally allyarn sql <folder>does — runnode test/fixtures/sql.js; read it if you want to see the conversion.)- Then in each test's
beforeEach,populate('fix1')just executes that pre-built SQL directly against the database — no ORM, no JS parsing, no validations. One bulk insert.
Going straight to SQL instead of through the ORM on every test is dramatically faster — easily 10–100× over a full suite, because the expensive conversion happens once rather than once per test.
yarn sql by hand
yarn test runs yarn sql fix1 for you (right after yarn lang) before jest starts — so the .sql file is always regenerated from the current fixtures. You only run yarn sql fix1 manually if you've just edited a fixture and want to refresh the SQL without running the whole suite. (The generated .sql file is build output — edit the fix1/*.js fixtures, never the .sql.)
Step 5 — Testing authentication
Most actions require a logged-in user. Call the login helper from helpers/tests.js at the top of each test that needs auth — it makes a real HTTP login against the test server and returns a JWT you attach via the authorization header. The prefix (jwt-admin / jwt-user) must match the user type.
const { adminLogin, userLogin } = require('../../../../helpers/tests');
it('[admin] should update self successfully', async () => {
const admin1 = adminFix[0];
// log in first — hits the real login endpoint, returns a JWT
const { token } = await adminLogin(app, routeVersion, request, admin1);
const res = await request(app)
.post(routeUrl)
.set('authorization', `jwt-admin ${token}`) // prefix must match the type
.send({ firstName: 'New Name' });
expect(res.statusCode).toBe(200);
});When a test needs multiple authenticated users (e.g. proving User A can't read User B's data), log each in separately and keep the tokens distinct:
it('[user] should not read another user\'s private data', async () => {
const user1 = userFix[0];
const user2 = userFix[1];
const { token: token1 } = await userLogin(app, routeVersion, request, user1);
const { token: token2 } = await userLogin(app, routeVersion, request, user2);
const res = await request(app)
.get(routeUrl)
.set('authorization', `jwt-user ${token1}`)
.send({ userId: user2.id }); // user1 attempts to read user2
expect(res.statusCode).toBe(403);
});Add a new user type to the app? Add its login helper in helpers/tests.js (both adminLogin/userLogin wrap a generic login).
Step 6 — What to assert
Every test asserts at least two things: the response, and the database state.
it('[admin] should update timezone successfully', async () => {
const { token } = await adminLogin(app, routeVersion, request, admin1);
const res = await request(app)
.post(routeUrl)
.set('authorization', `jwt-admin ${token}`)
.send({ timezone: 'America/New_York' });
// 1. assert the RESPONSE — status, success flag, payload shape
expect(res.statusCode).toBe(200);
expect(res.body.success).toBe(true);
// 2. assert the DATABASE — never trust the response alone
const updated = await models.admin.findByPk(admin1.id);
expect(updated.timezone).toBe('America/New_York');
});
// 3. if the action enqueues a job, assert that too (in the integration test that triggered it)
const jobs = await AdminQueue.getJobs();
expect(jobs).toHaveLength(1);
expect(jobs[0].name).toBe('V1ExportTask');For error cases, compare the full body to the error service output so the whole contract is verified:
expect(res.statusCode).toBe(400);
expect(res.body).toEqual(errorResponse(i18n, ERROR_CODES.ADMIN_BAD_REQUEST_INVALID_TIMEZONE));- Every
ERROR_CODEhas a test. The action's JSDocErrors:list is your checklist — go through it line by line. - Test who CANNOT do it. Logged-out →
401; wrong user type →401/403. Not optional — authorization gaps are how security bugs ship. - Assert response AND DB (and the queue when relevant) — an action can return success but fail to write, or vice-versa.
- Test names describe behavior, not code:
[role] should <outcome> when <condition>. Reading the names alone should explain what the action does and doesn't allow.
Step 7 — Third-party APIs
Scenario 1 — the vendor has a sandbox: use it. Point .env.test at the sandbox and let real requests run; don't mock. Some vendors offer magic test values (our SMS service accepts '000000' as a verification code in test mode) — use them and comment clearly.
Scenario 2 — no sandbox: mock at your service wrapper, never the third-party library directly. (That's why we write services/ wrappers — a clean seam to mock at.)
// services/stripe.js is YOUR wrapper — mock the wrapper method, not the Stripe SDK
const stripeService = require('../../../../services/stripe');
describe('Order.V1Charge', () => {
let chargeStub; // declare at describe scope so hooks can reference it
beforeEach(async () => {
// ... standard reset / populate ...
chargeStub = jest.spyOn(stripeService, 'charge').mockResolvedValue({
id: 'ch_test_123',
status: 'succeeded',
amount: 5000
});
});
afterEach(() => {
jest.restoreAllMocks(); // always restore after each spyOn test
});
it('[user] should charge successfully', async () => {
const { token } = await userLogin(app, routeVersion, request, userFix[0]);
const res = await request(app).post(routeUrl).set('authorization', `jwt-user ${token}`).send({ amount: 5000 });
expect(res.statusCode).toBe(200);
const order = await models.order.findOne({ where: { userId: userFix[0].id } });
expect(order.stripeChargeId).toBe('ch_test_123');
// also assert the service was CALLED correctly
expect(chargeStub).toHaveBeenCalledTimes(1);
expect(chargeStub).toHaveBeenCalledWith(expect.objectContaining({ amount: 5000 }));
});
it('[user] should handle a failed charge gracefully', async () => {
chargeStub.mockRejectedValueOnce(new Error('Your card was declined.')); // override for THIS test only
const { token } = await userLogin(app, routeVersion, request, userFix[0]);
const res = await request(app).post(routeUrl).set('authorization', `jwt-user ${token}`).send({ amount: 5000 });
expect(res.statusCode).toBe(400);
expect(res.body).toEqual(errorResponse(i18n, ERROR_CODES.ORDER_BAD_REQUEST_PAYMENT_FAILED));
});
});jest.spyOn() vs jest.mock()
Default to jest.spyOn() — it intercepts a method on an already-loaded module and jest.restoreAllMocks() cleans it up in afterEach. Use jest.mock() only when the dependency is required at module top-level — by test time it's already baked in, so spyOn is too late; jest.mock() is hoisted before any require. With jest.mock(): declare the mock fn first, pass it into the factory, require the task after, and mockReset() it in beforeEach. And always assert the mock was called (toHaveBeenCalledWith), not just the response/DB.
Step 8 — Soft deletes & hidden rows: scope(null)
Models often have a default scope (e.g. soft-delete hides rows where deletedAt IS NOT NULL). So a plain findByPk in an assertion silently returns null for a soft-deleted record. To assert on a record a default scope would hide, bypass it with .scope(null):
// plain query — returns null if deletedAt is set (filtered by the default scope)
const user = await models.user.findByPk(userId);
// scope(null) — bypasses ALL default scopes, returns the raw row
const raw = await models.user.scope(null).findOne({ where: { id: userId } });
expect(raw.deletedAt).not.toBeNull(); // now you can assert the soft-delete happenedIf a "was it deleted?" assertion keeps failing even though the action ran correctly, a default scope is almost certainly filtering the row out — reach for scope(null).
Step 9 — Socket actions (direct invocation)
Some actions can't go through HTTP because they depend on socket context (connect/disconnect handlers). Call the action directly, passing the params and the socket context object. Label the describe 'Action Only'; everything else (lifecycle, fixtures, cleanup) is identical to an integration test.
const { V1Connect } = require('../../../../app/UserSocket/actions/V1Connect');
describe('UserSocket.V1Connect', () => {
describe('Action Only', () => {
beforeEach(async () => {
await populate('fix1');
});
it('[action-only] should connect user socket successfully', async () => {
const user1 = userFix[0];
// call the action directly, passing socket context explicitly
const result = await V1Connect(
{ userId: user1.id, socketId: 'test-socket-id' },
{
io: socket.getIO(),
SOCKET_ROOMS: socket.SOCKET_ROOMS,
SOCKET_EVENTS: socket.SOCKET_EVENTS,
socketWrapper: socket.socketWrapper
}
);
// assert the return value AND the DB side effects
expect(result).toHaveProperty('success', true);
const userInDb = await models.user.findByPk(user1.id);
expect(userInDb.isOnline).toBe(true);
});
});
});Step 10 — Socket emits
Never guard an emit with if (NODE_ENV !== 'test') — that makes the emit impossible to verify. Because actions reach io via socket.getIO() at call time (not cached at require), you can spyOn it cleanly and assert the emit chain:
const socket = require('../../../../services/socket');
describe('Message.V1Create', () => {
let mockEmit, mockTo;
beforeEach(async () => {
await populate('fix1');
// mock getIO so no real socket server is needed
mockEmit = jest.fn();
mockTo = jest.fn(() => ({ emit: mockEmit }));
jest.spyOn(socket, 'getIO').mockReturnValue({ to: mockTo });
});
afterEach(() => {
jest.restoreAllMocks();
});
it('[user] should emit MESSAGE_CREATED to the conversation room', async () => {
// ... make the HTTP request or call the action directly ...
// assert the right ROOM was targeted
expect(mockTo).toHaveBeenCalledWith(`CONVERSATION${socket.socketWrapper(conversationId)}`);
// assert the right EVENT + data shape were emitted
expect(mockEmit).toHaveBeenCalledWith(
socket.SOCKET_EVENTS.MESSAGE_CREATED,
expect.objectContaining({ message: expect.any(Object) })
);
});
});Assert both mockTo (correct room) and mockEmit (correct event + payload). Emitting to multiple rooms? Assert each call with toHaveBeenNthCalledWith / toHaveBeenCalledTimes.
Step 11 — Task tests
A task test mirrors the integration lifecycle exactly (beforeAll/beforeEach/afterAll), but instead of an HTTP request you call the task function directly and assert its output and side effects (DB row updated? email sent? socket emitted?). Task tests do not assert that the job was enqueued — that's the job of whoever triggered it (the action or another task).
describe('Contact.V1ExampleTask', () => {
// ... same beforeAll / beforeEach / afterAll lifecycle ...
it('should update the contact record after the task runs', async () => {
// call the task directly (args come from job.data)
await V1ExampleTask({ contactId: contactFix[0].id });
// assert side effects — check the DB, not the queue
const updated = await models.contact.findByPk(contactFix[0].id);
expect(updated.processed).toBe(true);
});
}); // END Contact.V1ExampleTaskGlobal helpers & services have tests too
Don't forget the global layer (§2). Tests for global helpers/*.js and services/*.js live in test/helpers/ and test/services/, named after the file under test. These are pure unit tests — no server, no DB, no lifecycle hooks; you import the function and call it. If you find yourself booting a DB or server inside one, that's a sign the function isn't pure and its I/O needs extracting.
Postgres and Redis must be up. yarn test runs the full suite — it compiles i18n + fixtures first (yarn lang + yarn sql fix1), then jest --runInBand. Run a subset with npx jest <path> --runInBand. The --runInBand matters: suites share one test DB and parallel runs race on sync({ force: true }). Add new i18n keys before running, or the yarn lang step throws and the suite won't start.
The Feature Workflow
The end-to-end process for building or changing any feature — the same ordered steps every time, whether you're adding a whole table or just a column. The next section runs a real feature through it.
Two composable paths
Building a feature is one process. Path A and Path B run the exact same steps — they differ in only two places:
| Step | Path A — new feature | Path B — modify existing |
|---|---|---|
| Scaffold | Scaffold the whole folder (yarn gen Booking) and the actions/tasks/mailers inside it. | Folder exists, so skip that one command — but you still scaffold the new actions/tasks/mailers (yarn gen Booking -a/-t/-m). |
| Migration | yarn model — creates the whole new table. | yarn migration — alters the existing table (add columns). |
Everything else is identical — and you scaffold in both paths. "Modifying doesn't scaffold" is wrong: when you add a method you scaffold the action; add a background job, you scaffold the task; add an email, you scaffold the mailer. The only thing Path B skips is the one-time whole-folder scaffold (because the folder already exists).
Most real features are a mix: one product feature might create a Bookings table (A), add a defaultBookingId column to Users (B), and add an action to the existing CalendarAccount feature (B). Plan all the schema changes together, then run the steps below for each table touched.
Step 0 — Plan (agree before scaffolding)
Design/extend the table and columns in database/schema.sql, and decide the actions/tasks. For a net-new table, get a quick sign-off on the schema + surface — that's the product/engineering boundary — then execute the rest.
The steps, in order
# SCAFFOLD — A scaffolds the whole folder; B skips that and scaffolds just the new pieces
yarn gen Booking # A only — new feature folder (singular PascalCase)
yarn gen Booking -a V1Create # A & B — scaffold each action
yarn gen Booking -a V1Query
yarn gen Booking -t V1SendConfirmationTask # A & B — scaffold each task
yarn gen Booking -m Confirmation # A & B — scaffold a mailer
# MIGRATION — A creates the table, B alters it
yarn model # A — create-table migration
yarn migration # B — add-columns / alter-table migration- Schema — design/extend the table in
database/schema.sql(§7: UUID id, FKs,is*booleans, named indexes, owner FK on every descendant). Sign off net-new tables here. - Scaffold — A:
yarn gen Booking(whole folder + adds the model tosequence.js), then scaffold its actions/tasks/mailers. B: folder exists, so scaffold only the new pieces (yarn gen Booking -a V1X/-t V1XTask/-m Mailer). Either way never hand-create — you always scaffold. - Model — fill in (A) or add fields to (B)
app/Booking/model.jsby hand fromschema.sql(§7). The test DB syncs from the model, so it must exist before tests run. - Migration — A:
yarn model→<ts>-create-Booking-model.js(creates the table). B:yarn migration→<ts>-add-cols-…-to-Bookings-tbl.js(adds columns — never drop/rename in place). Both transaction-wrapped, named indexes matching the model. - Routes — add each route in
app/Booking/routes.js(lowercase, no separators); register the feature in the globalroutes.js(A only — already registered for B). - Controller — a thin method per route, picking the action by role/device (§5).
- Actions and/or tasks — write each scaffolded action (§6) / task (§9). For
V1Queryuse the pagination/filter/sort helpers (getOffset,getOrdering,parseUrlQueryFilter); register task processors inworker.js. Add error codes (§8), i18n +yarn lang, constants, and mailers as the logic needs. - Helpers and/or services — extract pure logic into the feature's
helper.js, or the globalhelpers/if it's shared across features; write or extend a globalservices/wrapper for a third party / shared infra (§2). - Test — write a test for everything you wrote: each action/task (
tests/integration/,tests/tasks/; every error code, who-cannot — §12), feature helpers (helper.test.js), and any global helper/service (test/helpers/,test/services/). Add fixtures (test/fixtures/fix1/booking.js+yarn sql fix1). - Run —
npx jest app/Booking/tests --runInBand, then the fullyarn test.
schema → scaffold → model → migration → routes → controller → actions/tasks → helpers/services → test → run. Path A and Path B follow this same sequence; the only differences are at scaffold (A scaffolds the whole folder; B scaffolds just the new actions/tasks/mailers) and migration (A: yarn model to create the table; B: yarn migration to alter it). Model and migration come right after the scaffold so the table exists before anything builds on it.
Build a Feature: Worked Example
Let's actually build it. The product ask: "a logged-in user can book a table; on success we email them a confirmation in the background, and we remember their most recent booking on their profile." That last clause makes this a real mix: a new Bookings table (Path A) and a new column on the existing Users table (Path B). We'll walk all ten steps with real, commented code, and call out the Path-B pieces as they come up.
1. Schema — document the table in database/schema.sql
Design the table first, as documentation. (Not executed — it's the human-readable reference; the migration is what actually builds it.)
-- Bookings: a table reservation made by a user
CREATE TABLE "Bookings" (
id UUID NOT NULL DEFAULT uuid_generate_v4(), -- UUID v4 PK
"userId" UUID NOT NULL, -- FK → Users (the owner)
status BOOKINGSTATUS NOT NULL DEFAULT 'PENDING', -- ENUM type, ALL-CAPS values
"startTime" TIMESTAMPTZ NOT NULL, -- when the reservation is for (UTC)
"partySize" INTEGER NOT NULL DEFAULT 2,
"isConfirmed" BOOLEAN NOT NULL DEFAULT false, -- boolean cols start with is/has/can/does
notes TEXT NULL,
"deletedAt" TIMESTAMPTZ NULL, -- paranoid soft-delete
"createdAt" TIMESTAMPTZ NOT NULL,
"updatedAt" TIMESTAMPTZ NOT NULL,
PRIMARY KEY (id)
);
-- index every FK
CREATE INDEX "Bookings_userId_idx" ON "Bookings" ("userId");2. Scaffold — generate the folder and its pieces
yarn gen Booking # the whole feature folder (+ adds model to sequence.js)
yarn gen Booking -a V1Create # scaffold the create action
yarn gen Booking -a V1Query # scaffold the list/query action
yarn gen Booking -t V1SendConfirmationTask # scaffold the background task3. Model — app/Booking/model.js
'use strict';
// constants used by the model (ENUM values live in a constant — see §17)
const { BOOKING_STATUSES } = require('../../helpers/constants');
// fields never exposed in any query response
const sensitiveData = [];
module.exports = (sequelize, DataTypes) => {
const Booking = sequelize.define('booking', {
id: {
type: DataTypes.UUID,
allowNull: false,
defaultValue: DataTypes.UUIDV4,
primaryKey: true,
validate: {
isUUID: 4
}
},
// foreign keys are declared in associate(), not here
status: {
type: DataTypes.ENUM(...BOOKING_STATUSES),
allowNull: false,
defaultValue: 'PENDING'
},
startTime: {
type: DataTypes.DATE,
allowNull: false
},
partySize: {
type: DataTypes.INTEGER,
allowNull: false,
defaultValue: 2
},
isConfirmed: {
type: DataTypes.BOOLEAN,
allowNull: false,
defaultValue: false
},
notes: {
type: DataTypes.TEXT,
allowNull: true,
defaultValue: null
}
}, {
timestamps: true,
paranoid: true, // soft delete
freezeTableName: true,
tableName: 'Bookings',
defaultScope: {
attributes: {
exclude: sensitiveData
}
},
indexes: [
{ name: 'Bookings_userId_idx', fields: ['userId'] } // index every FK
]
});
// the foreign key section
Booking.associate = models => {
Booking.belongsTo(models.user, {
as: 'user',
foreignKey: {
name: 'userId',
allowNull: false
},
onDelete: 'CASCADE',
onUpdate: 'CASCADE'
});
};
Booking.getSensitiveData = () => sensitiveData;
return Booking;
};4. Migration — create the table
yarn model generates the skeleton; rename it to the convention and fill it in by hand so it matches the model exactly.
// migrations/<ts>-create-Booking-model.js
'use strict';
module.exports = {
up(queryInterface, Sequelize) {
return queryInterface.sequelize.transaction(async t => {
await queryInterface.createTable('Bookings', {
id: { type: Sequelize.UUID, defaultValue: Sequelize.UUIDV4, primaryKey: true, allowNull: false },
userId: {
type: Sequelize.UUID,
allowNull: false,
references: { model: 'Users', key: 'id' },
onDelete: 'CASCADE',
onUpdate: 'CASCADE'
},
status: { type: Sequelize.ENUM('PENDING', 'CONFIRMED', 'CANCELLED'), allowNull: false, defaultValue: 'PENDING' },
startTime: { type: Sequelize.DATE, allowNull: false },
partySize: { type: Sequelize.INTEGER, allowNull: false, defaultValue: 2 },
isConfirmed: { type: Sequelize.BOOLEAN, allowNull: false, defaultValue: false },
notes: { type: Sequelize.TEXT, allowNull: true, defaultValue: null },
deletedAt: { type: Sequelize.DATE },
createdAt: { type: Sequelize.DATE, allowNull: false },
updatedAt: { type: Sequelize.DATE, allowNull: false }
}, { transaction: t });
// named index, matching the model exactly
await queryInterface.addIndex('Bookings', ['userId'], { name: 'Bookings_userId_idx', transaction: t });
});
},
down(queryInterface) {
return queryInterface.sequelize.transaction(async t => {
await queryInterface.dropTable('Bookings', { transaction: t });
});
}
};5. Routes — app/Booking/routes.js + register globally
// app/Booking/routes.js — lowercase, no separators
module.exports = (passport, router) => {
const controller = require('./controller');
router.all('/v1/bookings/create', controller.V1Create);
router.all('/v1/bookings/query', controller.V1Query);
return router;
};
// routes.js (global) — register the feature once
// app.use('/', require('./app/Booking/routes')(passport, express.Router()));6. Controller — app/Booking/controller.js
'use strict';
const { errorResponse, ERROR_CODES } = require('../../services/error');
const actions = require('./actions');
module.exports = {
V1Create,
V1Query
};
// pick the action by role; only a logged-in user can create a booking
async function V1Create(req, res, next) {
let method = null;
if (req.user) {
method = 'V1CreateByUser';
} else {
return res.status(401).json(errorResponse(req, ERROR_CODES.UNAUTHORIZED));
}
try {
const result = await actions[method](req, res);
return res.status(result.status).json(result);
} catch (error) {
return next(error);
}
} // END V1Create
async function V1Query(req, res, next) {
let method = null;
if (req.user) {
method = 'V1QueryByUser';
} else {
return res.status(401).json(errorResponse(req, ERROR_CODES.UNAUTHORIZED));
}
try {
const result = await actions[method](req, res);
return res.status(result.status).json(result);
} catch (error) {
return next(error);
}
} // END V1Query7. Actions and the task
The create action: validate → transaction → pure helper → commit → enqueue the confirmation task → respond 201. (Header/imports trimmed for focus — they follow §6.)
// app/Booking/actions/V1Create.js
const queue = require('../../../services/queue');
const helper = require('../helper'); // feature-local pure logic (step 8)
// queues — grabbed right after models
const BookingQueue = queue.get('BookingQueue');
module.exports = {
V1CreateByUser
};
/**
* Create a booking
*
* POST /v1/bookings/create
* Must be logged in · Roles: ['user']
*
* req.args = {
* @startTime - (STRING - REQUIRED): ISO datetime of the reservation
* @partySize - (NUMBER - OPTIONAL) [DEFAULT 2]
* @notes - (STRING - OPTIONAL)
* }
* Success: 201, returns the booking
* Errors:
* 400: BAD_REQUEST_INVALID_ARGUMENTS
* 400: BOOKING_BAD_REQUEST_TIME_IN_PAST
*/
async function V1CreateByUser(req, res) {
const schema = joi.object({
startTime: joi.date().iso().required(),
partySize: joi.number().integer().min(1).max(20).default(2),
notes: joi.string().trim().allow(null).optional()
});
const { error, value } = schema.validate(req.args);
if (error) {
return errorResponse(req, ERROR_CODES.BAD_REQUEST_INVALID_ARGUMENTS, joiErrorsMessage(error));
}
req.args = value;
// pure business rule lives in the helper — unit-tested without a DB (step 8)
if (!helper.isFutureBooking(req.args.startTime)) {
return errorResponse(req, ERROR_CODES.BOOKING_BAD_REQUEST_TIME_IN_PAST);
}
const t = await models.db.transaction();
try {
const booking = await models.booking.create({
userId: req.user.id, // owner FK comes from the authenticated user
startTime: req.args.startTime,
partySize: req.args.partySize,
notes: req.args.notes
}, { transaction: t });
// Path-B touch: stamp the user's most recent booking (column added to Users below).
// Same transaction, so the booking and the stamp commit together or not at all.
await models.user.update(
{ mostRecentBookingId: booking.id },
{ where: { id: req.user.id }, transaction: t }
);
await t.commit(); // commit BEFORE enqueueing
// hand off the email to the background worker
await BookingQueue.add('V1SendConfirmationTask', {
bookingId: booking.id
});
return {
status: 201,
success: true,
booking: booking.dataValues
};
} catch (error) {
await t.rollback();
throw error;
}
} // END V1CreateByUserThe task: load the booking, send the email through a service, flip isConfirmed, push a socket event. On failure it throws.
// app/Booking/tasks/V1SendConfirmationTask.js
const notify = require('../../../services/notify'); // fake global service (step 8)
module.exports = {
V1SendConfirmationTask
};
/**
* @job.data = { @bookingId - (STRING - REQUIRED) }
* Success: returns true
*/
async function V1SendConfirmationTask(job) {
const schema = joi.object({
bookingId: joi.string().uuid().required()
});
const { error, value } = schema.validate(job.data);
if (error) {
throw new Error(joiErrorsMessage(error)); // tasks throw
}
job.data = value;
try {
const booking = await models.booking.findByPk(job.data.bookingId);
// call the (fake) notification service wrapper
await notify.sendBookingConfirmation(booking);
await booking.update({ isConfirmed: true });
// tell connected clients in real time (emit after the write)
socket.getIO()
.to(`${SOCKET_ROOMS.USER}${socketWrapper(booking.userId)}`)
.emit(SOCKET_EVENTS.BOOKING_CONFIRMED, { id: booking.id });
return true;
} catch (error) {
throw error;
}
} // END V1SendConfirmationTask8. Helpers and services (the fake ones)
Pure business logic goes in the feature's helper.js — no DB, no req, so it's trivially unit-testable. The external integration goes behind a global service wrapper, so tests have a clean seam to mock.
// app/Booking/helper.js — pure feature-local logic
'use strict';
module.exports = {
isFutureBooking
};
// returns true if the booking time is in the future (plain in → plain out, no I/O)
function isFutureBooking(startTime) {
return new Date(startTime).getTime() > Date.now();
} // END isFutureBooking// services/notify.js — a GLOBAL service: our wrapper around a (fake) email/SMS provider.
// Lives in services/ because it wraps a third party and is shared infra. Tests mock THIS,
// never the underlying SDK.
'use strict';
module.exports = {
sendBookingConfirmation
};
async function sendBookingConfirmation(booking) {
// pretend this calls SendGrid / Twilio / etc.
return fakeProvider.send({
template: 'booking-confirmation',
to: booking.userId,
data: { startTime: booking.startTime }
});
} // END sendBookingConfirmationAlong the way you'd also add the error code BOOKING_BAD_REQUEST_TIME_IN_PAST (§8), its i18n string + yarn lang, and the BOOKING_STATUSES constant (§17).
9. Test — one per action/task, plus the helper
// app/Booking/tests/integration/V1Create.test.js (lifecycle trimmed — see §12)
describe('Booking.V1Create', () => {
describe('Role: User', () => {
beforeEach(async () => {
await populate('fix1');
});
it('[user] should create a booking and enqueue the confirmation', async () => {
const { token } = await userLogin(app, routeVersion, request, userFix[0]);
const res = await request(app)
.post(routeUrl)
.set('authorization', `jwt-user ${token}`)
.send({ startTime: '2999-01-01T19:00:00Z', partySize: 4 });
// 1. assert the response
expect(res.statusCode).toBe(201);
expect(res.body.booking).toHaveProperty('partySize', 4);
// 2. assert the database — the booking row AND the Path-B stamp on the user
const booking = await models.booking.findByPk(res.body.booking.id);
expect(booking.userId).toBe(userFix[0].id);
const user = await models.user.findByPk(userFix[0].id);
expect(user.mostRecentBookingId).toBe(booking.id); // the Path-B column got set
// 3. assert the job was enqueued
const jobs = await BookingQueue.getJobs();
expect(jobs).toHaveLength(1);
expect(jobs[0].name).toBe('V1SendConfirmationTask');
});
it('[user] should reject a booking in the past', async () => {
const { token } = await userLogin(app, routeVersion, request, userFix[0]);
const res = await request(app)
.post(routeUrl)
.set('authorization', `jwt-user ${token}`)
.send({ startTime: '2000-01-01T19:00:00Z' });
expect(res.statusCode).toBe(400);
expect(res.body).toEqual(errorResponse(i18n, ERROR_CODES.BOOKING_BAD_REQUEST_TIME_IN_PAST));
});
});
// who-cannot: logged-out → 401
describe('Role: Logged Out', () => {
beforeEach(async () => { await populate('fix1'); });
it('[logged-out] should fail to create a booking', async () => {
const res = await request(app).post(routeUrl).send({ startTime: '2999-01-01T19:00:00Z' });
expect(res.statusCode).toBe(401);
});
});
});// app/Booking/tests/helper.test.js — pure unit test, no server/DB
const { isFutureBooking } = require('../helper');
describe('Booking helper.isFutureBooking', () => {
it('returns true for a future time', () => {
expect(isFutureBooking('2999-01-01T00:00:00Z')).toBe(true);
});
it('returns false for a past time', () => {
expect(isFutureBooking('2000-01-01T00:00:00Z')).toBe(false);
});
});10. Run
yarn migrate # apply the create-table migration to your dev DB
npx jest app/Booking/tests --runInBand # just this feature
yarn test # the full suite (compiles i18n + fixtures first)The Path-B half: add mostRecentBookingId to Users
The action above writes user.mostRecentBookingId — but that column doesn't exist yet, and we also want an endpoint to read it back. Adding both is a Path-B change to the existing User feature: same steps, but skip the whole-folder scaffold (it exists) and use yarn migration to ALTER instead of yarn model to create. We'll do the full slice: schema → scaffold the new action → model → migration → route → controller → action → test.
Schema — add the column to the Users block in schema.sql:
-- inside the existing "Users" table
"mostRecentBookingId" UUID NULL, -- FK → Bookings; the user's latest booking (nullable)Model — add the FK to app/User/model.js's associate() (not the attributes block):
// app/User/model.js — inside User.associate = models => { ... }
User.belongsTo(models.booking, {
as: 'mostRecentBooking',
foreignKey: {
name: 'mostRecentBookingId',
allowNull: true // nullable — a brand-new user has no bookings yet
},
onDelete: 'SET NULL', // if that booking is deleted, just clear the pointer
onUpdate: 'CASCADE'
});Migration — ALTER, not create (yarn migration → rename to the convention). Add the column; never drop/rename in place:
// migrations/<ts>-add-cols-mostRecentBookingId-to-Users-tbl.js
'use strict';
module.exports = {
up(queryInterface, Sequelize) {
return queryInterface.sequelize.transaction(async t => {
await queryInterface.addColumn('Users', 'mostRecentBookingId', {
type: Sequelize.UUID,
allowNull: true,
references: { model: 'Bookings', key: 'id' },
onDelete: 'SET NULL',
onUpdate: 'CASCADE'
}, { transaction: t });
await queryInterface.addIndex('Users', ['mostRecentBookingId'], {
name: 'Users_mostRecentBookingId_idx',
transaction: t
});
});
},
down(queryInterface) {
return queryInterface.sequelize.transaction(async t => {
await queryInterface.removeColumn('Users', 'mostRecentBookingId', { transaction: t });
});
}
};This is a real FK dependency: Bookings must exist before Users can reference it. Run the create-Bookings migration first, then this add-column migration — migrations apply in timestamp order, so the Booking one (created earlier in the flow) naturally lands first.
Scaffold the new action on the existing User feature (folder already exists, so just the action):
yarn gen User -a V1ReadMostRecentBooking # scaffolds the action + its testRoute — add it to app/User/routes.js (lowercase, no separators):
router.all('/v1/users/readmostrecentbooking', controller.V1ReadMostRecentBooking);Controller — a thin method in app/User/controller.js:
async function V1ReadMostRecentBooking(req, res, next) {
let method = null;
if (req.user) {
method = 'V1ReadMostRecentBookingByUser';
} else {
return res.status(401).json(errorResponse(req, ERROR_CODES.UNAUTHORIZED));
}
try {
const result = await actions[method](req, res);
return res.status(result.status).json(result);
} catch (error) {
return next(error);
}
} // END V1ReadMostRecentBookingAction — read the booking the user's mostRecentBookingId points to. No req.args to validate (it's just "my latest"); scope to the caller and 404 if they have none:
// app/User/actions/V1ReadMostRecentBooking.js
module.exports = {
V1ReadMostRecentBookingByUser
};
/**
* Read the logged-in user's most recent booking
*
* GET/POST /v1/users/readmostrecentbooking
* Must be logged in · Roles: ['user']
*
* req.args = {} (none — it's always "my latest")
* Success: 200, returns the booking
* Errors:
* 404: USER_NOT_FOUND_NO_RECENT_BOOKING
*/
async function V1ReadMostRecentBookingByUser(req, res) {
try {
// the logged-in user (req.user) already carries the pointer column
if (!req.user.mostRecentBookingId) {
return errorResponse(req, ERROR_CODES.USER_NOT_FOUND_NO_RECENT_BOOKING);
}
// fetch the booking, scoped to the owner so a user can only read their own
const booking = await models.booking.findOne({
where: {
id: req.user.mostRecentBookingId,
userId: req.user.id
}
});
if (!booking) {
return errorResponse(req, ERROR_CODES.USER_NOT_FOUND_NO_RECENT_BOOKING);
}
return {
status: 200,
success: true,
booking: booking.dataValues
};
} catch (error) {
throw error;
}
} // END V1ReadMostRecentBookingByUser(That introduces one error code — USER_NOT_FOUND_NO_RECENT_BOOKING — in app/User/error.js plus its i18n string, exactly as in §8.)
Test — happy path, the 404, and who-cannot:
// app/User/tests/integration/V1ReadMostRecentBooking.test.js (lifecycle trimmed — see §12)
describe('User.V1ReadMostRecentBooking', () => {
describe('Role: User', () => {
beforeEach(async () => {
await populate('fix1');
});
it('[user] should return the user\'s most recent booking', async () => {
const user1 = userFix[0];
const { token } = await userLogin(app, routeVersion, request, user1);
// arrange: create a booking and point the user at it (mutate the baseline in-test)
const booking = await models.booking.create({
userId: user1.id,
startTime: '2999-01-01T19:00:00Z',
partySize: 2
});
await models.user.update({ mostRecentBookingId: booking.id }, { where: { id: user1.id } });
const res = await request(app)
.post(routeUrl)
.set('authorization', `jwt-user ${token}`);
expect(res.statusCode).toBe(200);
expect(res.body.booking).toHaveProperty('id', booking.id);
});
it('[user] should 404 when the user has no recent booking', async () => {
const { token } = await userLogin(app, routeVersion, request, userFix[0]); // baseline: no booking
const res = await request(app)
.post(routeUrl)
.set('authorization', `jwt-user ${token}`);
expect(res.statusCode).toBe(404);
expect(res.body).toEqual(errorResponse(i18n, ERROR_CODES.USER_NOT_FOUND_NO_RECENT_BOOKING));
});
});
// who-cannot: logged-out → 401
describe('Role: Logged Out', () => {
beforeEach(async () => { await populate('fix1'); });
it('[logged-out] should be rejected', async () => {
const res = await request(app).post(routeUrl);
expect(res.statusCode).toBe(401);
});
});
});That's the full mix, both halves end to end: a new table + folder with two actions, a task, a pure helper, and a service wrapper (Path A), plus a new FK column and a new read action on an existing table (Path B). Both were planned together in Step 0 and built in one pass, each running the same ordered steps — Path B just skipped the folder scaffold and used yarn migration to ALTER.
Before calling it done, run through the conventions checklist (the review-conventions skill, §15) against your changed files — naming, response shape, error codes tested, owner-scoping, named indexes, yarn lang run.
Working with AI Agents
This repo is built to be developed with AI agents (Claude Code, Cursor, etc.). The conventions are committed so any agent picks them up on clone.
Where the agent instructions live
| File | Role |
|---|---|
AGENTS.md | Canonical, tool-agnostic agent guide (Cursor/Codex/Copilot read this). Philosophy, golden rules, command cheat-sheet, doc map, skills index. |
CLAUDE.md | Claude Code's entry point — imports AGENTS.md + Claude-specific notes. Auto-loaded every session. |
.claude/skills/<name>/SKILL.md | On-demand playbooks for specific tasks. Claude auto-discovers and invokes them by description; other tools open them by path. |
docs/conventions.txt | The detailed, authoritative rulebook the above point to. |
The skills (playbooks)
Each is a step-by-step procedure that references the conventions: create-feature, modify-feature, add-action, add-query-action, add-task, add-migration, add-auth-user-type, add-error-code, add-constant, add-fixtures, add-mailer, add-cronjob, add-socket-event, add-locale, write-tests, review-conventions, sync-docs, setup-and-ops. When a task matches one, the agent follows that playbook.
Documentation stays in sync (enforced)
The same conventions are written in several places for different audiences — this page, README.md, docs/conventions.txt, AGENTS.md, CLAUDE.md, database/schema.sql, and the skills. They must agree. The sync-docs skill defines the procedure (name the change → grep every surface → update the ones that cover it, including the relevant skill → report what was synced), and a committed PostToolUse hook (.claude/hooks/doc-sync-reminder.py) fires on every documentation edit to remind the agent. A doc change isn't done until the parallel surfaces match.
The key principle: awareness vs enforcement
Prose (README, conventions) raises the probability the agent follows the rules — it's not a guarantee. The reliable path is to push conventions down to deterministic layers: the generator (correct by construction), then lint + tests + CI (the build fails if the agent drifts). Awareness is probabilistic; enforcement is deterministic.
The bridge until full lint/CI exists is the review-conventions skill — an explicit self-audit the agent runs against its own changes before declaring work done.
How to drive it
Describe the feature; the agent plans the schema + surface with you, gets a quick sign-off on net-new tables, then builds it end-to-end (routes, controller, actions, tasks, i18n, errors, model, migration, tests) and self-audits. The skills keep index.js/sequence.js correct and the conventions applied.
API Endpoint Reference
Every endpoint is one action, and the action's JSDoc header is its contract (route, auth, req.args, success, error codes). This section catalogs them; when in doubt, the action file is the source of truth.
Conventions for every endpoint
- Routes are
/v1/<plural>/<action>, all lowercase, no separators; methods are POST/GET only. - Auth via header:
Authorization: jwt-user <token>orjwt-admin <token>. - Success is a flat object
{ status, success: true, … }; errors are{ success: false, status, error, message }.
Auth surface (worked example)
The authentication endpoints, identical in shape for each user type (/v1/users/* and /v1/admins/*):
| Endpoint | Auth | req.args | Returns |
|---|---|---|---|
POST /v1/<type>/login | Logged out | email, password | 201 · token, refreshToken, <type> (sets refresh cookie) |
POST /v1/<type>/refresh | Public (refresh token) | refreshToken (or cookie) | 200 · new token + refreshToken |
POST /v1/<type>/logout | Logged in | refreshToken (or cookie) | 200 · revokes current session |
POST /v1/<type>/logoutall | Logged in | — | 200 · revokes all sessions, bumps tokenVersion |
Plus user-specific flows like /v1/users/smssendcode, /v1/users/smsverifycode, /v1/users/register, /v1/users/read, and admin flows like /v1/admins/read, /v1/admins/create, /v1/admins/resetpassword. Each follows the same documented contract.
Because each action's JSDoc is structured (route + req.args + Success + Errors), this catalog can be generated from the action files. Until that's automated, treat the action JSDoc as canonical and update this table when endpoints change.
Conventions Quick-Reference
The rulebook in one place. The authoritative source is docs/conventions.txt; this is the scannable version.
Naming
| Feature folder | singular PascalCase — Order |
| Action / task | V{version}{Action}[By{Role}][On{Device}]; tasks append Task |
| Controller | plural feature, version+action only (no role/device) |
| Table / column | PascalCase plural / camelCase |
| Boolean column | starts with is/has/can/does |
| FK column | <entity>Id → PascalCase plural table |
| Constant | ALL_CAPS_WITH_UNDERSCORES; dual-export object + array |
| i18n key | NAMESPACE[snake_case] |
| Error code | key NAMESPACE_STATUS_DESC; .error NAMESPACE.STATUS_DESC |
| Index | {Table}_{col}_{idx|unique} |
File structure (every .js)
Top to bottom, every file follows this order:
- Header comment
'use strict'- Env (
const { X } = process.env) - Built-in node modules
- Third-party modules
- Services
- Helpers
- Models
- Queues — the
queue.get('XQueue')instances (right after models) - Module-level constants
module.exports— listing the methods, before their definitions- The method definitions
Two more rules:
- Order imports within each group by increasing length; plain
requires before destructured ones. - Close every function with
// END <name>.
HTTP & responses
| Methods | POST & GET only |
| Route URL | lowercase, no separators — /v1/users/logoutall (mirror action name; not logout_all/-all/camelCase) |
| Args | req.args (never req.body/req.query) |
| Response | flat { status, success: true, … } — no data nesting |
| Status | 200 default · 201 create · 202 background-job handoff |
| Errors | HTTP actions errorResponse(req, …); tasks/socket throw; never manual 500 |
Data
- UUID v4 primary keys.
paranoid: true(soft delete) — usescope(null)to bypass.- Foreign keys declared in
associate, each with explicitonDelete/onUpdate. - Index every foreign key.
- Carry the owner FK down to every descendant, protected by a composite FK.
- Migrations are transaction-wrapped; never drop or rename in place.
- All times stored in UTC.
Workflow
- Always
yarn gen— never hand-create files. - Exact-version installs only (
--exact). - Run
yarn langafter any i18n change. - Run tests with
--runInBand(Postgres + Redis must be up). - Every action and task has a test; every error code is tested; test who cannot do something.
Operations & Deploy
Running, maintaining, and shipping the app.
Database operations
yarn migrate # run pending migrations on the dev DB
yarn mg # alias of `yarn migrate`
yarn migrate:prod # run pending migrations on production
yarn rollback # undo the last migration on the dev DB
yarn rb # alias of `yarn rollback`
yarn rollback:prod # undo the last migration on production
yarn model # generate a CREATE-table migration skeleton (new table), then rename
yarn migration # generate an ALTER-table migration skeleton (add columns), then rename
yarn backup # dump the dev DB to database/backups/backup.sql
yarn restore # drop + recreate the dev DB from that backup (DESTRUCTIVE)
yarn seed # load database/seed/ into the dev DB (local sample data — see below)Seed data for local development (yarn seed)
Seed data is sample data for your development database — a handful of representative admins, users, and records so the app isn't empty when you run it locally. It is the dev-time cousin of test fixtures (§12): same idea, different target. Seed data populates the DB you click around in by hand; fixtures populate the throwaway test DB.
- It lives in
database/seed/, organized into sets (set1/, …) with one file per table (admin.js,user.js, …) — mirroring the table structure. yarn seedloads it into the dev DB. Insert order followsdatabase/sequence.js, so parents land before children (FK order).- It's optional and local-only — purely to make development pleasant. It never runs in production.
database/seed/ → yarn seed → your dev DB, loaded once, to develop against. test/fixtures/ → the test DB, reloaded before every test as a clean baseline (§12). Same shape (one file per table, sequence-ordered); different lifecycle and target.
i18n / email / fixtures
yarn lang # compile languages/*.js → locales/*.json and validate keys
yarn gulp # watch mailers/languages and regenerate preview.html / locales
yarn sql fix1 # (re)generate the test fixtures' .sql file — usually run for you by `yarn test`Testing inbound webhooks locally (yarn ngrok)
Some third parties (Stripe, Google/Microsoft calendar push, Twilio, …) deliver events by calling a webhook URL on your server. That's impossible against localhost — the provider can't reach your machine. ngrok solves it by opening a public tunnel to your local port, giving you a temporary public URL you register with the provider; their webhook calls then flow straight to your locally-running server, so you can develop and debug the handler with real payloads.
yarn ngrok:auth <token> # one-time: set your ngrok auth token
yarn ngrok # open a public tunnel to the local server (port 8000)yarn ngrokprints a publichttps://…ngrok…URL. Register<that-url>/v1/<feature>/<webhook-action>with the third party.- The URL changes each run (on the free tier) — re-register when it does.
- This is for the inbound direction (provider → you). For testing your code that calls out to a third party, use their sandbox or mock the service wrapper (§12).
The gulpfile
yarn gulp runs a watcher that regenerates mailer preview.html files when an index.ejs changes, and recompiles locales/ when a language file changes.
Health & readiness probes
Two infrastructure endpoints, defined in routes.js alongside the other top-level routes (/, /socket) — the convention is that global routes live there, and routes.js is mounted near the end of server.js. They're deliberately not features — just two routes. Because they sit behind the middleware chain, they pass through it like any request — with one deliberate exception: the global rate limiter has a skip for /health and /ready, so probes are never throttled (otherwise a shared egress IP exhausting the budget could 429 a probe and trick the orchestrator into restarting a healthy dyno). During graceful drain middleware/exit.js answers them with 503 for free (it runs before routes).
| Endpoint | Answers | Checks |
|---|---|---|
GET /health (liveness) | "Is the process up?" — the platform uses it to decide whether to restart the dyno. | Nothing — cheap, no dependencies. Stays 200 even during graceful drain (the process is alive). |
GET /ready (readiness) | "Can I serve traffic right now?" — the load balancer uses it to decide whether to route to this instance. | models.db.authenticate() + a Redis ping. Returns 503 if a dependency is down. During shutdown the exit middleware already 503s every request (including this one) so the LB drains us — no extra drain check needed in the handler. |
An instance can be alive but not ready (DB still connecting, or draining mid-deploy) — you want the LB to stop routing without the platform killing the process. Heroku doesn't use readiness probes the way k8s does (it routes by port-binding and restarts on crash), so on Heroku /health is mainly for your uptime monitor and /ready for your own dashboards — but both become essential the day you move to k8s/ECS, and they're cheap insurance now.
Deploy (Heroku)
- Create the Heroku app; add the Heroku Postgres and Heroku Redis add-ons.
- Sync config vars:
node ./config/heroku-sync .env.production <app-name> true(or set them manually). - Connect the GitHub
mainbranch and deploy. TheProcfiledeclares the web, worker, and clock processes; migrations run automatically on deploy.
Locally, Postgres + project-local Redis (yarn redis). In production, both are managed add-ons reached via config vars (DATABASE_URL, REDIS_URL) — you never build/run Redis from the repo on a deploy. Run exactly one clock dyno.
Appendix
Troubleshooting — common gotchas
The things that trip up almost everyone in their first week, and the one-line fix:
| Symptom | Cause & fix |
|---|---|
| Tests hang and never exit after passing | Connections weren't closed. Every test file needs afterAll closing the queue, socket, DB, and app (§12). |
| Test suite won't even start; error about a missing i18n key | You used a .__('KEY') that isn't defined. Add it to languages/*.js and run yarn lang — yarn test runs it first and fails fast if a key is missing (§8). |
Env variables are undefined in a test | You read process.env before loading dotenv. In test files, require('dotenv').config(...) must run before you destructure process.env (§12). |
| A job is enqueued but nothing happens | The worker isn't running. Background jobs only run while yarn w is up (and Redis). The web server enqueues; the worker processes (§9). |
| A scheduled cronjob never fires | The clock process isn't running — start yarn cron (and yarn w to process what it enqueues). In prod, ensure exactly one clock dyno (§9). |
A row you know exists comes back null in a query/assertion | A default scope (usually soft-delete) is hiding it. Use Model.scope(null) to bypass (§7/§12). |
| Parallel test suites fail intermittently | Run with --runInBand — suites share one test DB and race on sync({ force: true }) otherwise (§12). |
401 on a request you expected to be authed | The Authorization header prefix must match the type: jwt-user / jwt-admin (§11). |
| Connection refused on boot | Postgres and/or Redis aren't running. Both are required for the app and the test suite (§3). |
| i18n / locale changes don't show up | You edited locales/*.json directly — those are compiled output. Edit languages/*.js and run yarn lang (§8). |
Glossary
| Term | Meaning |
|---|---|
| Feature folder | Everything for one table, under app/<Feature>/. |
| Action | A real-time HTTP handler; returns a response immediately. |
| Task | A background job run by the worker off a Redis queue. |
| Aggregate / singular task | Fan-out pattern: aggregate enqueues one singular task per record. |
| Fixture | Baseline test data (test/fixtures/fix1/); mutate in-test for scenarios. |
| Seed | Baseline data for the dev DB (database/seed/). |
| scope(null) | Bypass default scopes to see soft-deleted / hidden rows. |
| paranoid | Soft delete — destroy() sets deletedAt instead of deleting. |
| tokenVersion | Counter in the access token; bumping it invalidates all of a user's tokens. |
| Audience / client kind | aud = user type × web/app; the token's security scope. |
| User type vs role | Type = own table + login (Admin/User); role = column within a type. |
Command cheat-sheet
Scaffold (always use the generator — never hand-create):
yarn gen <Feature> # whole feature folder
yarn gen <Feature> -a V1<Action> # add an action
yarn gen <Feature> -t V1<Action>Task # add a background task
yarn gen <Feature> -m <Mailer> # add a mailer
yarn del <Feature> [-a|-t|-m ...] # remove any of the aboveRun the three processes (each has a short alias and a long one):
yarn s # web/API server (dev: nodemon + .env.development)
yarn server # web server via the Procfile (= yarn start)
yarn w # background worker
yarn worker # same worker (long form)
yarn cron # clock process — runs the cronjobs
yarn redis # start the project-local Redis (separate terminal)
yarn redis:stop # stop the project-local RedisDatabase:
yarn migrate # run pending migrations on the dev DB
yarn mg # alias of `yarn migrate`
yarn migrate:prod # run pending migrations on production
yarn rollback # undo the last migration on the dev DB
yarn rb # alias of `yarn rollback`
yarn rollback:prod # undo the last migration on production
yarn model # generate a CREATE-table migration skeleton (new table), then rename
yarn migration # generate an ALTER-table migration skeleton (add columns), then rename
yarn backup # dump the dev DB to database/backups/backup.sql
yarn restore # drop + recreate the dev DB from that backup (DESTRUCTIVE — wipes current dev data)
yarn seed # load database/seed/ data into the dev DB (your local sample data)i18n & tests:
yarn lang # compile languages/*.js → locales/*.json and validate keys
yarn test # full suite (runs `yarn lang` + `yarn sql fix1` first, then jest --runInBand)
yarn t # alias of `yarn test`
npx jest <path> --runInBand # run a single file / subset
yarn sql fix1 # (re)generate the test fixtures' .sql file — usually run for you by `yarn test`Other tooling:
yarn gulp # watch mailers/languages → regenerate preview.html / locales
yarn ngrok # expose the local server publicly (for testing inbound webhooks)
yarn ngrok:auth <token> # one-time: set your ngrok auth tokenRepo map
app/<Feature>/ feature folders (model, routes, controller, actions, tasks, tests, …)
middleware/ id, args, auth, error, exit
services/ email, error, language, socket, passport, queue, redis, …
helpers/ constants, cruqd, logic, schemas, session, tests
database/ index.js, schema.sql, seed/, sequence.js
migrations/ timestamped migration files
config/ .env.* (gitignored) + .env.template + config.js
locales/ compiled i18n (generated — don't edit)
redis/ project-local Redis build (gitignored)
index.js · server.js · worker.js · cronjobs.js # the entry points
AGENTS.md · CLAUDE.md · .claude/skills/ # agent docs
README.md · docs/ · documentation.html # human docsYou now understand Orbital-Express end-to-end — the processes, the feature folder, the request lifecycle, the data layer, errors/i18n, background jobs, sockets, auth, and testing — and how to build a feature with all of it. The deep references are README.md and docs/conventions.txt; the playbooks are .claude/skills/. Welcome aboard. ↑ Back to top