Testing strategy — AI-first (backend-pms-dev)

TL;DR

Defer implementation until requirements stabilize and token budget allows. When ready: no Cucumber/QA ceremony; NestJS Swagger (decorators + exported swagger.json) is the live API contract; Jest specs (unit + HTTP) are AI-generated and maintained via scaffold scripts, agent rules, and a route-coverage gate. Vault openapi.yaml stays planning only.

Why deferred (2026-05-22)

• Token cost — harness + specs across modules is a large agent pass; requirements still moving (masters, project aggregate, OpenAPI parity).
• Requirement churn — tests written now would be rewritten often; better to lock strategy in vault first, implement when API surface stabilizes.

Implementation tracker: [[Projects/anabatic-project-manager/backlog/p2-ai-first-testing-harness|p2 — AI-first testing harness]].

Concepts confirmed

BDD without QA: Good practice when it means behavior-focused automated tests. Poor when it means Cucumber workshops and living .feature files the team does not use.

Current state (backend-pms-dev)

• Jest 28 + @nestjs/testing + supertest configured in package.json.
• No *.spec.ts under src/ → npm run test exits “No tests found”.
• Stale test/app.e2e-spec.ts (Hello World) — fails (path aliases, no real route).
• No CI test workflow in repo.
• Sibling contract portal uses Jest unit specs (e.g. SLA helpers) — useful pattern, not Cucumber.

Target architecture

1. Implement feature with full Swagger decorators (@ApiOperation, @ApiProperty + example).
2. Export OpenAPI artifact (openapi:export).
3. AI adds/updates specs beside module (template).
4. test:routes fails if a swagger path has no corresponding test manifest entry.

Test layers

Layer 1 — Unit (always, fast)

• Target: *.util.ts, serializers, DTO/class-validator rules.
• File: customers.util.spec.ts next to customers.util.ts.
• Style: it.each tables; behavior titles.

Layer 2 — HTTP / module (default for new endpoints)

• Target: each *.controller.ts.
• File: customers.http.spec.ts (or customers.controller.spec.ts).
• Stack: Nest TestingModule + supertest; mock AuthService, repos, FCM adapter.
• Structure (BDD-ish, no Gherkin):

describe('POST /customers (createCustomer)', () => {
  describe('Given valid body', () => {
    it('When posted Then returns 201 and customer shape', async () => { /* ... */ });
  });
});

Layer 3 — Integration (sparse, tagged)

• Target: transactions, nested POST /projects, migration-sensitive flows.
• Tag @integration, runInBand, test DB or Testcontainers.
• One smoke per epic, not every endpoint.

Layer 4 — Contract check (from Nest Swagger)

• Export swagger.json after build or via script.
• test/manifest/routes.json — AI updates when adding routes.
• test:routes — untested paths fail unless whitelisted (/api/docs, health, etc.).

NPM scripts (to add at implementation)

Agent workflow (definition of done)

After changing controllers/services:

1. npm run test:scaffold -- <ControllerName> if new controller.
2. Fill spec TODOs using @ApiProperty examples.
3. npm run test:ai until green.
4. npm run test:routes until green.
5. Swagger decorators complete on all new public routes.

Document in repo: docs/testing/AI-TESTING.md and/or .cursor/rules/testing.mdc in backend-pms-dev.

Repo conventions

1. One spec pair per module: *.util.spec.ts + *.http.spec.ts.
2. Naming: describe('<METHOD> <path>', …); prefer @ApiOperation({ operationId: 'createCustomer' }).
3. Fixtures: test/fixtures/<module>.json from decorator examples.
4. Auth: test/support/mock-auth.module.ts — never real SSO in tests.
5. DB: mocked repos by default; @integration only when necessary.
6. Shallow tests: require shape asserts + at least one negative case per endpoint.

Nest Swagger rules (enforcement for AI features)

Already in src/main.ts: /api/docs, swagger.json in dev/staging.

For every new public route:

• @ApiOperation, @ApiResponse (success + relevant errors).
• @ApiProperty / @ApiPropertyOptional with example on DTOs.
• Paths align with api prefix + URI version v1.

AI generates supertest payloads from decorators, not vault YAML.

What we explicitly drop

Priority scenarios (when implementing)

P0 — Harness

• Jest moduleNameMapper for src/... imports.
• Remove or rewrite stale test/app.e2e-spec.ts.
• test/support/create-test-app.ts, mock auth, mock repos.
• openapi:export, test:ai, agent rules.

P1 — Reference module (pick one: customers or projects)

• Scaffold + full customers.util.spec.ts + customers.http.spec.ts as AI template.

P2 — Route coverage gate

• test/manifest/routes.json + test:routes.

P3 — Per module (ongoing, AI-driven)

• projects, chat, notification — scaffold → specs → manifest.

P4 — Integration smokes

• e.g. one happy-path POST /projects tagged @integration.

Open decisions (lock at implementation time)

1. Integration DB: Testcontainers vs mocked repos only for v1? (Recommendation: mocks + 1–2 integration smokes.)
2. OpenAPI export: build-time script without running server vs curl dev swagger.json?
3. Reference module: customers vs projects as first template?

Comparison: contract portal

Related

• Repo: Project Manager/backend-pms-dev/
• Planning OpenAPI: Project Manager/docs/openapi/openapi.yaml
• Swagger UI (dev): /api/docs on running app
• [[Projects/anabatic-project-manager/reports/runner-helper-audit-core-vs-claim|RunnerHelper audit]] — patterns for service tests