CI/CD - Testing and Publishing

Language: JavaScript

How helper modules are tested on every push and published to GitHub Packages. The framework uses a single unified workflow at .github/workflows/ci-helper-modules.yml. This guide is the canonical reference for that pipeline. Every positive rule below exists because a real failure taught it; those failures are journaled in pitfalls.md.

How It Works

A single unified workflow (.github/workflows/ci-helper-modules.yml) handles everything:

detect -- inspects the commit and the registry to decide which modules need testing and which need publishing
test-* (per-module) -- runs lint and tests on every push and PR for any module the detect job marks
publish-* (per-module) -- runs only on main pushes for modules the detect job marks as needing publish; includes a registry safety-net that skips if the version is already published

No manual tokens are required. The GITHUB_TOKEN is created automatically by GitHub Actions for every workflow run and expires when the workflow finishes.

Detect: What Triggers Test and Publish

The detect job answers two questions:

Question	Used for	Source of truth
Which modules had any file changes since the previous commit?	`test_modules` (gates `test-*` jobs)	`git diff HEAD~1 HEAD`
Which modules have a current version that is not yet on the registry?	`publish_modules` (gates `publish-*` jobs)	`npm view <name>@<version>` against GitHub Packages

test_modules is the union of:

Modules with file changes in this commit
Modules in publish_modules (so we always run tests before publishing, even if no file changed in this commit)

publish_modules is the set of modules whose package.json version is not currently published. This subsumes both:

Steady-state version bumps -- the new version is by definition not yet on the registry, so it gets published
Fresh-state recovery -- if the registry has been wiped (or never populated), every module's current version is "not published" and all of them get republished

The publish job retains a per-job safety-net that calls npm view again immediately before npm publish, so a redundant publish attempt (e.g., due to a transient registry error during detect) never overwrites a real version.

What Gets Published When

Scenario	`test_modules`	`publish_modules`	Tests run?	Publish runs?
PR opened / updated	Changed only	empty	Yes (changed only)	No
Push to main, no version bump, all already published	Changed only	empty	Yes (changed only)	No
Push to main, version bumped on module X	`[X]` (or wider if X also unpublished elsewhere)	`[X]`	Yes for X	Yes for X
Push to main, registry wiped, all 17 modules at 1.0.0	All 17	All 17	Yes for all 17	Yes for all 17
Push to main, repo's first commit (orphan)	All 17	All 17	Yes for all 17	Yes for all 17
Push to main, force-pushed reset that retains same version on disk	Changed (via diff) plus all unpublished modules	All unpublished	Yes for all unpublished	Yes for all unpublished

Fresh-State Recovery

The pipeline must work the very first time, when nothing has been published yet, and after a registry reset. The detect job's registry-existence check is what makes this work.

When you would use it

Initial repo bootstrap, all modules at 1.0.0
After deleting all packages from GitHub Packages and pushing a recovery commit
After importing the codebase into a new organization with a different @scope

What you must do

Make sure each module's package.json declares the version you want published
Make sure each module's publishConfig.registry is https://npm.pkg.github.com and the package name uses the correct scope
Push to main

The detect job will list every module whose <name>@<version> is not yet on the registry, schedule its test job, and (on success) schedule its publish job. The safety-net inside each publish job protects against accidentally republishing.

Important guidelines

Publishing is CI-only. Always use the pipeline rather than npm publish directly. The pipeline ensures tests pass before publishing and keeps version-vs-source tracking consistent.
Version numbers move forward only in normal operation. Even when restoring an old build, bump the version forward. Downstream consumers rely on ^x.y.z resolution working predictably.
Same-version republish after a registry wipe is the one sanctioned exception. GitHub Packages accepts a previously-used version name once the version is deleted, so a deliberate re-baseline (wipe all packages, push to main, let detect republish everything at unchanged versions) works end to end. Deletion mechanics and the confirmed behavior: pitfalls entry 16.

Why a Single Workflow

Earlier iterations had two separate workflows (test.yml and publish-helper-module.yml) running in parallel on the same commit. That caused:

Duplicate runs in the Actions tab. Every commit showed at least 2 workflow runs
Race conditions on publish. The publish workflow tried to npm publish on every main push, failing with 409 You cannot publish over the previously published versions whenever the version had not been bumped

The unified workflow fixes both by:

Running tests once per commit
Triggering npm publish only when a module is missing from the registry
Sequencing publish after its corresponding test job via needs:

This pattern (registry-existence check + version-bump-by-implication + per-job safety-net) is a lightweight alternative to Changesets / semantic-release. It suits quick-iteration monorepos where versions are bumped manually per Conventional Commits.

Module Execution Sequence

The pipeline processes modules in three groups, organized by dependency boundaries and runtime environment:

┌─────────────────────────────────────────────────────────────────┐
│  GROUP 1: Core Modules (Foundation Utilities)                  │
│  Location: src/helper-modules-core/                            │
│  Pattern: Class A - Zero dependencies, pure JavaScript         │
├─────────────────────────────────────────────────────────────────┤
│  js-helper-utils → js-helper-debug → js-helper-time            │
│  → js-helper-money → js-client-helper-crypto                   │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  GROUP 2: Server Modules (Node.js Runtime Required)            │
│  Location: src/helper-modules-server/                          │
│  Pattern: Classes B-F - Node built-ins, drivers, services    │
├─────────────────────────────────────────────────────────────────┤
│  Crypto → Instance → HTTP → Gateway → Storage/Database         │
│  → Verification → Logging → Auth → Queueing                    │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  GROUP 3: Client Modules (Browser/Universal Runtime)         │
│  Location: src/helper-modules-client/                          │
│  Pattern: Class A - Universal, often UI-adjacent               │
├─────────────────────────────────────────────────────────────────┤
│  js-client-helper-styler → (future client modules)             │
└─────────────────────────────────────────────────────────────────┘

Why This Order Matters

Dependency Direction

Core modules have zero external dependencies - they must publish first
Server modules can depend on Core modules
Client modules can depend on Core modules (and sometimes Server modules via extension pattern)

Runtime Boundaries

Core: Pure JavaScript, runs anywhere (Node, browser, edge, mobile)
Server: Requires Node.js runtime (built-in modules like crypto, fs, networking)
Client: Universal runtime (browser-focused but still pure JS)

Testing Requirements

Core modules: No Docker, no external services - fast unit tests
Server modules: Often need databases, emulators, or Docker - integration tests
Client modules: No framework dependencies - pure JS tests (React/Vue/etc. bindings live in extension modules)

Adding a New Module

When adding a module to the pipeline:

Determine its class using module-classes.md
Place it in the correct group based on its directory (helper-modules-core/, helper-modules-server/, or helper-modules-client/)
Position within group based on its dependencies - if Module B imports Module A, Module A must come first
Add both test-* and publish-* jobs - they run sequentially per module
Chain a dependent module's test job after its dependency's publish-* job, never its test-* job. If the new module's _test/package.json installs another in-repo package from the registry (extension modules, store adapters), its test job must needs that package's publish job so the package exists on the registry before npm install runs. Wrong chaining passes in steady state and only fails during bootstrap or a registry re-baseline (pitfalls entry 23)

The workflow file groups jobs visually with comment banners showing the group boundaries. Maintain this structure when adding new modules.

Workflow Location

.github/workflows/ci-helper-modules.yml

GitHub Actions only reads workflows from .github/workflows/ at the repository root. Placing workflow files anywhere else (e.g., src/.github/workflows/) is silently ignored.

Publishing a New Version

The step-by-step bump procedure (SemVer classification, commit format, multi-module bumps, post-publish verification) lives in ../languages/js/versioning/bump-checklist.md. This page covers the pipeline mechanics; the checklist covers the contributor's procedure.

GITHUB_TOKEN Permissions

The detect job needs packages: read so it can call npm view to check the registry. The publish-* jobs need packages: write:

yaml

detect:
  permissions:
    contents: read
    packages: read

publish-utils:
  permissions:
    contents: read
    packages: write

Without packages: write, npm publish fails with:

npm ERR! 403 Forbidden - PUT https://npm.pkg.github.com/@superloomdev%2fjs-helper-utils

Enabling Write Permissions (Repository Setting)

If your workflow still fails after adding permissions:, check the repository-level setting:

Go to Repository -> Settings -> Actions -> General
Under Workflow permissions, select Read and write permissions
Save

This allows GITHUB_TOKEN to be granted write permissions by individual jobs.

Known Failure Modes

Every CI symptom, root cause, and durable fix this pipeline has ever uncovered is journaled in pitfalls.md → CI/CD Publishing.

When you hit a new CI failure: reproduce it, confirm the root cause, then add an entry to pitfalls.md under CI/CD Publishing (Symptom → Cause → Lesson). Do not add it here. This file is for positive rules only. If the rule is small enough to live in AGENTS.md, recompile the compact one-liner into it (AGENTS.md is a derived artifact, never edited directly).

Troubleshooting

Workflow Not Triggered

Workflow file in wrong directory. Must be at .github/workflows/, not src/.github/workflows/
Branch mismatch. Only triggers on pushes and PRs against main

Publish Step Skipped

Already on registry. The detect job's registry check found <name>@<version> already published, so the module was excluded from publish_modules. The safety-net log inside the publish job (when triggered through other paths) reads <name>@<version> is already published - skipping. This is normal
Version is the empty string or missing. package.json must have a non-empty version field

403 Forbidden on Publish

Add permissions: { contents: read, packages: write } to the publish job
Enable "Read and write permissions" in Repository -> Settings -> Actions -> General

Check that publishConfig.registry in package.json is correct:

json

"publishConfig": {
  "registry": "https://npm.pkg.github.com"
}

Why Not Fine-grained PAT for Publishing

GitHub fine-grained PATs do not currently include read:packages or write:packages permissions in the UI. This is a known GitHub limitation -- there is no package permission under any other category for fine-grained PATs.

For publishing, we use the CI/CD approach exclusively:

CI/CD publishing uses the built-in GITHUB_TOKEN (automatic, secure, no token management)
Local package install uses a Classic PAT with read:packages scope (stored in __dev__/.env)

CI/CD - Testing and Publishing

On This Page

How It Works

Detect: What Triggers Test and Publish

What Gets Published When

Fresh-State Recovery

When you would use it

What you must do

Important guidelines

Why a Single Workflow

Module Execution Sequence

Why This Order Matters

Adding a New Module

Workflow Location

Publishing a New Version

GITHUB_TOKEN Permissions

Enabling Write Permissions (Repository Setting)

Known Failure Modes

Troubleshooting

Workflow Not Triggered

Publish Step Skipped

403 Forbidden on Publish

Why Not Fine-grained PAT for Publishing

References

CI/CD - Testing and Publishing ​

On This Page ​

How It Works ​

Detect: What Triggers Test and Publish ​

What Gets Published When ​

Fresh-State Recovery ​

When you would use it ​

What you must do ​

Important guidelines ​

Why a Single Workflow ​

Module Execution Sequence ​

Why This Order Matters ​

Adding a New Module ​

Workflow Location ​

Publishing a New Version ​

GITHUB_TOKEN Permissions ​

Enabling Write Permissions (Repository Setting) ​

Known Failure Modes ​

Troubleshooting ​

Workflow Not Triggered ​

Publish Step Skipped ​

403 Forbidden on Publish ​

Why Not Fine-grained PAT for Publishing ​

References ​

CI/CD - Testing and Publishing

On This Page

How It Works

Detect: What Triggers Test and Publish

What Gets Published When

Fresh-State Recovery

When you would use it

What you must do

Important guidelines

Why a Single Workflow

Module Execution Sequence

Why This Order Matters

Adding a New Module

Workflow Location

Publishing a New Version

GITHUB_TOKEN Permissions

Enabling Write Permissions (Repository Setting)

Known Failure Modes

Troubleshooting

Workflow Not Triggered

Publish Step Skipped

403 Forbidden on Publish

Why Not Fine-grained PAT for Publishing

References