Skip to content

Conversation

ariawisp
Copy link

@ariawisp ariawisp commented Sep 7, 2025

Rationale

This PR modernizes the JS test infrastructure for speed and reliability:

  • Switch to pnpm (with lockfile and CI cache) for reproducible, fast installs — no more ad‑hoc npx download drift.
  • Migrate browser tests from deprecated Karma to Web Test Runner (Playwright), improving performance and developer UX.

Refs:

Why Web Test Runner vs Karma (key advantages)

  • First‑class Playwright integration (Chrome for Testing; WTR supports headful + devtools when enabled — we default to headless)
  • Rich feedback: reports browser console logs, 404s, and errors inline in the runner output
  • ESM‑native: supports ES modules and import maps/mocking without extra glue
  • Parallelism & isolation: runs tests in parallel across files with isolated sessions
  • Fast iteration: watch mode reruns only changed tests; powered by esbuild for fast transforms
  • Simpler, less flaky ergonomics than the previous Karma setup

Why pnpm vs one‑off npx

  • Reproducible installs with a committed pnpm-lock.yaml
  • Faster CI with setup-node cache for pnpm store
  • No runtime version surprises (tooling pinned by lockfile)

Changes

What’s Included

Tasks

  • jsBrowserTest: single WTR run over all split WebCrypto files
  • jsAllTest: JS Node + browser
  • Per‑group tasks: jsBrowserTestCore (KDF/digests), jsBrowserTestAes, jsBrowserTestMac, jsBrowserTestEc, jsBrowserTestRsa, jsBrowserTestCompat

Flags and defaults

  • Filter: -Pckbuild.wtrGrep="<regex>"
  • Compatibility sub‑steps mapping: -Pckbuild.providerTests.step=
    • compatibility.generateCompatibilityTest.*generateStep$
    • compatibility.generateStressCompatibilityTest.*generateStressStep$
    • compatibility.validateCompatibilityTest.*validateStep$
  • Logs: -Pckbuild.wtrLogs=true (local default on)
  • Live progress (less buffered): -Pckbuild.wtrLive=true (local default on)
  • Fast mode (fewer iterations/sizes): -Pckbuild.fast=true (local default on)
  • Concurrency across files: -Pckbuild.wtrConcurrency=4 (local default 4)
  • Headful/devtools (optional): -Pckbuild.wtrHeadful=true, -Pckbuild.wtrDevtools=true

Default filtering:

  • Browser runs exclude compatibility sub‑steps by default; opt‑in via providerTests.step or grep.

Fast mode tuning

  • New TestTuning.fast read by shared tests
    • JS browser sets window.__CK_FAST__ from WTR_FAST env (wired from Gradle property)
    • JVM can use -Dck.fast=true (native stays off)
  • Reduced loops and sizes in:
    • DigestTest: fewer log‑scale iterations, fewer chunked repeats, smaller large buffers
    • HmacCompatibilityTest: reduced max data size

Test structure split (for parallelism)

  • Split WebCrypto browser tests into 6 modules (core/aes/mac/ec/rsa/compat) so WTR can parallelize across files.

CI changes

  • JS workflows cache pnpm and install Playwright (Chrome for Testing)
  • Default workflow uses jsAllTest for JS jobs
  • Compatibility workflow continues to use providerTests.step filtering

Infra & build

  • Aggregate tasks start the Testtool server once (BuildService), stream output via Exec
  • WTR grep is suite‑aware; unmatched tests are skipped (no false failures)
  • Switch to pnpm for JS tooling; cached in CI for speed and reproducibility
    • Reproducible installs via pnpm-lock.yaml
    • Faster installs and smaller footprint vs ad‑hoc npx downloads
  • Added WTR/Playwright dev deps (package.json), removed Karma config

Usage

  • Local fast run (browser):
    • ./gradlew jsBrowserTest
    • Local defaults: logs/live/fast enabled; concurrency=4
  • Focus a group:
    • ./gradlew jsBrowserTestAes
  • Compatibility only:
    • ./gradlew jsBrowserTestCompat -Pckbuild.providerTests.step=compatibility.generate
    • ./gradlew jsBrowserTestCompat -Pckbuild.providerTests.step=compatibility.validate
  • Explicit filtering:
    • ./gradlew jsBrowserTest -Pckbuild.wtrGrep="WebCrypto_(EdDsaTest|XdhTest).*(testSignVerify|testDeriveSharedSecret)$"
    • Headful/devtools locally: ./gradlew jsBrowserTest -Pckbuild.wtrHeadful=true -Pckbuild.wtrDevtools=true

Speed & iteration

  • Local dev runs drop to ~tens of seconds with fast mode + concurrency across multiple test files
  • Parallelism across files (not just within a single bundle)
  • One WTR per run avoids dev‑server port races; clearer progress output
  • pnpm + CI cache: faster, consistent tooling without npx drift

Breaking/Notable changes

  • Karma config deleted; WTR is now required for browser tests (Playwright install step added)
  • Exec tasks opt‑out of Gradle configuration cache (expected message in logs)
  • Compatibility steps are excluded by default in browser runs (opt‑in via providerTests.step or grep)

Validation

  • Install browsers once: ./gradlew installBrowsers
  • Quick local sanity:
    • ./gradlew jsBrowserTest
    • Expect ~tens of seconds locally with fast mode defaults; logs show SKIPs for unsupported WebCrypto features, and “all tests passed!”
  • Compatibility sample:
    • ./gradlew jsBrowserTestCompat -Pckbuild.providerTests.step=compatibility.generate

Notes

  • Exec tasks opt‑out of configuration cache intentionally (to keep streaming output and services)
  • Suite‑aware skip shim only affects grep behavior (no functional changes)

Follow‑ups (optional)

  • Extend fast mode to additional heavy tests (RSA function loops, etc.)
  • Consider sharding compatibility flows in CI matrix for better isolation

… tests for speed; add fast mode + local defaults
@whyoleg
Copy link
Owner

whyoleg commented Sep 7, 2025

This is something definitely not needed at this moment and too intrusive

@whyoleg whyoleg closed this Sep 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants