Scaffold tests

The PHPUnit suite in .eddy/tests validates the scaffold itself. It covers these parts:

• The install.php flow and the projects it generates
• The tooling commands and their installer
• The command wrappers, the workflows and the repository configuration
• The docs and the update skill, checked against the code
• The generator of the animated demos

This page covers running the suite, what each group of tests needs and regenerating the snapshot fixtures.

Run the self-tests

Run the suite's commands from the repository root, except composer update-snapshots, which runs from .eddy/tests. To install the suite's dependencies, run the following command once:

composer --working-dir=.eddy/tests install

The following table lists the commands of the suite:

Action	Command
All tests	composer --working-dir=.eddy/tests test
A single group	composer --working-dir=.eddy/tests test -- --group=p0
A single class	composer --working-dir=.eddy/tests test -- --filter=InstallTest
With coverage	composer --working-dir=.eddy/tests test-coverage -- --group=p0
Coding standards	composer --working-dir=.eddy/tests lint
Fix standards	composer --working-dir=.eddy/tests lint-fix
Regenerate the demos	composer --working-dir=.eddy/tests update-assets
Regenerate the fixtures	composer update-snapshots, from .eddy/tests

lint runs PHPCS, PHPStan and a Rector dry run. lint-fix runs Rector and PHPCBF. The demos are the animated SVGs that these docs embed, and .eddy/CLAUDE.md lists what regenerating them needs.

--filter matches classes by name. The suite has 2 AssembleTest classes and 2 ToolingInstallerTest classes: a unit class in p0 and a functional class in p2. To run only 1 of them, add --group. For example, the following command runs the functional AssembleTest:

composer --working-dir=.eddy/tests test -- --filter=AssembleTest --group=p2

The coverage run needs pcov. It writes a Clover file to .eddy/tests/.logs/coverage/phpunit/clover.xml and an HTML report to .eddy/tests/.logs/coverage/phpunit/.coverage-html. Open the report's index.html to browse it. CI uploads the coverage of p0 to Codecov when the CODECOV_TOKEN secret is set.

The suite writes Clover, where the phpunit.xml of a generated project writes Cobertura. Cobertura leaves out code outside functions, and most of every tooling command sits outside functions.

phpunit.xml stops the run at the first error, failure, warning or risky test. A skipped test fails the run without stopping it. A run that stops at a warning or a risky test can still print an "OK, but there were issues!" footer. So check the exit code: a failed run exits with 1. To see how many tests the run should have reached, run it again with the same options and --list-tests, which prints 1 line per test.

Test groups

Tests are tagged p0 to p5 so CI can run them as parallel jobs. In the Needs column, the build setup is PHP with the gd, sqlite and pdo_sqlite extensions, plus Node.js with npm. A WebDriver backend is a Selenium container or a local Chrome driven by chromedriver.

Group	What it covers	Needs
p0	The unit suite in .eddy/tests/src/Unit.	PHP, make, bash, git, tar and the OpenSSH client.
p1	InstallTest, the snapshot test of install.php and of its download into an empty directory.	PHP, git and tar.
p2	AssembleTest (the codebase assembly) and ToolingInstallerTest (the tooling installer driving a real Composer).	The build setup.
p3	AhoyTest, plus AutoPortDiscoveryTest and WaitForPortTest, which run real webservers and sockets.	The build setup, Ahoy and a WebDriver backend.
p4	MakeTest, plus GeneratedProjectTest, which generates a theme and a module that each bundle an extension of the other type, then builds, lints and tests them through make.	The build setup, make and a WebDriver backend.
p5	XdebugTest, the XDebug toggle through both wrappers.	The build setup, Ahoy, make and xdebug with xdebug.mode=off.

Every group needs Composer and PHP 8.3 or newer, which the suite's dependencies require. CI also upgrades SQLite for every group except p0. p5 is the only group that loads the xdebug extension in place of pcov.

Only p0 collects coverage in CI. The other groups run the tooling and install.php as subprocesses, where pcov records nothing. Their reports would hold only the static analysis of files they never load, and Codecov would count lines such as the shebang there as missed.

Regenerate the snapshot fixtures

InstallTest runs install.php end to end and compares the result with the fixtures in .eddy/tests/fixtures/install/: a _baseline directory plus 1 directory per dataset. The fixtures are generated, so never edit them by hand. The .ignorecontent rule files are the 1 exception: you write those by hand.

Regenerate the fixtures after any change that affects the output of install.php. A change confined to .eddy/tooling/ doesn't, because the tooling isn't part of a generated project.

To regenerate the fixtures:

1. Commit your source changes first, as their own commit.

2. From .eddy/tests, run the update command:

   cd .eddy/tests
   composer update-snapshots

   It makes 1 commit: the changed baseline together with the dataset fixtures, or the dataset fixtures alone when the baseline didn't change. It exits with 0 when every dataset passed or had its fixtures rewritten. It exits with a non-zero code when a dataset fails on something other than its fixtures, or times out.

3. Review the commit with git show --stat.

4. Still in .eddy/tests, confirm that composer test -- --filter=InstallTest passes before you push.

Continuous integration

The Test Eddy workflow, .github/workflows/eddy-test.yml, runs on pushes to main and on pull requests against main or a feature/** branch. It has 3 jobs:

• eddy-test-tooling runs the suite in the p0 to p5 groups on Ubuntu and macOS. It runs p3 and p4 once per WebDriver backend: Selenium on Ubuntu only and chromedriver on both. Its p0 jobs also lint the suite, validate the composer.json files of the suite and the tooling and check that they're normalized.
• eddy-test-docs builds and tests these docs and deploys them to Netlify (see Documentation).
• eddy-test-actions lints the workflows with yamllint and actionlint, and checks them for security issues with Zizmor.

install.php deletes every .github/workflows/eddy-*.yml from generated projects (see What lives where).