* [FEATURE] One command to fix them all
Fixes#1386
* Add help text for the `fix` command
* Include `composer normalize` in the `fix` command
* Document the new `fix` Composer script in the manual
---------
Co-authored-by: Oliver Klee <typo3-coding@oliverklee.de>
The project doesn't have a `composer.lock`. That's why one should use
`composer update` instead of `composer lock`.
This can be done by using the existing `-s composer` and appending
`update.
Furthermore, the command itself was not documented or used anyway.
Relates: #1412
`composer install` does not make sense as we do not have a
`composer.lock`. One should use `composer update` instead.
This is reflected by updating the script names.
Relates: #1412
As ts is usually understood as TypeScript instead of TypoScript.
TYPO3 itself also changed its file extension from `.ts` to `.typoscript`.
We reflect that by adopting the script name.
Resolves: #1385
This change modifies the `Build/Script/runTests.sh`
command dispatcher to allow passing composer options
to the `composer install` subcommand, for example
Build/Scripts/runTests.sh -s composerInstall \
-- --prefer-source
Note that this is already possible using the generic
`-s composer` dispatcher but would resolve some user
confusion.
This aligns our implementation of runTests.sh with TYPO3 core,
see: https://review.typo3.org/c/Packages/TYPO3.CMS/+/84870Resolves: #1346
The new php based rendering container has been added
recently to render the documentation. The image come
with a internal uid/gid switch to mitigate permission
issues with docker due to a missimplementation, which
podman not suffers by proper using the kernal namespacing.
This change ensures to create a folder before hand with
the correct permissions to mitigate this for docker. On
top of that, the external user set is added for the direct
invokiation and also the interactive switch to avoid failure
in CI context usages.
To avoid permission issues with previously created folder,
a `chown` command is added to ensure correct permission on
that folder.
Resolves: #1283
Extended core-testing image has been used to
support extended `find` options not provided
by the busybox implementation.
After streamlining used `find` command usages
towards a stripped down version with #1209 it
is now possible to use the TYPO3 core-testing
images directly.
This change modifes `Build/Scripts/runTests.sh`
to use `typo3/core-testing-*` images now.
Resolves: #1275
Related: #1209
This change now uses a bash trap to ensure proper container
cleanup after some signals happend. The internal `waitFor()`
is modified to emit a signal to cleanup containers if the
waiting process does not process correctly and stop the
script execution.
Additionally, `--pull-never` as CI_PARAMS is no longer derived
from the `$CI` variable and must now be handed over from the
calling process.
Further, the max retry value in `waitFor()` is increased to 20
to mitigate the slowed down startup of MySQL 8.0 container.
Resolves: #1280
In the PHP world there are only one driver for SQLite and
Postgres available. `runTests.sh` disallows to specify a
driver for SQLite or Postgres and therefore setting the
variable would make that check more complex.
This change modifies the driver information for SQLite and
Postgres to use hardcoded driver output to keep the cross
check for specifing a driver for these database vendors.
Resolves: #1278
Command for rendering:
./Build/Scripts/runTests.sh -s docsGenerate
or just:
composer docs:generate
The reference to the TYPO3 Documentation Rendering Guide was adjusted as the Quickstart page does not exist anymore.
Additionally, the genindex isn't supported anymore. It is planned to consider the `index` directive in the global search on docs.typo3.org.
Resolves: #1189
First iteration of the `Build/Scripts/runTests.sh` supporting
`docker` and `podman` introduced a switch for direct usage in
local environments while keeping `docker` as strong default.
This change removes the hardcoded default for local and ci runs
and introduces a detection of available binaries, prefering the
`podman` over `docker` if both are installed on the host and no
specific binary selected using `-b <container-bin>`.
If no specific binary is provided, the detection chain is now:
* podman
* docker
Default binary variables are removed due to beeing obsolete now.
Resolves: #1148
The `Build/Scripts/runTests.sh` has been streamlined recently
towards the `docker-compose less` approach of the TYPO3 core.
That means, that the `runTest.sh` has been heavily modified,
and multiple changes in the core variant has been implemented.
One of these changes included streamlining the Database Version
handling for the different vendors and combining multiple options
into the `-i` option and using a generic version variable instead
of vendor specific ones. Sadly, it has been missed to align the
version output on the end to the generic variable name.
This change modifies the dbms version output at the end of the
functional test execution to use the correct generic version
variable and thus fixing the missing version information in the
output. The switch has been simplified in the same step.
Resolves: #1272
The TYPO3 core `runTests.sh` contains the `-u` option to
update the images and remove dangling images. Providing
an image name and a `--filter` option is not supported
by podmand and has not been discovered yet in the TYPO3
core implementation yet - and adopted with the recently
sync of this repository implementation to the TYPO3 one.
Both supported docker binaries `docker` and `podman`
supportes filtering images using the `reference` filter
option. Therefore, this change modifies the dangling
image remove command to use double filter options.
To verify that double filtering can be used, following
command can be executed if already some images have
been downloaded:
```shell
podman images \
--filter "reference=ghcr.io/sbuerk/demo-core-testing-*"
--filter "dangling=false" --format "{{.ID}}"
docker images \
--filter "reference=ghcr.io/sbuerk/demo-core-testing-*"
--filter "dangling=false" --format "{{.ID}}"
```
**Note:** `dangling=false` is used here instead of
`dangling=true` like in `runTests.sh`
Resolves: #1088
The TYPO3 core runTests.sh has been using the
uid option for the tmpfs mount in the past
and removed it due to the fact it's not supported
by the podman container binary.
This has been missed to change along with the
introduction of the `-b` container binary option
in #1075 and therefore the podmand and docker dual
support.
For example, following
```shell
--tmpfs ${ROOT_DIR}/typo3temp/var/tests/functional-sqlite-dbs/:rw,noexec,nosuid,uid=${HOST_UID}
```
is changed to
```shell
--tmpfs ${ROOT_DIR}/typo3temp/var/tests/functional-sqlite-dbs/:rw,noexec,nosuid
```
which works for both binaries in case of tmpfs mounts.
Resolves: #1087Resolves: #1083
Related: #1075
With 8a2e0cb the `-b` option to select the container
binary (podman or docker) has been added. During the
adoption from the TYPO3 core implementation, it has
been discoverd that the simplified checks are not
really working and changed to the better supported
version.
Sadly, it have been missed to do the at all places
correctly.
This change covers that and modifies `runTests.sh`
to use the better supported condition form for the
container binary checks introduced with 8a2e0cb.
For example, conditions like
```shell
if [ ${CONTAINER_BIN} = "" ]; then ..
```
are changed to
```shell
if [ "${CONTAINER_BIN}" == "" ]; then ..
```
Resolves: #1085
The TYPO3 core testing images only adds stuffs which is
required for the core testing itself and which can be
considered as generic project and extension testing
requirement.
This extension uses advanced `find` options which are
not part of the base busybox provided tools used by
the alpine base image, for example for the `lintYaml`
command in `Build/Scripts/runTests.sh`.
The used `-regextype` option of the `find` binary does
not exists in the stripped down busybox implementation
used by the alpine base images. The `findutils` package
is required to provide the gnu `find` command supporting
this and other advanced options, which can be found as
a solution here:
https://github.com/nextcloud/docker/issues/282
Due to the fact that the additional package will not
make the way into the TYPO3 core images like other
requirements some project or extension may have, a
demonstration github repository has been created some
time ago to demonstrate automatic extended image builds
based on the core testing images. These images already
contains stuff which is required by some projects and
extensions - and the gnu `findutils` package has been
added to these images now with the feature commit
9f8b125b6a
This change switches the used php images from the
`typo3/core-testing-phpXY` images to the extended
`sbuerk/demo-core-testing-phpXY` images.
Can be tested and verified by using
```shell
./Build/Scripts/runTests.sh -s lintYaml
```
with and without this change.
Resolves: #966
TYPO3 core introduced the `-b` option for the
`Build/Scripts/runTests.sh` script to provide
the ability to select the container binary,
`docker` or `podman` and enforces podman for
CI (GitLab runner) runs after switching the
Core CI infrastructure from Docker-In-Docker
to Podman-In-Podman.
The `ext:tea` adoption of the `runTests.sh`
has been already adopted on a intermediate
version and dropped the `docker-compose`
based container definitions. Along with
other minor prepration patches this allows
now to add the container binary selection
also.
This change modifies `runTests.sh` to
* add the `-b` option with a corresponding
helptext addition and allows `podman` or
`docker` - whereas the latter is the default.
* integrates the determined and fixed issues
with tty in CI environments by setting some
paramater for the commands based on the CI
detection (GitHub Action or GitLab Runner).
Note: The implemention for `ext:tea` does not
enforce hardcoed the `podman` binary for
CI environments. If no user selection is
provided, it fallbacks to `docker` for
all environments.
It's worth to mention that the implementation to
deal properly with user namespacing and permission
handling on mountend volumns it's required to have
at least `podman 4.x` installed. Earlier versions
does not support the required handlinging and options.
Minor moving of variable declaration are done as a
drive-by change.
The `docsGenerate` suite has been guard and display
a concrete message when executed using the `podman`
binary.
Resolves: #1075
Related: #969
The adopted `Build/Scripts/runTests.sh` contains traces
of acceptance and acceptanceInstall, mainly in the help
text.
The occurances are left-overs from the adoption phase
and are now removed due to the fact they are currently
not available at all.
Resolves: #1068
Related: #969
TYPO3 provides core testing docker images which
the core itself uses for the Core testing. Core
stopped publishing new image builds to the docker
hub already and only publish to the GitHub container
registry in the future.
Therefore, the image prefix switch between CI and
local does not make sense anymore.
This change removes the image prefix variables and
use public images directly from docker hub and the
core testing images from the TYPO3 `ghcr.io` repo.
Resolves: #1066
Related: #969
Until now three different dbmns version options
has been used like the TYPO3 core in older days.
This change aligns with the TYPO3 core and merge
these options together into one flag, keeping
`-i` for all dbms vendors now and dropping `-k`
for postgres and `-j` for mysql specific versions.
The help text is modified to align with the changed
options.
Resolves: #1062
Related: #969
`Build/Scripts/runTests.sh` provides a test-suite `composer`
and allows passing additional command and options directly
down to the composer command.
Due to an invalid quoting the passed options were broking
and not regonized by the `composer` binary.
This change removes the superflous argument passing
and therefore allows passing additional options to the
`composer` command.
Resolves: #967
TYPO3 Core and related repositories like the `typo3/testing-framework`
or `typo3/styleguide` uses a bash script arround docker to execute all
scripts over different operating systems in a controlled and ensured
manner. This helps to reproduce locally failing jobs from a CI like
GitHub actions or GitLab.
TYPO3 core dropped recently the requirement for `docker-compose`,
reducing it to `docker` and preparing for dual usage with docker
and podman in the future.
This change integrates the current state as extension wrapper,
aligning contained suits with existing tools. Additionally, a
TYPO3 core version switch is integrated.
The added script is excluded from git archive building and
during releasing.
A list of possible options can be display with:
```shell
Build/Scripts/runTests.sh -h
```
Overview of integrated features:
* `-p` to select the PHP version (7.4 - 8.3)
* `-s` to select the suite to execute
* `-t` to select the TYPO3 core version, mainly
needed for the `composer` suits executions
* `-e` to provide additionally flags for phpunit
executions, e.g. for unit and functional tests
* `-x` to enable xdebug trigger - this helps with
debugging during unit or functional test
* `-d` to select the database backend to use, needed
for functional test execution to start the correct
database
* `-a` to select the used php driver for the `-d`
database backend, if multiple options are possible
* `-i` to specify the mariadb version
* `-j` to specify the mysql version
* `-k` to specify the postgres version
Available suits (`-s`):
- cgl: Checks the code style with the PHP Coding Standards Fixer
(PHP-CS-Fixer).
- cglFix: Fixes the code style with PHP-CS-Fixer."
- clean: clean up build, cache and testing related files and
folders
- cleanBuild: clean up build related files and folders
- cleanCache: clean up cache related files and folders
- cleanRenderedDocumentation: clean up rendered documentation
files and folders (Documentation-GENERATED-temp)
- cleanTests: clean up test related files and folders
- composer: "composer" with all remaining arguments dispatched.
- composerInstallMax: "composer update", with no platform.php config.
- composerInstallMin: "composer update --prefer-lowest", with
platform.php set to PHP version x.x.0.
- docsGenerate: Renders the extension ReST documentation.
- functional: PHP functional tests
- lintTypoScript: TypoScript linting
- lintPhp: PHP linting
- lintJson: JSON linting
- lintYaml: YAML linting
- phpstan: phpstan tests
- phpstanGenerateBaseline: regenerate phpstan baseline, handy after
phpstan updates
- unit (default): PHP unit tests
- unitRandom: PHP unit tests in random order, add -o <number> to use
specific seed
Notes:
* Not everything is available directly for now in the `runTests.sh`
and therefore not enabled in CI.
* Coverage related execution needs extended work and are therefore
left out for now.
* `composer.json` protection for core version changes not included
yet like the one or other advanced option.
* Intermediate cleanings for changing php versions and core versions
not included yet and will follow.
* typo3/testing-framework template files **must** be copied to the
extension or project and **must not** be used from the vendor
folder.
Resolves: #899
Releases: main
