History ¶

The Web Camp Venlo started as a TYPO3 Camp back in 2015. I still have the T-shirt, and it was my very first international TYPO3 Camp. I'm living next to the Dutch border and wanted to get in touch with none German people of the community. I learned friendly people from the Netherlands and other countries and even sponsored the camp in the beginning, so it could happen.

But things go on and the camp opened by changing its name, now known as “Web Camp Venlo” instead of “TYPO3 Camp Venlo”.

Humans ¶

Why covering the history you might ask? Well, because the shift from TYPO3 to Web feels very much like what this year's camp felt to me. It was not about TYPO3 or anything specific. It was about something bigger, it was about humans.

The Camp started with the Keynote “Keynote - The human touch in software development” but Carlijn Compen from Canon, the printer company. There was a talk “Defend FOSS: From innovation to world-wide positive change” by Jeffrey A. "jam" McGuire and Mathias Bolt Lesniak. As well as a Talk “Domain-Driven Design: The Basics” by Stefan Koopmanschap. I also attended “The latest computer virus is called burnout and only motivated people can get it” from Jeroen Baten and “Community collaboration” by Jaap Van Otterdijk as well as his Talk “Documentation”.

All these talks have one thing in common, they are about humans, about community and therefore social aspects. I am a Backend software developer working with TYPO3 all day, building websites and allowing editors to publish content to the world. I am building websites for visitors and do so by using a free open source software content management system where humans can be creative. They can share there thoughts, just like I am doing right now typing these words into my TYPO3 installation, serving the content to you as a reader.

There were also workshops provided a day before the usual camp days. I attend the workshop “Advanced Application Architecture” by Matthias Noback. Which again was about humans. The technical part as about encapsulation of domain and system architecture. But the concepts behind that were about humans as well.

It is all about Humans, about interactions. Domain Driven Design (=DDD) is about reflecting human beings and their processes and interactions within software code. We write documentation for humans in order to understand and use something, like a CMS, build by humans.

Some might be afraid of meeting people with well known names, but they are all human. They all have been where starters are today. They attend events like this because they share the same passion. We all want to exchange knowledge. We want to become better and gain new ideas and learn from each other.

Also, the camp is organized by a small group of nice people. There are human speakers sharing their ideas, opinions, and experiences to other humans.

Everything we do influences other humans.

I want to thank the Organizers of the camp, the speakers, the sponsors and all the attendees for making events like this web camp happen. I had a great time, as every other year in the past. And I'm looking forward to joining the camp next year.

• You can find the live streams at YouTube: https://www.youtube.com/@typo3campvenlo
• You can find the program of 2024 online: https://www.webcampvenlo.nl/program-2024
• You can find photos made by great Jonathan IROULIN at: https://u.pcloud.link/publink/show?code=kZp3EJ0ZCb5USzM8YjkAO6eGYClctFihAR7V
• And next year's camp will be from 13 - 15 Feb 2025

File structure ¶

We store one composer.json which is project specific inside the root folder of a project. We also have a folder holding all project specific composer packages, e.g. specific extensions. The file directory looks like this:

.
├── composer.json
├── composer.lock
└── localPackages
    ├── client_one
    ├── client_two
    ├── client_three
    ├── css_styled_content
    ├── …
    ├── e2_survey
    └── typo3_scripts

Each local composer package has its own composer.json file. Composer can look up the local packages by adding the folder as repository within the project root composer.json:

"repositories": [
    {
        "type": "path",
        "url": "localPackages/*"
    }
]

composer.json of project specific packages ¶

Each composer.json of our local packages looks like this:

{
    "name": "vendor/client-one",
    "description": "Client one specific system adjustments.",
    "type": "typo3-cms-extension",
    "license": [
        "GPL-2.0-or-later"
    ],
    "version": "v12.0.0",
    "autoload": {
        "psr-4": {
            "Vendor\\ExtName\\": "Classes/"
        }
    },
    "extra": {
        "typo3/cms": {
            "extension-key": "ext_name"
        }
    }
}

We add a version number as this reduces the amount of changes within composer.lock files. It also doesn't force allowing unstable versions. We use the major version of the corresponding TYPO3 version for TYPO3 extensions. But hat is only a convention without real impact so far.

Those files also can contain dependencies. This is important in order to know which parts of a project actually need a dependency. But we always use the asterisk as version constraint. The project root will repeat the dependency with a proper version constraint. That eases maintenance when updating versions as there is only one place to touch. Still we have the info which part introduces a dependency and the TYPO3 composer installer can set up a proper loading order of TYPO3 extensions.

Validating composer.json files ¶

We always make use of a CI (=Continuous Integration) for all our projects. That ensures that we keep our minimum quality, even for small tasks. One task within our CI ensures that our composer.json files are okay. We use the following two lines to validate the project root file as well as files for local packages:

"find localPackages/ -name 'composer.json' -print0 | xargs -0 -n 1 -P 4 composer validate --no-check-publish --no-check-all --no-check-version --strict"
composer validate --no-check-publish --no-check-all --strict

The first lines will search all composer.json files within the folder localPackages/. This is the folder where we store our local composer packages. The -P 4 in xargs defined to parallelize the checks in 4 threads. We add the --no-check-version option as we hardcode a version number within the composer.json files for internal packages.
The second line validates in project root folder where we store our project composer.json file.

Thanks to Gerald Rintisch for motivating me to create and publish this post.

• Composer documentation describing the local path repository
• Composer documentation
• TYPO3 documentation regarding composer.json files

Note ¶

I'm not a frontend developer, but a backend developer. I have not enough experience in order to explain everything here or provide proper pointers. But the following is working for me in one of our projects.

The consuming code ¶

Let's start with our actual TypeScript code consuming ES6 modules not providing types.

import Modal from '@typo3/backend/modal.js';
import { SeverityEnum } from '@typo3/backend/enum/severity.js';

The error ¶

The above code will result in the following error output, which will lead to missing compiled JavaScript.

Src/Libs/Export/Ui.ts(4,19): error TS7016: Could not find a declaration file for module '@typo3/backend/modal.js'. 'vendor/typo3/cms-backend/Resources/Public/JavaScript/modal.js' implicitly has an 'any' type.
Src/Libs/Export/Ui.ts(6,30): error TS7016: Could not find a declaration file for module '@typo3/backend/enum/severity.js'. 'vendor/typo3/cms-backend/Resources/Public/JavaScript/enum/severity.js' implicitly has an 'any' type.
TypeScript: 4 semantic errors
TypeScript: emit succeeded (with errors)

Adding the necessary setup ¶

Now let's continue with the necessary adjustments in order to get rid of the errors.

We need to adjust the existing tsconfig.json to provide information where to find type information. Add the following entry and adjust the path to a custom folder:

{
   "compilerOptions": {
       "typeRoots": [
           "./typescript_types"
       ]
   }
}

The folder structure looks like this (again, adjust to match your own folder names):

typescript_types
└── TYPO3
   └── index.d.ts

And the content of the file looks like this:

declare module '@typo3/backend/enum/severity.js';
declare module '@typo3/backend/modal.js';

The solution is copied from the TYPO3 Extension https://github.com/quellenform/t3x-iconpack/.

DataHandler ¶

I always would suggest DataHandler first for the following reasons:

• API is pretty simple, build a single array and submit.
• It handles everything you can think of, and everything you don't remember:
• History
• Logging
• Relations
• Permissions
• Hooks (indexing in solr, or whatever your system uses hooks for)
• Cache flushing
• … (things I forgot)

But there are some downsides:

• It is slow (You can tweak the DataHandler instance to skip some things and be more performant)
• It might do more than you want, e.g. history, logging, relations, permissions, hooks, cache flushing, ….

Doctrine DBAL ¶

I than would recommend Doctrine DBAL for the following reasons:

• You are in full control. You can do exactly what you want the way you want, and can optimize for memory and performance.

But it also has downsides:

• You need to remember all necessary steps and rebuild integrations, e.g. indexing into solr, triggering cache flushes, etc.

Extbase ¶

And then there is Extbase for the following reasons:

• You stay in OOP
• Most developers are probably more familiar with Extbase than the other APIs

With downsides:

• It was not build for that task.
• You need to add workarounds to keep performance okay.
• You still need to do extra work like triggering indexing, cache flushing, etc.

It depends ¶

I would say it depends on:

• What do you need from the system, do you need hooks, logging (history) and cache flushing?
• Do you know what you do and want to have full control over every aspect?
• How much data do you have, how much memory and time do you have?
• Do you want to keep OOP?

This is a recurring topic, but Eric Harrer finally made me write this blog post by bringing up the topic once more at the Fediverse: https://phpc.social/@ErHaWeb/111096230843826340.

Read more about:

• TCE (TYPO3 Core engine) & DataHandler: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Typo3CoreEngine/Index.html
• Doctrine DBAL: https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Database/Index.html
• Example Extension importing JSON data mapped to Objects using DataHandler in the end: https://git.daniel-siepmann.de/Customers/thuecat / https://github.com/werkraum-media/thuecat
• Example Extension importing JSON data mapped to Extbase Entities using mostly Extbase for importing: https://git.daniel-siepmann.de/Customers/events /  https://github.com/werkraum-media/events

I can't cover everything as that would be way too much. I won't cover my time as JavaScript developer when jQuery was available and made JavaScript finally available to everyone. I won't cover my time with Internet Explorer 6 and its unusable developer tools. I won't cover the time before Google Chrome when Mozilla Firefox with Firebug was the only way to debug websites. And I won't cover my journey through all different kind of Editors and IDEs. I'll also not cover my journey from Windows 3.11 over Windows XP, Windows 7, different Mac versions, Gentoo and Ubuntu up to NixOS. I'll focus on the relevant parts of becoming a web developer and TYPO3.

Today ¶

I'm currently employed at Codappix GmbH a small web manufacturer in my home town, that I've founded with some friends and old fellow workers, see the corresponding section on the about me page. I'm mostly focusing on TYPO3 CMS as backend developer and integrator right now, mostly doing PHP and a bit of TypoScript. TypoScript is a specific configuration language developed for TYPO3 itself.

The idea behind TypoScript is that a developer doesn't need to know any PHP or programming. The developer should still be able to build a full-blown website with CSS, JavaScript and HTML. The whole rendering is defined by TypoScript that glues everything together. I won't go into much detail here, but the rendering of a page can look like this, so you can get an idea:

page = PAGE
page {
  config.htmlTag.attributes.class = no-js
  includeCSS.styles = EXT:site_package/Resources/Public/Css/styles.css
  10 = FLUIDTEMPLATE
  10 {
    file = EXT:site_package/Resources/Private/Templates/Page/Default.html
    variables {
      content < styles.content.get
    }
  }
}

My main goal is to develop maintainable feature rich websites solving the customer needs and making editing content as smooth as possible. TYPO3 itself is a great piece of software, most of the time, helping me to achieve the goal. It is written in PHP and open sourced, licensed under GNU GPL 2 +. It was developed by a single person back at the end of 90s. The history of the project is available at https://typo3.org/project/history. The old history is one of the main drawbacks of the system. It still has code dating back to the ancient time. But that proves the code worked for more than 20 years already. Still one wouldn't write the code the same with todays knowledge and language possibilities. One needs to remember that PHP 3 was just released, there were no OOP (=object oriented programming) nor namespaces or anything related. We didn't have composer or any other dependency manager. TYPO3 has features dating back to the beginning that are still state of the art (and were way ahead of time back than). A small list of those features:

• Extendability. The system works much like todays Laravel, Symfony, Ruby on Rails, etc. But it had the extendability back in the 90s. The system itself is build by a set of extensions (Symfony calls it bundles). Developers can create and share own Extensions. TYPO3 itself has an Extension Manager and a central Extension Repository, the TER (=TYPO3 Extension Repository). TYPO3 also supports Composer, allowing you to draw any composer package. Packages with a dedicated type are considered to be TYPO3 extensions and handled in a special way.
• Configurability. Most parts of the system can be configured. There is most often no need to build an Extension, one can configure stuff, hide inputs, alter existing inputs, change behaviour, etc.
• Versioning. TYPO3 is a CMS and has versioning of content built into it. Every change made by any user within the TYPO3 Backend is tracked. Users are able to check the history of every single record and can revert changes.
• Workspaces. TYPO3 provides different workspaces, allowing users to create content in their own workspace. It supports workflows, allowing other users to review the content and publishing content. A preview URL can be created for content within a workspace, allowing to share a draft with foreign visitors, customers, etc.
• Multilanguage. TYPO3 allows to maintain content in any language, content can be translated, maintained independent of different languages. It offers overviews what parts of a site are lacking a translation, etc.
• Multisite. TYPO3 has multisite built in. It is possible to manage hundreds of websites within a single installation, sharing users, permissions, and content.
• Permissions. TYPO3 has a fine grained permission system of users and groups with permissions to different record types, content types, mount points, languages, and way more.
• Robustness. I only list this cause I've heard from other projects that this issue exists. But TYPO3 has a smooth upgrade path. It is possible to update from an old TYPO3 version 4 to an current TYPO3 version 12 LTS (=Long Term Supported). TYPO3 always provides so-called update wizards that will handle the migration of data to stay compatible. Only rendering is constantly changing, so one might have its own rendering (templates) anyway and might need to do some small adjustments.
• Image processing. Another one I took for granted, but it looks like many systems don't have it out of the box. TYPO3 allows using any server side supported file formats and will process them. Users can define an area within an image to be rendered on the website, they can crop and resize images.
• FAL (=File Abstraction Layer). TYPO3 abstracts the file system, which allows to add other drivers like Amazon, Google Drive, Dropbox, etc. It is not necessary to store all files, e.g. images or PDFs, on the same server.
• TCA (=Table Configuration Array). TYPO3 is like a big improved version of a database tool like Sequel Pro, Sequel Ace, PHPMyadmin, etc. It has a backend with many types of inputs and the TCA is the configuration that TYPO3 reads in order to render the different forms for users to edit content. It allows to rapidly create an interface familiar to users and focus on actual business logic or rendering and features instead of building forms and validation for adding content.

A more complete list is available at https://typo3.org/cms/features.

I'm really happy that we have customers like https://werkraum-media.de/ that live open source the same way we and TYPO3 do. That's why we have some open source TYPO3 extensions available: https://packagist.org/packages/werkraummedia/.

I'm also nowadays very active within the TYPO3 community, since some years. I've listed all my activities at the TYPO3 section on the about me page.

The meantime ¶

I've already worked at two agencies and for one freelancer before I've joined the efforts of my friends to start our own company. One reason we started our own company and that we don't call it an agency is that we hate how most agencies seem to work in web development here in Germany. There is a lot of stress, time pressure, bad decisions, bad leadership and foremost bad technical decisions. There is no time to do things right and think about your work. We had to spend extra hours fixing stuff, altering implementations to follow ever-changing requirements. It is a nightmare. Still in Germany everyone talks about “Fachkräftemangel”, we lack well-educated people on our business. But most companies don't provide an environment that allows to work as expected. There are so many companies, do your self a favour if you can and don't stop looking for the right company. Settle down once you find it and be happy. Or become a freelancer or found your own, as we did.

We still always tried to become better. We checked solutions like https://www.selenium.dev/ to automate end to end website tests. Furthermore, we started to use automated deployments and continues integrations including static code analysis, code style checks and a lot more. We always tried to get used to testing. Things that we take for granted nowadays, but it was a hard time back then. The first deployment I established was more an accident. It was during my apprenticeship. We wanted to relaunch a huge website and had multiple different staging systems in order to test different features in parallel. I still used windows and wanted to test out shell scripting. I installed Cygwin on Windows and tried very small scripts. I then had the job to update all the different staging systems. Furthermore, I needed to sync content between them and update the code. That's when I started to build my first deployment via shell scripts on Windows. We then had a day when our boss asked when we could go productive and how the procedure would look like. I said that the script should work fine for production as well, and we had a deployment. The bad thing was that it was running on my machine, and a colleague and I always hat to set up and test it on his machine every time I went on holidays. They moved the deployment into a virtual machine when I left the company.

That was the time when I got deeper knowledge of TYPO3. I was only a backend developer back then, now knowing much about integration or features provided by TYPO3 except for developing extensions. But not knowing the features of TYPO3 itself, or the interface is a big drawback. You don't know how users will actually use TYPO3 and can't integrate solutions into the system. You most likely, like I did, always start from scratch and build your own stuff that you will add to the system, not playing nice with the existing system. E.g. you will build custom modules for the TYPO3 backend to get things done, instead of using existing modules with a bit of configuration and small modifications.

That was also the time when I wanted to give something back. I understood that this system, being open source, offered with no fees, was great and enabled me to work and earn money. I joined TYPO3 Bar Camps and User groups. Check the Wikipedia page if you don't know what a Bar Camp is. That way I encountered the great community. It was hard for me at the beginning. I'm not that extrovert at the beginning in foreign environments with unknown people. I visited different sessions and listened, but didn't talk to anyone. Nowadays, the community feels like a big family for me. There were stories where people help a community member to build a ramp because she is now bound to a wheelchair. People ask whether everything is okay and everyone is happy if we meet. It is way more than exchanging experience, knowledge. It is about the same mindset, being open source, sharing ideas and concepts, becoming motivated.

The beginning ¶

My dad was a programming for banks doing Cobol all the time. He taught me the dual system when I was pretty young and forced me to watch movies in English. I didn't like anything of that. But I also was able to buy his old PC when I was in ground school, it was a Dos 6.22 with Windows 3.11. I only used it to play around, using everything that was installed on the PC. There were games like Settlers and software like Word for Windows, a tool to draw diagrams and another tool to build any kind of prints, like business cards. I also bought his old printer. That way I already had a business card when I wasn't even able to write “computer” but wrote “compjuter” on it. I never thought I would become a programmer. But my Dad also was curious about the Internet. We had ISDN and I got a 56k Modem. My dad had his own website www.srui.de. This was when I was around 13 years old. I read his book about HTML 4 which I still own. I used some editor and build my first websites which I couldn't host anywhere but could view them via Netscape on my own local PC. Still I had no idea what I would become. I thought I'd work in a library or book store, where I did my first two internships each two weeks.

We than had IT at school in class 10, and I asked the teacher whether I could do the teaching about HTML and JavaScript. That was the time when JavaScript was used to do stupid animations like eyes following the mouse, stupid alerts and scrolling status bars (Yes browser once had a status bar with a loading bar and infos about links, etc.). Still I didn't think about a job as programmer. I didn't finish school that good, because I was way too lazy. I ended in another school after class 10, there I were additional 3 years becoming a “Staatlich geprüfter kaufmännischer Assistent mit Fachrichtung Informationsverarbeitung”. That means I, at least in theory, know the business side and could support a CEO. But the focus was on IT again. And we learend about database normalization, programming and debugging. All using Windows tools like Excel and Visual Basic as well as Microsoft Access. The school forced us to do two internships, one six and another four weeks long. I travelled to our industrial area and walked from door to door looking for a business where I could do my internship. I visited a company doing lightning and sound for events, and many IT companies. I settled for the one where a tall man with long hair was sitting in front of two monitors, reminding me of Neo from Matrix. This company allowed me to do an internship without any further questions. All other companies wanted to know details, I should write a letter, provide certificates, etc. I was lazy and did a six-week internship at that web agency, mostly doing image processing within Photoshop. Cause I thought I was too dumb for programming, but I could repeat what someone will show me within an image processing program. But one frontend guy also introduced me into HTML and CSS, nice technics how to build layouts with div Tags and floats, how to use a 1px image with repeat as background.

I finished off the school and knew I won't do anything related to business. I had some months until my social service started, and I asked the old company whether it would be possible to help them out for five months, again mostly doing image processing within Photoshop. Once finished with social services I still had no plan, so I asked whether I could work at that company again. But my parents wanted me to do another education and that's why I did another three years education within the company. It was a hard decision, but I decided that I won't do image processing but would like to finally learn programming. I settled, and the company didn't do a great job. My first task was a very long-running project without much help and way ahead of what I could do. I needed to program a web interface allowing citizens of our home town to apply for becoming a company. Including the frontend forms, the editing interface for the city, all data crypted within the database. The city already had a TYPO3, but everyone told me to just program, and they would integrate the result into the website. I really thought I would abort the education at least three or five times within the first one and a half year. I constantly got bad feedback but way to less help. And I constantly needed to do the same things, building web forms to insert data that would be stored within a database and representing it to website visitors.

I then thought that this TYPO3 CMS we were using must be good for something and took a first look into the docs. I discovered the TCA (=Table Configuration Array) which would allow me to configure the forms. That was the first time I got positive feedback and other employees asked me for help, because no one ever had a look at the features and docs of the software. That was the turning point.

This blog post was made for @array@fosstodon.org as he requested the post at https://fosstodon.org/@array/110701165059012368. I hope you liked the post. Feel free to contact me and request other blog post.

You might like those sources:

• https://typo3.org/ The official website of the TYPO3 CMS project.
• https://typo3.org/cms/features The features of TYPO3
• https://en.wikipedia.org/wiki/BarCamp Wikipedia explaining Bar Camps.
• https://git.daniel-siepmann.de/Customers a list of open source projects I am or were working on for customers.
• https://docs.typo3.org/ The official docs of TYPO3 CMS.
• https://extensions.typo3.org/ The extension repository of TYPO3.
• https://packagist.org/?type=typo3-cms-extension TYPO3 Extensions on Packagist.

Check out my other blog posts:

• Video: Short introduction into TYPO3 - What is Content?
• Using PHP_CodeSniffer for automated code migrations, when rector wasn't a thing.
• I still plan to create a curated list of my TYPO3 talks on YouTube, but I'm too lazy, still I've collected some from TYPO3 and vim: https://www.youtube.com/playlist?list=PLDXiDr43HScExqOdpcQbCTmxfdBtIn63O.

Meet rector ¶

rector is an open source PHP framework that allows to auto migrate source code. The project is available at https://getrector.com/ and https://github.com/rectorphp/rector/. At its core it provides so-called rectors that check source code and adjust the source code. Rector itself delivers some rectors already for certain tasks, a full list is available at https://getrector.com/documentation/rules-overview. It includes rules (=rectors) to migrate based on PHP version switches, Symfony, PHPUnit, etc. It is possible to write custom rules (rectors) https://getrector.com/documentation/custom-rule covered with tests.

It is also possible to re-use existing rectors with different configuration. E.g. there is one rector to rename methods. This rector can be configured to migrate method calls for example if a framework switches from getView() to view().

Rector for TYPO3 ¶

TYPO3 developers have an easy job, there is already a community project maintaining specific rectors for upgrading TYPO3 versions. You can check the project at https://www.typo3-rector.com/ as well as https://github.com/sabbelasichon/typo3-rector. It comes with a documentation and explains how to copy the template configuration and how to execute the update. I'll not go into much detail about that.

It is possible to extend the boilerplate configuration. That way TYPO3 extensions can provide their own rectors that can be added to your project setup, in order to not only auto migrate source code related to TYPO3 CMS itself, but also to TYPO3 Extensions provided by 3rd Parties.

Rector for Aimeos ¶

Aimeos doesn't have anything regarding rector out of the box. But one can use that as an opportunity to get used to rector. We had one project where we had to upgrade from TYPO3 v8 to v11 and Aimeos 2018 to 2022. We first started with migrations by hand until we realized how much work it is for the concrete code base in that project. So I've started to add rector rules. The result looks like this:

use Rector\Renaming\Rector\ClassConstFetch\RenameClassConstFetchRector;
use Rector\Renaming\Rector\MethodCall\RenameMethodRector;
use Rector\Renaming\Rector\Name\RenameClassRector;
use Rector\Renaming\Rector\StaticCall\RenameStaticMethodRector;
use Rector\Renaming\ValueObject\MethodCallRename;
use Rector\Renaming\ValueObject\RenameClassAndConstFetch;
use Rector\Renaming\ValueObject\RenameStaticMethod;

// Custom Aimeos migration
$rectorConfig->ruleWithConfiguration(RenameClassRector::class, [
    'Aimeos\MShop\Context\Item\Iface' => 'Aimeos\MShop\ContextIface',
]);

$rectorConfig->ruleWithConfiguration(RenameMethodRector::class, [
    new MethodCallRename('Aimeos\Admin\JQAdm\Base', 'getContext', 'context'),
    new MethodCallRename('Aimeos\Admin\JQAdm\Base', 'getView', 'view'),
    new MethodCallRename('Aimeos\Base\Mail\Message\Iface', 'setSubject', 'subject'),
    new MethodCallRename('Aimeos\MShop\Common\Manager\Iface', 'createItem', 'create'),
    new MethodCallRename('Aimeos\MShop\Common\Manager\Iface', 'createSearch', 'filter'),
    new MethodCallRename('Aimeos\MShop\Common\Manager\Iface', 'deleteItems', 'delete'),
    new MethodCallRename('Aimeos\MShop\Common\Manager\Iface', 'findItem', 'find'),
    new MethodCallRename('Aimeos\MShop\Common\Manager\Iface', 'saveItem', 'save'),
    new MethodCallRename('Aimeos\MShop\Common\Manager\Iface', 'searchItems', 'search'),
    new MethodCallRename('Aimeos\MShop\ContextIface', 'getCache', 'cache'),
    new MethodCallRename('Aimeos\MShop\ContextIface', 'getConfig', 'config'),
    new MethodCallRename('Aimeos\MShop\ContextIface', 'getLocale', 'locale'),
    new MethodCallRename('Aimeos\MShop\ContextIface', 'getLogger', 'logger'),
]);

$rectorConfig->ruleWithConfiguration(RenameStaticMethodRector::class, [
    new RenameStaticMethod('Aimeos\MShop\Factory', 'createManager', 'Aimeos\MShop', 'create'),
]);

$rectorConfig->ruleWithConfiguration(RenameClassConstFetchRector::class, [
    new RenameClassAndConstFetch('Aimeos\MShop\Common\Item\Address\Base', 'SALUTATION_MISS', 'Aimeos\MShop\Common\Item\Address\Base', 'SALUTATION_MS'),
    new RenameClassAndConstFetch('Aimeos\MShop\Common\Item\Address\Base', 'SALUTATION_MRS', 'Aimeos\MShop\Common\Item\Address\Base', 'SALUTATION_MS'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'ALERT', 'Aimeos\Base\Logger\Iface', 'ALERT'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'CRIT', 'Aimeos\Base\Logger\Iface', 'CRIT'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'DEBUG', 'Aimeos\Base\Logger\Iface', 'DEBUG'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'EMERG', 'Aimeos\Base\Logger\Iface', 'EMERG'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'ERR', 'Aimeos\Base\Logger\Iface', 'ERR'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'INFO', 'Aimeos\Base\Logger\Iface', 'INFO'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'NOTICE', 'Aimeos\Base\Logger\Iface', 'NOTICE'),
    new RenameClassAndConstFetch('Aimeos\MW\Logger\Base', 'WARN', 'Aimeos\Base\Logger\Iface', 'WARN'),
]);

That way we don't have to write actual code but configure rector to migrate method calls, classes, constants, etc. Thanks to rector generic rules.

Thanks to werkraum_media who allowed me to do my first Aimeos upgrade in order to get used to configure generic rector rules.

Read more about rector:

• Project website: https://getrector.com/
• Project repository on GitHub: https://github.com/rectorphp/rector/
• Adding custom rules: https://getrector.com/documentation/custom-rule
• Overview of existing rules: https://github.com/rectorphp/rector/blob/main/docs/rector_rules_overview.md

Read more about TYPO3 rector:

• Project website: https://www.typo3-rector.com/
• Project repository on GitHub: https://github.com/sabbelasichon/typo3-rector

Why I blog ¶

I wouldn't blog if I'm not motivated. I currently believe you either need motivation or pressure in order to do things. Some might feel the urgent pressure to rescue our planet, others might feel the pressure to finish something in time. But some might find it motivating to help other people. I'd say motivation is the better reason to do things. And I don't have any pressure to publish blog post.

I blog because I feel motivated. I'm motivated to blog because it is a way to share my knowledge with others. It allows me to demonstrate things I've done and am proud of. That's the same as being at a user group or on a Bar Camp. All three allow me to share my knowledge with others.

But it wouldn't motivate me if it would be a one way. It motivates me because of the feedback. The feedback regarding blogging is very small. I can see the number of page views, but I don't know if it is actually useful for visitors. Unless someone contacts me and says thanks. That doesn't happen very frequently, leading to the feeling my blog posts aren't helpful, leading to the feeling this is wasted time, leading to lack of motivation.

The feedback during camps and user groups on the other hand is much more. At least one visitor of a session always says thanks or ask questions. Questions are an indicator that they find the session interesting. And some people also say thanks for one or the other blog posts, but they wouldn't write me, its just that they see me and think they will share their gratitude. That's very motivating.

Camps and user groups also allowed me to share what I've done since the last camp. Most of the time I couldn't wait until I could share the latest ideas and concepts and their outcome. Some people could follow my ideas and found them useful, which ended up in me writing a blog post to share with everyone else.

Why I lack motivation ¶

I've now covered what motivates me and why I blog. I'll now share why I currently lack the motivation.

I don't visit many bar camps any more and I don't visit any user group at all. That happened during the COVID-19 pandemic, for obvious reasons. But I also no longer visit them after the pandemic. I'm too lazy, the habits have changed. I'm now in a relationship, and it is more important to have time with my partner than to invest time in visiting camps or user groups. That leads to missing motivation, as most feedback came from personal talks. That also leads to missing exchange of knowledge and I no longer share what I've done.

I also didn't share my ideas any more. I noticed that during the TYPO3 Developer Days 2022 when Benni asked me what I've done in recent years. I couldn't answer that question, it felt like I've done boring daily business. But of course I've done the same daily business as before the pandemic, always questioning existing solutions and looking for something better. Developing new concepts and ideas and trying them out. There would have been so much to talk about and to share and blog. But my mood has changed due to not sharing my excitement with others for some years.

What you could learn ¶

Every one of us needs motivation to pass the days, weeks, months, and years. Motivation is the positive influence that's keeping us alive. That's why I believe every one of us should find what motivates us and focus that in our lives. Motivation can change though so can our focus. We shouldn't feel bad because of that. Instead, we should become aware of our shift of motivation or focus and adjust our live.

My focus has shifted towards my current relationship and becoming more friendly to our planet. I can't use that as focus or motivation for my job, but for my live. It is currently a bit complicated to keep up the same level of motivation for my job. But the issue might not be the lack of motivation but the lack of self awareness of motivation. Sometimes we don't feel motivated because we are not aware of certain things. Things might become commonplace, like your job or relationship. But that doesn't mean they become less motivation, just you might not notice any more. We can train our self awareness for those kinds of things and become motivated again. That's what I'll try to do next. You might notice the outcome based on the upcoming or missing new blog posts.

I hope this blog post helps some of you to become motivated again. Feel free to contact me. And feel free to contact every one who motivates you or who helps you. People from your private live, family, partners, friends. As well as within your jobs like your boss or colleagues. And don't forget people like open source maintainers, writers, video producer people in your library, in your supermarket, and everyone else who makes your live better. I believe it doesn't harm our world and society to say “thank you” more often.

Thank you for reading this blog post.

Update vs Upgrade ¶

People might have different definitions for “Upgrade” as well as “Update”. Here is my definition, in order to align expectation for this blog post. Both mean to update a software. E.g. update a TYPO3 installation from any version to any higher version. The goal is to keep the system running with all expected features and only minor necessary adjustments. The difference between an upgrade and an update is to also make use of new features and migrate existing code and features. E.g. make use of TYPO3 dashboard when updating to v10 or higher. Or migrate to use the new TYPO3 queuing when updating to TYPO3 v12 or later. This is mostly useful for very long running and feature heavy installations.

Proper Setup ¶

I first start with a proper setup. Some installations were maintained by other agencies, some might be set up some years ago. I always start with migrating the existing setup into our own current best practice setup.

1. Create or migrate Git. Some installations might not provide a repository already.
2. Migrate foreign dependencies (e.g. from infrastructure of old agency) into mono repository, see my blog post Create Git mono repository.
   This is done for all dependencies that are no longer maintained somewhere else, but crucial for the installation.
3. Get rid of foreign dependencies. Some setups might fetch sources from the infrastructure of the old agency. This could be circumvented already by step 2.
4. Ensure you have a locally running system. That way you can switch back and forth and compare the update with the old system.
5. Migrate to composer. Remove version constraints from project specific packages. Add a * as version constraint. That way we don't need to adjust the constraints for upgrades over and over again. Still we define dependencies and loading order. See blog post TYPO3 Composer Best Practices.
6. Set up proper CI (=Continuous Integration).
   This makes it easier to catch issues by applying static code analysis and prevent unnecessary merge conflicts by applying a CGL (=Coding Guide Line). Add proper linters within CI.
7. Add a CGL with proper tooling.
8. Add PHPStan with baseline.
9. Add linter for: TypoScript, XLIFF (Translation files), Yaml, PHP.

That's the proper setup I need prior starting with the actual update.

The update ¶

I'm doing the following steps to update a TYPO3 installation:

1. Install the target version.
   1. Therefore, I remove all installed files (e.g. typo3/sysext, typo3conf/ext, vendor, …) and composer.lock file.
   2. I then manually update composer.json. It will include the target versions of all packages, TYPO3 system extensions and 3rd Party Extensions.
2. Add ssch/typo3-rector package and configure it.
3. Execute migration via rector.
4. Replace auto generated config.persistence.classes TypoScript configuration. Rector is unable to split this up, so we will do in next step.
5. Execute cache flush from command line and fix all issues by hand until this succeeds. We then should have a running TYPO3 install tool.
6. Remove extensionScannerIgnoreLine and extensionScannerIgnoreFile leftovers from previous upgrades. False positives from previous upgrades might be real matches for the new upgrade.
7. Open TYPO3 install tool and execute the various checks within the Upgrade module. Fix remaining by hand.
8. Create site configuration. (If updating from pre site handling versions)
9. Open backend and frontend and fix all remaining issues by hand.
   Only open the backend and startpage, not all pages.
   That gives a good feeling, and allows other team members to work in parallel.
10. Use automated tools to find and fix issues:
    • Execute and investigate / fix new PHPStan issues
    • Execute and fix Unit Tests
    • Execute and fix Functional Tests
11. Configure routing within site configuration to match original URLs.
12. Configure EXT:seo to create sitemaps for all detail views. (Not necessary, more like an upgrade)
13. Crawl original setup prior update for all URLs. You can read my basic approach in a separate blog post.
14. Crawl new setup after update for all URLs.
15. Compare output of both crawls and fix all remaining issues. I also wrote a separate blog post describing my basic approach.
16. Manually check backend features. (Or cover with end to end tests, to benefit in the future)

PHPStan + rector ¶

Why do we use PHPStan within all our projects? It allows us to prevent many stupid bugs. It forces us to write clean understandable PHP Code. It eases updates as rector uses PHPStan under the hood and only is as good as the understanding of the code by PHPStan. Furthermore, there is a huge ecosystem around PHPStan that can improve the benefits of PHPStan.

rector is used as we save a lot of time. Rector can do the following modifications to our PHP sourcecode:

• Migrate PHP source code to newer PHP versions, e.g. update our code base from PHP 7.4 to 8.2.
• Migrate TYPO3 source code to newer TYPO3 versions, e.g. from TYPO3 8.7 to TYPO3 12.
• Migrate code related to 3rd Party libraries, e.g. PHPUnit tests.
• We often can configure additional rules to save further time (and share them upstream).

We most often use the following packages related to updates:

• ssch/typo3-rector
• phpstan/phpstan
• saschaegerer/phpstan-typo3
• phpstan/phpstan-phpunit

I'll also share some learnings from using rector for updates:

• Do not use UP_TO_PHP_82 sets. Those include all the rules from the lowest version up to the defined version and will slow things down.
  Instead use dedicated Versions, e.g. PHP_80, PHP_81 and PHP_82 when updating from PHP 7.4 to 8.2. This will be much faster.
  Same is true for TYPO3. And I highly recmmend that, as TYPO3 does not include all lower versions. E.g. using UP_TO_TYPO3_12 will only include V11 and V12, not earlier versions.
• Use a separate commit to where you execute rector. That eases rebases as you can remove the previous result and re-run rector instead.
• Write and share your own rules if useful, e.g. we created and shared some of our rules here: https://github.com/sabbelasichon/typo3-rector/issues/3936 That also means you should probably check open issues before you start configuring own rules.

Ease updates in the future ¶

You sometimes make changes where you know they need adjustments once you update the system. E.g. you backport a certain feature and can remove the backport. There are some ways to make this easier. We did not settle for any of them yet but still play around with them:

Maintain a proper documentation. We did this for some of our public extensions. You can see an example at https://docs.typo3.org/p/werkraummedia/thuecat/2.1/en-us/Maintenance.html. There we document the changes which makes it easy to check the documentation during each update and process the information. Another way is to document inline within the code. You need a proper rule how to document in order to properly search code for those comments.

Thanks to some people:

• Peter Kraume for reviewing the post
• Justus Moroni for reviewing the post
• Julian Hofman for noticing a wrong statement about UP_TO Constants regarding TYPO3, this was fixed. One can also use the TYPO3_12 instead of the UP_TO_ version.

And thanks to companies I've done updates in that way:

• reuter.de where I'm working on the same instance from v7 up to v12 (at time of writing)
• werkraum_media where I could do a lot of TYPO3 update on various versions.

• Official TYPO3 Documentation on upgrading/updating: https://docs.typo3.org/Home/UpgradingTYPO3.html
• Official TYPO3 Documentation on Extension Scanner: https://docs.typo3.org/m/typo3/reference-coreapi/12.4/en-us/ExtensionArchitecture/HowTo/UpdateExtensions/ExtensionScanner.html#extension-scanner
• Community driven rector extension for TYPO3 updates: https://packagist.org/packages/ssch/typo3-rector
• Blog post on how to create a monorepository
• Blog post on how to compare output of TYPO3 frontend. Alternative: https://sitediff.io
• Blog post regarding TYPO3 Composer Best Practices
• PHPStan static code analyzer: https://phpstan.org

Our Goal ¶

The goal is to add a custom entry to the drop down as shown in i56. It will show the Git SVG Icon, the title “Git Commit” and the actual Git commit.

Adding EventListener ¶

We will need to register a new event listener via Services.yaml of our Extension:

services:
  Vendor\ExtName\EventHandler\SystemInformationToolbarCollectorHandler:
   tags:
     - name: 'event.listener'
       event: 'TYPO3\CMS\Backend\Backend\Event\SystemInformationToolbarCollectorEvent'

That will ensure our own class Vendor\ExtName\EventHandler\SystemInformationToolbarCollectorHandler is called for the corresponding event.

The class itself needs to implement the PHP magic method __invoke(). That method is called for the event, as we didn't provide a dedicated method when registering our event listener. The class itself can look like:

<?php

namespace Vendor\ExtName\EventHandler;

use TYPO3\CMS\Backend\Backend\Event\SystemInformationToolbarCollectorEvent;
use TYPO3\CMS\Backend\Backend\ToolbarItems\SystemInformationToolbarItem;
use TYPO3\CMS\Backend\Toolbar\Enumeration\InformationStatus;
use TYPO3\CMS\Core\Core\Environment;

class SystemInformationToolbarCollectorHandler
{
    public function __invoke(SystemInformationToolbarCollectorEvent $event): void
    {
        $status = 'unknown';
        $content = file_get_contents(Environment::getProjectPath() . DIRECTORY_SEPARATOR . 'REVISION');
        if (is_string($content)) {
            $status = mb_substr($content, 0, 7);
        }

        $event->getToolbarItem()->addSystemInformation(
            'Git Commit',
            $status,
            'actions-brand-git',
            $status === 'unknown' ? InformationStatus::STATUS_WARNING : InformationStatus::STATUS_NOTICE
        );
    }
}

It will read the contents of the file REVISION from project (not web) root. That way the deployment can create the file and add the commit hash as content. We will strip down to first 7 characters of the hash. That's usually enough to be unique within a project.

• Documentation of Event: https://docs.typo3.org/m/typo3/reference-coreapi/11.5/en-us/ApiOverview/Events/Events/Backend/SystemInformationToolbarCollectorEvent.html
• Documentation of PSR-14 (Event Dispatcher): https://docs.typo3.org/m/typo3/reference-coreapi/11.5/en-us/ApiOverview/Events/EventDispatcher/Index.html
• Documentation of PHP magic method __invoke(): https://www.php.net/manual/en/language.oop5.magic.php#object.invoke

Using the accounts ¶

You can either follow the different accounts with your own Fediverse account. Each account also is available as an Atom Feed which you can add to your feed reader. You can also check each accounts contacts and add the RSS feeds they mirror to your reader.

TYPO3 News Account ¶

The first and probably most important account is TYPO3 News which is available at https://friendica.daniel-siepmann.de/profile/typo3news. You can see the mirrored feeds within the contacts page at https://friendica.daniel-siepmann.de/profile/typo3news/contacts.

TYPO3 Security Advisories Account ¶

TYPO3 Security Advisories is available at https://friendica.daniel-siepmann.de/profile/typo3securityadvisories. You can see the mirrored feeds within the contacts page at https://friendica.daniel-siepmann.de/profile/typo3securityadvisories/contacts. It will mirror the RSS feed of the official TYPO3 Security Advisories.

TYPO3 Commits Account ¶

Another accounts mirrors each TYPO3 commit made to the main branch. The account is available here: https://friendica.daniel-siepmann.de/profile/typo3commits.

TYPO3 TYPO3 Issues Account ¶

Another accounts mirrors each TYPO3 issue created at forge.typo3.org. The account is available here: https://friendica.daniel-siepmann.de/profile/typo3issues.

TYPO3 Blogs Account ¶

And there is an account mirror different community blog feeds. The account is available at https://friendica.daniel-siepmann.de/profile/typo3blogs and you can check all the blog feeds at https://friendica.daniel-siepmann.de/profile/typo3blogs/contacts.

TYPO3 Videos Account ¶

There is also an account mirroring feeds with Videos, e.g. TYPO3 YouTube Channel. The account is available here: https://friendica.daniel-siepmann.de/profile/typo3videos.

TYPO3 Decisions and Talk Accounts ¶

One account mirrors each new TYPO3 decisions topic from https://decisions.typo3.org/latest and is available at https://friendica.daniel-siepmann.de/profile/typo3decisions/.

The other mirrors each new TYPO3 talk topic from https://talk.typo3.org/ and is available at https://friendica.daniel-siepmann.de/profile/typo3talk/.

TYPO3 (lemmy) Forum ¶

This one isn't provided by myself, but Matengor. A forum is more or less the same concept as Reddit has. You can find the forum at https://lemmy.ml/c/typo3. Thanks to Matengor for providing the forum. You don't need an account there, as lemmy is part of the Fediverse.You can follow the forum as you can follow other user accounts. And you can post to the Forum as you can post to a user.

The Problem ¶

Sometimes editors need to insert line breaks <br> or special characters like ⓒ and a reduced set of formatting like ^superscript or italic within headlines or other input fields of TYPO3. This might be necessary because a headline references a product or company.

But that's not possible with TYPO3 native input fields. Editors might not be aware of system features to insert special characters and even then they won't be able to insert line breaks or formatting within those fields.

Existing Approaches ¶

Some approach we've seen so far:

Provide a documentation with special characters for copy and paste.
We don't like this approach as editors need to switch between TYPO3 and another system holding the documentation for copy and paste which itself is not a nice workflow.

Provide special sequences like [BR] (good old BB-Code style 😉). Those become replaced via TypoScript.
We don't like this as well as you need to document those sequences. And they are more or less HTML in the end, just different braces. One big goal of most CMS is to remove the need for editors to write HTML.

Our Approach ¶

We thought we could use a very reduced RTE configuration for those fields. The goal would be to have a small input field instead of a huge text area. Furthermore, only the necessary formatting should be provided, resulting in a very small toolbar. That would allow an already known way to format text and insert special characters. Editors won't need to learn something new.

TYPO3 offers the option rows for <textarea> inputs to define how large an input should be. Furthermore, TYPO3 provides a max option for input to limit the number of characters. That's important to not lose input after persisting to database. Both should be respected as well. Allowing integrators and developers to use existing approaches and documented options.

TYPO3 by default processes the input when storing and retrieving from database. It for example will wrap lines with <p> tags. We don't want that, as it should be handled like a normal input field.

We use a dedicated RTE configuration and a small EventListener. The configuration will provide the wordcount plugin, adjust the processing of the input and define which formatting options should be available. The Listener should connect the TCA configuration for rows and max to the RTE, by adjusting the configuration.

The Configuration ¶

The configuration of figure i52 looks like this:

imports:
  - {resource: "EXT:rte_ckeditor/Configuration/RTE/Editor/Base.yaml"}
  - {resource: "EXT:e2_core/Configuration/RTE/Editor/SpecialChars.yaml"}

editor:
  config:
    allowedContent: true
    enableContextMenu: false
    forcePasteAsPlainText: true
    clipboard_defaultContentType: 'text'

    toolbar:
      - ['Subscript', 'Superscript', '-', 'SpecialChar']

    wordcount:
      showRemaining: true
      showParagraphs: false
      showWordCount: false
      showCharCount: true
      # Filled by Event based on TCA configuration
      # maxCharCount: 255

    autoParagraph: false
    enterMode: 2 # <br> instead of <p>

    extraPlugins:
      - wordcount
      - specialchar

    removePlugins:
      - resize
      - autogrow

processing:
  overruleMode: nothing
  allowTags:
    - sub
    - sup
    - br

This removes some unwanted features like autogrow, manual resize and context menu.

It also only allows the expected formatting and tags. Furthermore, it configures that line breaks instead of paragraphs should be inserted.

It also imports a global configuration defining the actual special characters needed by editors in this system. TYPO3 v12 will update to CKEditor 5 where special characters UI becomes event better, see: https://ckeditor.com/docs/ckeditor5/latest/features/special-characters.html.

Wordcount is also pre-configured. It will behave like max of input fields and prevent further characters exceeding the maximum count. The count itself is not defined but will be inserted via EventListener based on TCA configuration. One could pre-configure a value within the configuration which can be overwritten via TCA.

The overruleMode: nothing blocks TYPO3 default processing of HTML while inserting and reading from database. This blocks wrapping within <p> tags.

The Event Listener ¶

The Listener looks like this:

<?php

declare(strict_types=1);

namespace E2\E2Core\EventListener;

use TYPO3\CMS\RteCKEditor\Form\Element\Event\AfterPrepareConfigurationForEditorEvent;

/**
 * Transports some of the TCA features into RTE.
 *
 * RTE can limit input based on a count, just like TCA 'max' property.
 */
class TcaToRteConfiguration
{
    public function __invoke(AfterPrepareConfigurationForEditorEvent $event): void
    {
        $rteConfiguration = $event->getConfiguration();
        $fieldConfiguration = $event->getData()['parameterArray']['fieldConf']['config'] ?? [];

        $rteConfiguration = $this->setRows($rteConfiguration, $fieldConfiguration);
        $rteConfiguration = $this->setMax($rteConfiguration, $fieldConfiguration);

        $event->setConfiguration($rteConfiguration);
    }

    private function setRows(array $rteConfiguration, array $fieldConfiguration): array
    {
        $rows = $fieldConfiguration['rows'] ?? 0;

        if ($rows > 0) {
            $rteConfiguration['height'] = ($rows * 6) . 'rem';
        }

        return $rteConfiguration;
    }

    private function setMax(array $rteConfiguration, array $fieldConfiguration): array
    {
        $max = $fieldConfiguration['max'] ?? 0;

        if ($max > 0) {
            $rteConfiguration['wordcount']['maxCharCount'] = (int) $max;
        }

        return $rteConfiguration;
    }
}

And is registered like this (within Services.yaml of the extension):

services:
  _defaults:
    autowire: true
    autoconfigure: true
    public: false

  E2\E2Core\:
    resource: '../Classes/*'

  E2\E2Core\EventListener\TcaToRteConfiguration:
    tags:
      - name: 'event.listener'
        event: 'TYPO3\CMS\RteCKEditor\Form\Element\Event\AfterPrepareConfigurationForEditorEvent'

Combining the pieces ¶

We need to properly configure our TCA column to use the dedicated configuration. The column looks like this:

'tx_e2core_link_label_label' => [
    'label' => $languagePath . 'tx_e2core_link_label_label',
    'config' => [
        'type' => 'text',
        'eval' => 'required,trim',
        'max' => 255,
        'rows' => 1,
        'enableRichtext' => true,
        'richtextConfiguration' => 'minimal-input-field',
    ],
],

This converts our input to a text field. It is still required and gets trimmed. The max and rows are respected via above EventListener. The last two turn the field into an RTE with our dedicated configuration.

The RTE configuration itself is registered within ext_localconf.php:

\TYPO3\CMS\Core\Utility\ArrayUtility::mergeRecursiveWithOverrule(
    $GLOBALS['TYPO3_CONF_VARS'],
    [
        'RTE' => [
            'Presets' => [
                'default' => 'EXT:e2_core/Configuration/RTE/Editor/Custom.yaml',
                'minimal-input-field' => 'EXT:e2_core/Configuration/RTE/Editor/MinimalInputField.yaml',
            ],
        ],
    ]
);

We prefer the merge way as we don't have different ways to configure the $GLOBALS['TYPO3_CONF_VARS']. It is always this way within AdditionalConfiguration.php, LocalConfiguration.php and ext_localconf.php. That allows to copy and paste without adjusting the code.

Our Takeaway ¶

We first thought there must be an extension or we should build an extension. That one should provide a new renderType for TCA input columns. We knew many others have the same issue and one of them should have already done that.
But we didn't find an extension and thought there should be an easier way and came up with our solution. That one is so flexible and easy to integrate that it doesn't make sense to provide an extension but a blog post. Every installation and even some fields have very special needs which now can be covered with different configurations.

We love flexible and simple solutions as they are the best solutions most of the time. To us, it is important to overthink solutions and approaches from the past. It is also critical to not get driven by technology hype or by the feeling to release another extension. Find a proper solution for the actual problem, ignoring other influences from the outside. It is still possible to share those solutions. Either as blog post, video or a small gist or some other way.

Thanks to my co-worker Justus Moroni who once more helped to provide a good solution for a problem.

Thanks to our customer reuter.de who allows us to re-think existing solutions in order to improve editor UX.

Thanks to Codappix GmbH for allowing me to share our solutions on my own blog during working hours.

• https://docs.typo3.org/c/typo3/cms-rte-ckeditor/main/en-us/Configuration/ConfigureTypo3.html - General CKEditor documentation for TYPO3
• https://ckeditor.com/docs/ckeditor4/latest/api/CKEDITOR_config.html - All configuration options of CKEditor
• https://github.com/w8tcha/CKEditor-WordCount-Plugin - Configuration of WordCount plugin for CKEditor
• https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Events/EventDispatcher/Index.html - Event Dispatcher / PSR-14 Events documentation for TYPO3

How I used to think ¶

I always thought it is fine to use outdated versions as long as they are covered by my operating system like Ubuntu. I thought the operating system will invest time to keep those versions secure. I didn't mind to run web projects on older versions and to provide TYPO3 extensions or composer packages for outdated versions.

Sources that proof me wrong ¶

I then discovered those sources:

• https://unixsheikh.com/articles/the-delusions-of-debian.html
• https://www.unixsheikh.com/articles/linux-distribution-long-term-support-might-not-be-what-you-think-it-is.html
• https://repology.org/repositories/statistics
• https://wiki.debian.org/PHP#Notes_on_PHP_and_security

It also feels like the PHP ecosystem is changing. Libraries and tools are only covering official supported PHP versions as listed here: https://www.php.net/supported-versions.php.

My outcome ¶

This leads to issues if you stick to older PHP versions. You might lack security updates as they take some extra time. You might not be able to use some new library or latest versions of libraries including features and fixes.

Furthermore, you create technical debt as you might need to update to newer PHP versions anyway, but with larger code bases in case you keep adding lines of code.

There are tools like rector (https://getrector.org/) which allow you to upgrade your code base to support newer PHP versions nowadays. Also tools like PHPStan (https://phpstan.org/) allow you to check compatibility with PHP versions. It is not as hard to update as it was some years ago.

I recommend checking my Reddit post regarding this blog post, so you get further insights and opinions: https://www.reddit.com/r/PHP/comments/wumtvm/why_legacy_php_versions_maintained_by_os_might/.

You might also be interested in the following blog post: https://php.watch/articles/extend-lifetime-legacy-php.

The idea ¶

Connect the existing pieces. Susis blog post explains how to mock responses already. We only need to adapt for functional tests where we can't easily replace the concrete instance of the Guzzle Client.

This would work if TYPO3 would allow to load a different Services.yaml for Testing context, just like Symfony. But TYPO3 allows us to add handlers to Guzzle via configuration of $GLOBALS['TYPO3_CONF_VARS']['HTTP']['handler'].

The implementation ¶

We created a new PHP Class within the Test namespace. This class provides the API to register itself, to clean things up and to mock the responses:

<?php

declare(strict_types=1);

namespace Codappix\ExampleExtension\Tests\Functional;

use GuzzleHttp\Handler\MockHandler;
use GuzzleHttp\Psr7\Response;
use Symfony\Component\HttpFoundation\Response as SymfonyResponse;

class GuzzleClientFaker
{
    /**
     * @var MockHandler
     */
    private static $mockHandler;

    public static function registerClient(): void
    {
        $GLOBALS['TYPO3_CONF_VARS']['HTTP']['handler']['faker'] = function (callable $handler) {
            return self::getMockHandler();
        };

    }

    /**
     * Cleans things up, call it in tests tearDown() method.
     */
    public static function tearDown(): void
    {
        unset($GLOBALS['TYPO3_CONF_VARS']['HTTP']['handler']['faker']);
    }

    /**
     * Adds a new response to the stack with defaults, returning the file contents of given file.
     */
    public static function appendResponseFromFile(string $fileName): void
    {
        $fileContent = file_get_contents($fileName);
        if ($fileContent === false) {
            throw new \Exception('Could not load file: ' . $fileName, 1656485162);
        }

        self::appendResponseFromContent($fileContent);
    }

    private static function appendResponseFromContent(string $content): void
    {
        self::appendResponse(new Response(
            SymfonyResponse::HTTP_OK,
            [],
            $content
        ));
    }

    private static function getMockHandler(): MockHandler
    {
        if (!self::$mockHandler instanceof MockHandler) {
            self::$mockHandler = new MockHandler();
        }

        return self::$mockHandler;
    }

    private static function appendResponse(Response $response): void
    {
        self::getMockHandler()->append($response);
    }
}

How to use ¶

The new class can be used within tests:

<?php

namespace Codappix\ExampleExtension\Tests\Functional;

use Codappix\ExampleExtension\Command\Import;
use Codappix\ExampleExtension\Extension;
use Symfony\Component\Console\Tester\CommandTester;
use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase as TestCase;

class ImportTest extends TestCase
{
    protected function setUp(): void
    {
        parent::setUp();
        // Ensure the fake guzzle client is used when executing tests.
        GuzzleClientFaker::registerClient();
    }

    protected function tearDown(): void
    {
        // Ensure the guzzle client is cleaning up all leftovers
        GuzzleClientFaker::tearDown();
        parent::tearDown();
    }

    /**
     * @test
     */
    public function importIsFilteredByCompanyName(): void
    {
        // Register two responses with the content of those files.
        // The files only provide the content of the response, no header or status codes.
        GuzzleClientFaker::appendResponseFromFile(__DIR__ . '/ImportFixtures/Guzzle/example.com/api/oauth/token.json');
        GuzzleClientFaker::appendResponseFromFile(__DIR__ . '/ImportFixtures/GuzzleImportIsFilteredByCompanyName/example.com/api/v1/job_market/jobs/GET.json');

        // Execute the actual tests, that code will trigger two requests.
        // The actual code doesn't matter for this example.
        $this->importDataSet('EXT:example_extension/Tests/Functional/ImportFixtures/FilteredJobsByCompanyName.xml');
        $importer = $this->getContainer()->get(Import::class);
        $commandTester = new CommandTester($importer);
        $commandTester->execute([
            'storagePid' => '2',
        ]);
        $this->assertCSVDataSet('EXT:example_extension/Tests/Functional/ImportAssertions/FilteredJobsByCompanyName.csv');
    }
}

We register the fake class within the setUp() method. We also ensure it cleans up itself within the tearDown() method.

The class can now be used from within the Test. We currently offer one public method which allows to set the response via appendResponseFromFile().

Other methods could easily be exposed. E.g. the private method appendResponseFromContent() in order to dynamically build the response content. One also could create the whole response including status code within the test and pass it to a new method of the class.

And one could integrate the history as mentioned in Susis blog post in order to assert that certain requests were made.

Thanks to stadt.werk GmbH who paid us to implement the mocking within one of their extensions.

Thanks to Susi (https://twitter.com/sasunegomo) for providing an inspiring blog post on how to mock the Client itself.

Thanks to Mathias for motivating me to write the blog post.

• Read Susis blog post regarding mocking of HTTP api Responses with Guzzle: https://susi.dev/mock-http-api-responses-with-guzzle-psr-18-psr-7/
• Read Guzzle documentation on testing and mocking of Guzzle: https://docs.guzzlephp.org/en/stable/testing.html and Handlers and Middleware: https://docs.guzzlephp.org/en/stable/handlers-and-middleware.html

Context ¶

The customer switched the agency because the old agency created a very slow website. The website used EXT:ke_search. That itself wasn't a problem. The agency extended the search and destroyed the performance. An initial page load of search site, without actually searching, took over 1 minute.

This new website was build by https://werkraum-media.de/ one of our customers. We did the integration and programming part. Our main goal was to use as few dependencies as possible and stick to TYPO3. Still the website should be feature rich.

We decided to integrate EXT:solr as one of the few dependencies. The website has a lot of content and needs a proper search. Furthermore, EXT:solr allows rendering content lists with additional features. I've already published a post back in 2016 regarding this topic, make sure to read it to get the idea.

We also decided to work on editor experience for TYPO3 backend. Editors were overwhelmed by the complexity of the old TYPO3 installation. That installation used EXT:news as many other installations out there. And as very often, EXT:news was extended for events and other kind of contents. The setup was a mess. We decided to use TYPO3 pages instead of EXT:news. Rendering of lists will be done via EXT:solr instead, detail view is done by opening the actual TYPO3 page.

We added different page types as there are still many types from articles over calls for papers and events. Also, the page structure matches the different areas like book reviews, interviews and such things. Editors will find the same structure as visitors that actual matches the types of content.

Video ¶

Thanks to the customer https://www.soziopolis.de/ who trusted us and allowed us to build such a great website.

Thanks to our direct customer, the TYPO3 agency https://werkraum-media.de/ for trusting us for many years already. It is always a pleasure to work with them.

You might be interested in the following sources:

• Blog post Everything is Content, that can be served via Solr promoting the idea back in 2016

• Blog post TYPO3 Plugins as Content Elements promoting the idea of content elements instead of complicated technical plugins for editors

• TYPO3 extension solr: https://extensions.typo3.org/extension/solr/

The issue ¶

Let me explain the issue first. Hopefully that will ease understanding of the following sections.

TypoScript by concept is meant for Frontend. That concept didn't change, even if TypoScript now is also used in other context. Therefore, TypoScript is still bound to a concrete page, as each page could load different TypoScript. Even contexts without the concept of a page still need to determine the page to load TypoScript.

That's not an issue for backend modules with page tree component. Whenever a page is selected, that one will be passed as an argument to the URL of the module. That way TYPO3 already has a page to use. But that's not the case if no page is selected, or in context of a command where no page is available at all.

Determining the page ¶

TYPO3 needs to determine the page that defines TypoScript to load. TYPO3 in that context means Extbase. ViewHelpers and other components relying on TypoScript often use Extbase TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface to load TypoScript. I'll explain exactly that implementation:

1. ConfigurationManager use the concrete configuration manager based on context, which is TYPO3\CMS\Extbase\Configuratio\BackendConfigurationManager for all contexts except frontend.
2. BackendConfigurationManager first determine the page before loading TypoScript.

The logic to determine the page can be fetched from its getCurrentPageId() method:

1. Use id parameter of either GET or POST.
2. Use the first root page. A root page is determined by the following: is_siteroot has to be set, it has to be in default language and should not be within a workspace. "First" means the first one based on sorting.
   Most probably a page was found at this position and is used.
   
   Humans are not able to determine that in case of nested page trees with root pages on different levels. Those scenarios always force to check the database. That sorting can change at any time, as sorting is based on the actual level. So never relay on that.
3. Determine first TypoScript record which is defined as root. "First" means the first one created.
4. Use 0 as page uid.

How to overcome the limitation ¶

I never found a proper way, there is no API. The best suggestion:

Don't use TypoScript in such contexts. If you need TypoScript, for whatever reason, keep it to a minimum, always include it and don't use any features like conditions.

module. vs plugin. ¶

Extbase populates $this->settings within controllers. Extbase uses plugin. in frontend and module. in backend context. Some extensions add module.tx_extname < plugin.tx_extname in order to have the same settings available in both contexts.

That's specific to extension / plugin settings fetched by the API.

• Injecting TypoScript Settings with Dependency Injection since V10.
• Injecting TypoScript Settings with Extbase ObjectManager prior v10.

Don't expect to many explanations here. These are already available, see links at bottom. Instead, this blog post contains concrete examples as reference and copy & paste.

Inject DB Connection and QueryBuilder ¶

Given the new Dependency Injection, one can inject a concrete QueryBuilder instance, or Database Connection. As TYPO3 allows configuring multiple different database servers, developers need to fetch a proper instance based on the concrete database table. This involves some unnecessary code, which can be removed thanks to the Dependency Injection.

services:
  _defaults:
    autowire: true
    autoconfigure: true
    public: false

  DanielSiepmann\Tracking\:
    resource: '../Classes/*'

  dbconnection.tx_tracking_pageview:
    class: 'TYPO3\CMS\Core\Database\Connection'
    factory:
      - '@TYPO3\CMS\Core\Database\ConnectionPool'
      - 'getConnectionForTable'
    arguments:
      - 'tx_tracking_pageview'

  querybuilder.tx_tracking_pageview:
    class: 'TYPO3\CMS\Core\Database\Query\QueryBuilder'
    factory:
      - '@TYPO3\CMS\Core\Database\ConnectionPool'
      - 'getQueryBuilderForTable'
    arguments:
      - 'tx_tracking_pageview'

  DanielSiepmann\Tracking\Domain\Repository\Pageview:
    public: true
    arguments:
      - '@dbconnection.tx_tracking_pageview'

The corresponding PHP could look like:

<?php

namespace DanielSiepmann\Tracking\Domain\Repository;

use TYPO3\CMS\Core\Database\Connection;

class Pageview
{
    /**
     * @var Connection
     */
    private $connection;

    public function __construct(
        Connection $connection
    ) {
        $this->connection = $connection;
    }
}

Inject TypoScript Settings ¶

Back in the old days, you could inject the TypoScript Settings into Extbase classes, see my old blog post. Given the new Dependency Injection, those can be injected into all classes, with no additional code inside the class:

services:
  _defaults:
    autowire: true
    autoconfigure: true
    public: false

  Vendor\ExtName\:
    resource: '../Classes/*'

  extbaseSettings.ExtName.PluginName:
    class: 'array'
    factory:
      - '@TYPO3\CMS\Extbase\Configuration\ConfigurationManager'
      - 'getConfiguration'
    arguments:
      $configurationType: 'Settings'
      $extensionName: 'ExtName'
      $pluginName: 'PluginName'

  Vendor\ExtName\Domain\PeriodCreation\DataHandler:
    arguments:
      $settings: '@extbaseSettings.ExtName.PluginName'

The TypoScript settings are injected as plain PHP array into the constructor argument $settings.

Inject Extension Configuration ¶

TYPO3 offers a new API to retrieve extension configuration. This can be used as a factory to provide options via dependency injection. This might be handy under some circumstances, but there might be reasons to inject the API itself.

services:
  _defaults:
    autowire: true
    autoconfigure: true
    public: false

  Vendor\ExtName\:
    resource: '../Classes/*'

  extensionconfiguration.ext_key.title:
    class: 'string'
    factory:
      - '@TYPO3\CMS\Core\Configuration\ExtensionConfiguration'
      - 'get'
    arguments:
      - 'ext_key'
      - 'title'

  Vendor\ExtName\Namespace\Class:
    public: true
    arguments:
      $config: '@extensionconfiguration.ext_key.title'

This example injects a single property from extension configuration. Note that property class needs to be set to the appropriate type. The $config argument of the constructor needs to have the same type hint.

Another example would be to inject the whole configuration array. That would then be of type array and remove the 2nd argument from factory:

services:
  _defaults:
    autowire: true
    autoconfigure: true
    public: false

  Vendor\ExtName\:
    resource: '../Classes/*'

  extensionconfiguration.ext_key:
    class: 'array'
    factory:
      - '@TYPO3\CMS\Core\Configuration\ExtensionConfiguration'
      - 'get'
    arguments:
      - 'ext_key'

  Vendor\ExtName\Namespace\Class:
    public: true
    arguments:
      $config: '@extensionconfiguration.ext_key'

Thanks to Nikita Hovratov for providing feedback regarding 2nd example of Inject Extension Configuration. It was wrong as it contained extensionconfiguration.ext_key.title instead of extensionconfiguration.ext_key. It was updated 2021-08-17.

That are some concrete examples. For further information, check:

• https://docs.typo3.org/m/typo3/reference-coreapi/master/en-us/ApiOverview/DependencyInjection/Index.html
• https://usetypo3.com/di-and-events-example.html
• https://usetypo3.com/dependency-injection.html

Using Dependency Injection ¶

The BIG change with 10.4 is dependency injection through the whole code base and usage of Symfony container, and therefore configuration of dependency injection.

Actually older versions already have dependency injection which you can use. And that can be migrated via TYPO3 rector, once you update. Let me explain a bit more, before showing a concrete example, because there are two things to understand, which you also need to understand for 10.4 anyway.

1. TYPO3 has some "entry points" into your Code, such as hooks, user functions. Those entry points always use TYPO3 API like GeneralUtility::makeInstance to create instances of your classes. You don't have dependency injection within any TYPO3 version in such cases. In order to get dependency injection within 10.4, you need to explicitly mark those classes to be public. Find out more at docs.typo3.org.

2. TYPO3 has deprecated usage of Extbase ObjectManager, which provides Dependency Injection. One might think adding usages of that class will complicate updates, but TYPO3 rector will do the job for you. Don't fear that change in case you follow this post.
Once you understood those two points about TYPO3 10.4, you should have all you need to prepare older code base for an update.

Let's have a look at two concrete examples, all stripped down to what matters.

First concrete example for DI ¶

First, let's take a look at a concrete user function, which is an entry point from TYPO3 to our code base.

<?php

namespace E2\E2Core\Export\UserFunction;

use E2\E2Core\Export\JsonSerializer\JsonEncode;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Extbase\Object\ObjectManager;

class Convert
{
    /**
     * @var JsonEncode
     */
    private $jsonEncoder;

    public function __construct(
        JsonEncode $jsonEncoder = null
    ) {
        $this->jsonEncoder = $jsonEncoder ?? GeneralUtility::makeInstance(ObjectManager::class)->get(JsonEncode::class);
    }

    public function typo3LinkReference(string $link): string
    {
        return json_encode($this->jsonEncoder->normalizeTypo3Link($link));
    }

    public function typo3FilePath(string $filePath): string
    {
        return $this->jsonEncoder->normalizeFilePath($filePath);
    }
}

This class offers two user functions and already is using dependency injection. It expects an JsonEncode instance on creation. The little change compared to TYPO3 10.4? This dependency is marked optional by using = null. That's necessary as older TYPO3 versions don't resolve dependency for us. Therefore, we use GeneralUtility::makeInstance(ObjectManager::class)->get(JsonEncode::class); to fetch the dependency.

The usage of ObjectManager is part of our second concrete example.

Following this concrete example, our code is already prepared for 10.4 native dependency injection. We also can write our tests and inject dependencies. No need to change those tests for 10.4.

Once we update to 10.4, we will use TYPO3 rector to remove the = null and ?? GeneralUtility … call. No manual work is involved in updating this code.

Second concrete example for DI ¶

The second example is code from within our code base. We expect that this class is already injected, or at least fetched via Extbase ObjectManager.

<?php

namespace E2\E2Core\Export\JsonSerializer;

use E2\E2Core\Service\ImageServerService;
use E2\E2Core\Service\TypolinkService;

class JsonEncode
{
    /**
     * @var ImageServerService
     */
    private $imageService;

    /**
     * @var TypolinkService
     */
    private $typolinkService;

    public function __construct(
        ImageServerService $imageService,
        TypolinkService $typolinkService
    ) {
        $this->imageService = $imageService;
        $this->typolinkService = $typolinkService;
    }

    public function normalizeTypo3Link($value)
    {
        if (is_string($value) === false) {
            return $value;
        }

        $result = $this->typolinkService->process($value);
        return $result ?? $value;
    }

    public function normalizeFilePath(string $value): string
    {
        return $this->imageService->generateFileSrc($value);
    }
}

That's the class which is used by the first example. As instances are always injected, or fetched by ObjectManager, there is no need for the more complex code from first example. This code will not change in 10.4 and just work.

Using PSR-14 events ¶

TYPO3 10.4 introduced the PSR-14 events. Until then, there were hooks and signal / slots from Extbase. The events will replace those solutions step by step.

Some concepts can already be used within older TYPO3 code base, easing actual code in older versions and reducing amount of work during updates.

One can follow the new TYPO3 documentation regarding events, and already create events. Those events encapsulate information and provide APIs. (Let's not talk about using technical implementations for foreign concepts here). Events are simple classes that are passed through the dispatcher. The same is already possible in older versions. Only the dispatcher needs to be replaced during an update to 10.4, keeping the rest as is.

Concrete example for events ¶

Let's have a look at a concrete example on how to implement an event in 8.7. I'll not go into detail here, because you can find information at docs.typo3.org for 10.4 already. This example should only show how to use it in older versions.

private function jsonEncode($object): string
{
    $normalizer = new ObjectNormalizer();

    $event = new InitializeJsonEncode($object);
    $this->dispatcher->dispatch(__CLASS__, 'jsonEncodeNormalizer', [$event]);
    $normalizer->setCallbacks($event->getCallbacks());
    $normalizer->setIgnoredAttributes($event->getAttributesToIgnore());

    $serializer = new Serializer(
        [$normalizer],
        [new JsonEncoder()]
    );

    return $serializer->serialize($object, 'json');
}

The above method creates a new instance of an event. The event is dispatched by Extbase dispatcher. Once you update, replace the dispatcher and the line dispatching the event. Everything else can stay.

Let's see the counterpart, registering and using the event.

public static function register()
{
    $dispatcher = GeneralUtility::makeInstance(Dispatcher::class);

    $dispatcher->connect(
        JsonSerializer::class,
        'jsonEncodeNormalizer',
        JsonEncode::class,
        'initializeNormalizer'
    );
}

public function initializeNormalizer(InitializeJsonEncode $event)
{
    $event->addCallback('meta', [$this, 'normalizeFeUserGroups']);

    $object = $event->getObject();

    if ($object instanceof HasExtbaseFileReferences) {
        $this->initializeCallbackForAttributes(
            $event,
            'fileReferenceProperties',
            'normalizeExtbaseFileReference'
        );
    }
    if ($object instanceof HasTypo3LinkReferences) {
        $this->initializeCallbackForAttributes(
            $event,
            'linkProperties',
            'normalizeTypo3Link'
        );
    }
}

The first method is called from ext_localconf.php to connect the listener to the dispatcher. This will be removed with 10.4. Instead, the registration will be done via configuration inside Services.yaml. The public method which receives the event will stay when updating to 10.4, no change will be necessary.

Conclusion ¶

If you follow recent development, e.g. by reading the changelog, you can understand new concepts and ideas. By following these new concepts within older projects, updates will become smoother. You can already learn new concepts, which will make working with recent versions easier. Beside all of that, your code already follows those concepts, which should make it "better".

In some cases updates become even easier, thanks to preparation and usage of TYPO3 rector.

Thanks to Naderio for pushing me to write this blog post.

Thanks to Sebastian Schreiber for creating, maintaining and developing TYPO3 rector.

You might also be interested in the following sources, which touch the same topics:

• TYPO3 v10 feature outlook
• TYPO3 rector — auto migrate your TYPO3 code base
• https://docs.typo3.org/m/typo3/reference-coreapi/10.4/en-us/ApiOverview/DependencyInjection/Index.html
• https://docs.typo3.org/m/typo3/reference-coreapi/10.4/en-us/ApiOverview/Hooks/EventDispatcher/Index.html
• PSR-11: Container interface
• PSR-14: Event Dispatcher

The technical change ¶

The new feature was introduced in TYPO3 10.0 as an actual deprecation of the old way to register plugins and modules when working with Extbase. Refer to TYPO3 Changelog 10.0 Deprecation #87550 to have a detailed explanation of the change.

Before 10.0 one registered the vendor independent from the actual controller name. Controller class names were build by Extbase, based on their provided alias and the vendor. Since 10.0 there is no more vendor but fully qualified class names (=FQCN). An example configuration of a plugin looks like this:

\TYPO3\CMS\Extbase\Utility\ExtensionUtility::configurePlugin(
    $extensionName,
    $pluginName,
    [
        \DanielSiepmann\ExtensionName\Controller\Frontend\CalendarController::class => implode(',', [
            'month',
            'week',
            'day',
        ]),
    ]
);

The benefits ¶

This allows to use controllers from all possible namespaces. Therefore a plugin can be configured to use a controller from a foreign Extension. That Extension should be required in ext_emconf.php and composer.json.

Why is that actual a benefit? This small change allows to reuse existing controllers. The example above contains a calendar controller to view a month, week and day. The extension also has some models which are part of the controller actions. Another extension which might need a calendar could now define a dependency and configure their own plugins to reuse this controller and models.

That way the extension does not need to re implement logic to build up data structure for calendars. Instead it can focus on their own parts.

That was a very technical benefit. But there are some more improving the experience of integrators and developers:

Each plugin has its own namespace. Combining units of the installation into a single plugin will assign a single new namespace to that feature set. That way configuring routing and building links becomes way easier. Same goes for TypoScript configuration.

The conclusion ¶

In order to make all that work, controllers and models should be selfincluded. They should "just work" and solve a single issue. Otherwise one will easily mix up things and create bloated plugins.

As templates belong to the extension namespace configuring the plugin, the new extension can provide custom templates with rendering for month, week and day. Each of them can contain links back to other actions, e.g. of controllers provided by the second Extension.

Also in case the calendar Extension makes it possible to easily attach custom records to these models, they would be available in the templates already.

Possible Outlook ¶

I'm not investing time in developing Extbase and therefore my outlook might not match the actual future. But in my opinion Extbase is changing and so should developers. It looks like most Extensions are built without thinking in a Unix / composer way. Instead of putting everything into an Extension, it would make more sense to focus on one thing and reuse existing components, either TYPO3 Extensions or composer packages.

That way issues should be solved way faster. Developers would focus on the important things.

Imagine an calendar Extension that would do all the calendar stuff. Another extension would provide data and logic to buy tickets for events. Combine both with a third extension, or add support for the calendar Extension within the ticket Extension. Without further overhead, all would have less code with more features. That would make Extensions easier to maintain.

Maybe some "Extensions" no longer will be Extensions, but pure composer packages providing some specific logic that can be reused by TYPO3 Extbase Extensions, as well as other systems.

Thanks to Eike Starkmann for mentioning further benefits.

• Changelog entry regarding the change: https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/10.0/Deprecation-87550-UseControllerClassesWhenRegisteringPluginsmodules.html
• Tweet mentioning the idea to write this blog post: https://twitter.com/daniel_siepmann/status/1319900536683040768

Older versions of the extension did not properly initialize TYPO3 backend, e.g. when trying to edit an record. This could lead to an unusable state.

This version should fix that and always allow to edit records.

The issue is fixed for buttons in content, as well as the admin panel.

Contained bugfixes ¶

The following two bugfixes are part of the release:

• 3450ed6 Fix implementation of RequestEnricherInterface
• 3b3f156 Fix none working backend

Update now ¶

The current version can be installed via composer:

composer req friendsoftypo3/feedit:^10.0.2

It can be downloaded from GitHub: https://github.com/FriendsOfTYPO3/feedit/releases/tag/v10.0.2.

Features ¶

Right now everyone can use the extension. Either require from composer, or download the extension from GitHub and place it inside of typo3conf/ext folder.

The extension provides server side tracking without any JavaScript or Cookies. Still it lacks many of the powerful features provided by solutions like Google Analytics.

The extension allows to track visits of pages, as well of specific records. The statistics will be displayed via widgets and new ext:dashboard.

Integrators can fine control which requests should be tracked. E.g. by default visits with active backend user login are not tracked.

Development ¶

The development happens on GitHub (https://github.com/danielsiepmann/tracking) and is fully transparent and open source. All changes are made via pull requests. Also we have 100% code coverage (linewise) using Unit and Functional tests.

Contribution ¶

In case you wanna contribute, you can create issues or send in pull requests. Also we do paid development for the extension, in case you miss a specific feature.

Thanks to our sponsors so far which are web-to-date: https://www.web-to-date.com/ as well as werkraum_ media: https://www.werkraum-media.de/. They requested and paid me to extend and finish the extension for public usage.

I've initially published information about the extension on a subpage.

You can also find the official documentation at docs.typo3.org.

And source code, pull requests and issues can be found on GitHub: https://github.com/danielsiepmann/tracking

The composer package is available at: https://packagist.org/packages/danielsiepmann/tracking.

The blog posts expects a working composer environment. It will not explain how to migrate or setup a project using composer.

The blog post will follow a concrete example of an TYPO3 extension that is delivered to the public. The steps can be adapted to other PHP and TYPO3 projects.

Installation ¶

The package can be installed in different ways. Those are all mentioned in projects readme at https://packagist.org/packages/maglnet/composer-require-checker as well as on https://github.com/maglnet/ComposerRequireChecker. I'll add the package as development dependency to the extension:

composer req --dev maglnet/composer-require-checker

Depending on existing dependencies an older version might be necessary. E.g. "maglnet/composer-require-checker:2.0.*" which supports older versions of symfony/console. In order to ensure the package detects well known PHP native extensions and function, add PHP as dependency:

composer req "php:*"

Adjust the statement in case only specific PHP versions are supported.

Adding configuration file ¶

This step is optional, but suggested for TYPO3 extensions.

The package by default will check all files configured via composer autoloading. As TYPO3 extensions provide further files, e.g. configuration files, those need to be configured. This can happen with a custom JSON configuration file. No specific location is defined, it can be located in a subfolder or the projects root.

In the following example the file will be located in the extensions root folder and named dependency-checker.json. The contents of the file differ from project to project but might look like the following:

{
    "scan-files" : [
        "*.php",
        "Configuration/TCA/*.php",
        "Configuration/TCA/Overrides/*.php"
    ]
}

An example can be found inside the projects repository: https://github.com/maglnet/ComposerRequireChecker/blob/2.1.0/data/config.dist.json. The above example will configure the tool to scan further files. All PHP files in project root, as well as some PHP files configuring TCA.

Executing ¶

Once everything is setup, one can execute the tool, this is done via:

./vendor/bin/composer-require-checker

As a configuration file should be read, the full execution looks like:

./vendor/bin/composer-require-checker check --config-file dependency-checker.json

Last hint ¶

The project aims to find missing dependencies in production code. It will not be extended or support require-dev, suggest or autoload-dev.

The package can be found at:

• Packagist: https://packagist.org/packages/maglnet/composer-require-checker
• GitHub: https://github.com/maglnet/ComposerRequireChecker

The following TYPO3 Extension uses the package and can be used as an example:

• Example TYPO3 Extension: https://github.com/DanielSiepmann/tracking

Concepts of frontend editing ¶

First of all let me explain two different concepts how to edit frontend in TYPO3.

First Concept: Inline Editing ¶

The first concept is provided by EXT:frontend_editing. Some of you might know this concept from other systems like Neos. I would call this concept "live editing" or "inline editing". The basic idea is to wrap your existing frontend with all the tools you need. E.g. add a page tree to the left, an inspector to the right. This is more or less the same concept as word processors like LibreOffice or Microsoft Word use. You have a real live view of your content and edit it inline. Every change is visible in the context of your rendered website in current view port.

This approach is very intuitive for editors that didn't work with any older CMS. There is no system to learn. You log into the system and see your website. Go ahead and edit whatever you want to change. For more fine grained adjustments there is the inspector, which let's you edit the ratio of an image and other specific stuff.

Second Concept: Structured Editing ¶

One strength of TYPO3 is the structured content. In TYPO3 you have different types of content elements, all with their configured backend form. More or less all content information are saved in their own context and database fields. In order to keep this structure and still provide frontend editing, you would need to add information to your view and keep it in sync.

EXT:feedit still can provide a benefit, by not providing the same UX (=User Experience) to the editor as the concept above. Instead it will place little icons to interact with existing TYPO3 functionality. These icons are added to the admin panel and before or after all content elements and records. These Icons allow editors to hide or unhide, to sort and add elements, as well as opening an edit form for a content element, record or the page.

How EXT:feedit works ¶

Now that we are aware of different concepts and the features of EXT:feedit, how does it work?

Frontend editing is provided in three different areas. The first one is the admin panel, which allows editors to work with the current page. The admin panel allows to open the current page in TYPO3 backend, edit the page, move the page, open list view or add new pages.

Also the admin panel allows to check two options. The first option is called "Display edit icons", the second one is called "Display edit panels". Both of them will be explained in more detail below, as they need to be configured by an TYPO3 integrator.

Features of feedit ¶

Some features were already mentioned. Let's dig deeper into the features provided by EXT:feedit.

For an editor there is a single feature available by default: He can edit the current page via the admin panel.

All other features need to be configured by an integrator. The configuration is pretty easy, and TYPO3 EXT:fluid_styled_content is already pre configured.

The screenshots below will provide a better overview of the provided features.

Screenshots of EXT:feedit ¶

The editPanel ¶

Example is from this website:

lib.contentElement = FLUIDTEMPLATE
lib.contentElement {
    stdWrap {
        editPanel = 1
        editPanel {
            printBeforeContent = 1
            allow = edit, new, delete, move, hide
        }
    }
}

That's one of the simplest approaches. The integrator enables the editPanel and configures it. Per default the panel will be displayed after content elements. I switch the behavior using printBeforeContent = 1.

Also there are different actions one can allow for the panel:

• edit
  Opens the edit form. This is the same form as when opened from TYPO3 backend.
• new
  Opens the default content element (first item in drop down) in editing mode.
  This could be improved in future, to use the "New Content Element" wizard.
• delete
  Asks for confirmation before deleting the record.
• move
  Allows editors to move the record, the same way as in list view of TYPO3 backend.
• hide
  Asks for confirmation before hiding the record. If hidden records are displayed (which you can activate through admin panel), one can also enable the record again. There is no need for confirmation to enable a record.

The confirmations can be turned off, this is configurable via TSConfig options.alertPopups.

There are also options to configure the editPanel for custom records, e.g. news. As I follow the "pages approach", I don't have an example from this website yet.

The editIcons ¶

Beside editPanel there is also editIcons. While the panel adds a full blown toolbar, the icon is a single icon that allows editing of the record.

The icon not only prevents sorting, adding, deletion, etc. It also allows integrator to fine grain configure which fields to show. An example from this website looks like this:

lib.contentElement = FLUIDTEMPLATE
lib.contentElement {
    stdWrap {
        editIcons = tt_content:
        editIcons {
            beforeLastTag = -1
            iconTitle = Edit specific fields of content element
        }
    }
}

tt_content.text =< lib.contentElement
tt_content.text {
    stdWrap {
        editIcons := appendString(header, header_layout, layout, bodytext)
    }
}

First of all editIcons are enabled for all content elements. I also configure to add the icon before the content element and provide a title. As every content element might have other fields that need to be edited, these are configured on a per content element level. In above example only the following fields are shown and editable for text content elements: header, header_layout, layout, bodytext.

State of the extension ¶

For a very long time the extension was part of TYPO3 core. Since v10 the extension was moved to FriendsOfTYPO3 GitHub account and is available at https://github.com/FriendsOfTYPO3/feedit. The extension is also available as a composer package friendsoftypo3/feedit.

I've invested some hours to make it compatible with current TYPO3 master (v10) and to add two smaller features.

As the extension doesn't need much code and interacts in a clean way with TYPO3, it is very likely to stay for future versions.

I would highly appreciate more love to this hidden gem of TYPO3, which in my opinion improves the editing experience. It allows fast fixes to content and even provide a nice experience when creating new blog posts. All of that without heavy configuration.

This blog post was written using the extension.

• TypoScript cObject EDITPANEL: https://docs.typo3.org/m/typo3/reference-typoscript/master/en-us/ContentObjects/Editpanel/Index.html
• TypoScript stdWrap.editPanel: https://docs.typo3.org/m/typo3/reference-typoscript/master/en-us/Functions/Stdwrap.html#editpanel
• TypoScript stdWrap.editIcons: https://docs.typo3.org/m/typo3/reference-typoscript/master/en-us/Functions/Stdwrap.html#editicons
• TSconfig options.alertPopups: https://docs.typo3.org/m/typo3/reference-tsconfig/master/en-us/UserTsconfig/Options.html#alertpopups
• Extension in TER: https://extensions.typo3.org/extension/feedit
• Extension on packagist.org: https://packagist.org/packages/friendsoftypo3/feedit
• Extension on GitHub: https://github.com/FriendsOfTYPO3/feedit
• TYPO3 Extension: feedit on my site.

This block post should prove that TYPO3 sticks to his roots ans strengths. The biggest strength of TYPO3 is the basic idea of extensions. Do not build a single monolith, instead follow Unix philosophy. Each component should do a single thing very well. By adding clean APIs like PSR-15 or PSR-14 you can combine all those building blocks and create the system your client needs.

Also TYPO3 is going in a bright future. With all new standards and features, you can build awesome things. It is easier then ever to integrate stuff, leaving more time for other important parts, e.g. contribution to TYPO3.

What are you going to do with all the new awesome technical features? Which benefits will you create out of them for your customers or your own company?

TYPO3 version 10 has a new topic. Accordingly to the official news named "X Marks the Spot - TYPO3 v10.0 is here", "TYPO3 Version 10.1 - On the High Seas" and "TYPO3 Version 10.2 — Treasure Hunting!", the focus should be moved to everyone. Its not about the TYPO3 core team or any other team, its about the whole community. Everyone is involved and everyone can mark his own spot, sail on the high seas and find treasures.

This blog post should motivate you to find treasures and mark your own spot. Which feature will you find useful and what do you gonna do with it?

Did you know this website is running TYPO3 CMS v10.3 already?

PSR-14 Events aka. server side tracking ¶

It was never easier to attach code to events. This opens possibilities e.g. for easier logging or tracking. Wanna know how many people submitted your contact form? Or how many orders where placed? With GDPR and foreign services, this is the time to get back to server side tracking. With a very simple API, you could just attach any internal Event to the API and record what happens in your system. It should also be possible to attach any Event to your API. That could also open possibilities for error reporting. Improve your exceptions with information from past events. That way you always know what was going on.

Back in the old days people just copied and modified 3rd Party Extensions. With ease of PSR-14, it is even easier to add and use Events. Wanna add a new registered user to a 3rd Party Service, e.g. newsletter system? Just use EXT:form to register a user, use hooks or form finisher and submit the information to another system.

EXT:dashboard ¶

Agencies have a lot of customers. Those customers open issues, requesting features or reporting bugs. Some customers might have a service level agreement with some hours per month. Ever wanted to communicate all those information to your customers? Use the new dashboard. Create your own dashboards, connect your issue tracker and invoicing tool. Display how many issues are open, how many hours are used in the current month. Display open and payed invoices to your customers, right within TYPO3.

Combined with the mentioned server side tracking, or existing solutions like Google Analytics, you could also add important facts right there. No more need for your customer to log into multiple systems. TYPO3 can become the single system he needs to care about. Just integrate all necessary information. E.g. delivered newsletters, current solr index queue, etc.

Add some buttons to your widgets to allow users to take action. E.g. rank specific pages higher in a solr index queue. Or go into more detail in the system log, regarding current raised system errors shown in a dashboard.

The following screenshots demonstrate the current state of the dashboard.

PSR-15 Middlewares ¶

Sure, this one was delivered with TYPO3 CMS v9 LTS already, still this might be new to some of you, therefore I'll add my thoughts about that feature, too.

Middlewares are a stack of Code that is executed for each incoming request and outgoing response. That makes them more or less the heart of each web application. In combination with PSR-7 request and response, they allow developers to integrate more or less every clean system. For example TYPO3 can be run as a proxy before your shop system. Add some configuration and forward all incoming requests to your shop application. That way you can have a single entry point, and TYPO3 does not only handle multiple languages and websites, but also multiple applications.

The same reasons also allow the developers to integrate foreign frameworks, e.g. Slim or any other PHP framework. You need a website with an area where you build a web app, but maybe still need the layout and content of TYPO3? You could use a middleware and attach a php framework to integrate content. You could also use another framework to deliver an RESTful API or something else. Add some logic to a single middleware to detect whether requests should go to TYPO3 or your rest api integration.

Beside that awesome possibilities, middlewares also ease to adjust your request and response. E.g. adding specific header or resolving data. Let's say your TYPO3 is running behind a Proxy, e.g. Cloudflare or Varnish. The middleware can add further information that can be used by your Proxy. They can also resolve information added by the Proxy.

That were all very technical features, that would allow to integrate further systems and approaches. Yet there is another more basic example. In "PSR-14 Events aka. server side tracking" we already talked about server side tracking. A middleware is a perfect place to add basic page view tracking to your system. Just add an DB Entry for each request, with all necessary information. Adding Symfony Expression Language and you already have a black list configuration at your hand. E.g. disable tracking for active backend logins or crawler.

Notification API with actions ¶

The notification API was extended. Some might know the new feature from iOS or other operating systems. Each notification can now have attached actions.

Imagine an error in your TYPO3 installation. Instead of showing an exception, show a nice error page and raise a notification. Attach an "open issue" action and let the user create an issue in your issue tracker. Combine that one with ext:dashboard to display all those open issues and their current state.

It also allows you to make users decisions, without loosing context. One of our customers has a backend module that exports TYPO3 content to filesystem. There might be situation where things go wrong. Right now we display a modal with some actions. In future we could use a smoother user experience. Instead of a disturbing modal interrupting the current flow, we could display an notificitation which has actions attached.

Fluid based PageLayoutView ¶

There was some love in a specific PHP Class, the PageLayoutView. This class generates the output of the "Page" backend module. I guess that module is the most used module within TYPO3. More or less all developers, integrators and editors work with this module. It is one of the biggest benefits for Editors. The module displays a mock like version of your frontend and allows to place content exactly where you wanna have it.

Still it was very old and integrators had a hard time to add custom preview for more complex content elements. Also extensions like Flux and Gridelements adjusted the module to make user experience better and to add nested content elements. Some people like that, some don't. There are different approaches to the user needs.

With the upcoming LTS v10, this situation was changed. After a refactoring improving the code base, a rewrite was done. Beside the original implementation, a new feature toggle allows to switch the implementation. Integrators can now change every single part through Fluid, as the whole module is now generated through Fluid. Adding a custom preview is just a matter of adding the Fluidtemplate and configure template paths, as done for the Frontend. Also each existing template can be replaced. Custom renderings can be added in addition, in order to allow developers to add custom PHP code upfront to prepare data, e.g. add information from an API.

Native nested content elements ¶

Beside all that awesomeness, which will make live of integrators easier, and improve user experience of editors, that might come with another feature: Native nested content elements.

Thanks to the new code base, it is possible to configure layouts for content elements. That layout will be passed as structured information to the mentioned fluid templates. This allows native rendering of nested content elements inside the page module. A very long wished feature comes true for all developers, agencies and customers.

Some of you might ask: What about the structured content initiative? They still work on a big picture improving the overall editor experience for the upcoming versions. And they are in tight contact with developers working on that parts. Saying that, it is what makes TYPO3 great, a community effort.

Example extension "tracking" ¶

In order to test combinations of those features, I've created the extension "tracking". You can check the extension on GitHub: https://github.com/danielsiepmann/tracking and find further information on a separate site.

The extension demonstrates how to collect data from requests, and displaying such data through new widgets. It already combines EXT:dashboard and PSR-15 middlewares. Also PSR-11 dependency injection is heavily used.

Thanks to Kevin Appelt. The paragraph regarding "Nested content elements" was split up from "Fluid based PageLayoutView". Also a note was added that this feature is not yet released, together with further information.

Check the release news

• TYPO3 News "X Marks the Spot - TYPO3 v10.0 is here"
• TYPO3 News "TYPO3 Version 10.1 - On the High Seas"
• TYPO3 News "TYPO3 Version 10.2 — Treasure Hunting!"
• TYPO3 News "TYPO3 Version 10.3 — Almost There"
• Development Roadmap and release dates

And official documentation

• TYPO3 ChangeLog
• PSR-14 Events at docs.typo3.org
• Request Handling / Middlewars PSR-15 at docs.typo3.org
• Notification API at docs.typo3.org

And further resources

• Example extension "tracking" on GitHub and on this site.

This one is rather a short blog post. It will not go into detail and explain the used tools and concepts. Instead it is more or less used to keep and share the knowledge. I find it especially necessary if you are working with german TYPO3 hoster mittwald.de.

Of course one can also use SSH and some database GUIs instead, which will do exactly the same. Just I like the CLI approach.

Creating and using the tunnel ¶

ssh -L <local-port>:<database-server>:<remote-port> -N <server-used-for-tunnel>

So an example to create a tunnel to the database connection db1234.dbserver.com with Port 3306, which is only available to example.com could look like this:

ssh -L 3307:db1234.dbservers.com:3306 -N example.com

After the tunnel is created, the local port 3307 is passed through the tunnel. This enables the following mysqldump execution:

mysqldump -u <username> -P 3307 -h 127.0.0.1 -p <dbname> > dump.sql;

The important part is to provide -P 3307 -h 127.0.0.1, in order to use the tunnel.

• https://stackoverflow.com/questions/12031124/mysqldump-via-ssh-tunnel
• SSH Man Page: https://linux.die.net/man/1/ssh

This will only cover Extbase plugins, as most extensions only provide Extbase nowadays. But it also works, partly, for pibase extensions. The basic idea dates back to 2018, when I first started to work on this. We now make use of this concept within an actual project, so this covers not only abstract concepts, but real world examples.

Target audience ¶

This blog post requires some TYPO3 knowledge in order to understand everything. This post targets integrators and developers, who already know how to write and use TypoScript, TSconfig, Fluid and TCA configuration. You should also know what FlexForms are.

Also the official TYPO3 documentation section Adding your own content elements is required in order to follow this blog post. This post will provide a complete example, but will not explain every taken step in order to create the new content element. Instead we will focus on the plugin becoming a regular content element.

What is a TYPO3 plugin? ¶

To understand the whole blog post, one needs to understand the basics of plugins within TYPO3.

TYPO3 itself is nothing then a collection of so called “Extensions”. An extension is something that extends TYPO3 in any way. This can either be done by providing plugins and custom PHP code, or by providing CSS, JS, Fluid Templates, Hooks, or anything else. Within this post, we will only cover a specific aspect of plugins.

Plugins are a way to integrate custom PHP logic into TYPO3 for frontend websites. An editor is able to insert a new content element of type “Insert Plugin”, where he can select the specific plugin. This plugin can be something like “List news” or “List events”. A plugin can also be a search form or search result or some other kind of form. In the end, a plugin can be anything.

Most extensions provide plugins out of the box. Most likely you will have a single plugin per extension. The extension author allows the editor to select further configuration options through the content element, via so called FlexForms. E.g. the editor can select the “mode”, e.g. “list” or “detail” for something like news.

Within TYPO3 Extbase, a plugin consists of the following:

• A title for the editor within the TYPO3 backend
• An optional icon within TYPO3 backend
• TypoScript defining the rendering of the plugin
• A combination of callable controllers and actions
• A combination of non cached callable controllers and actions
• An identifier, so called “plugin signature”
• An optional FlexForm for further configuration via editors
• An optional “New Content Element Wizard” entry for new content elements

Why adding plugins for existing extensions? ¶

So extensions already provide plugins, why should one add further plugins to existing 3rd party extensions?

Example 1 EXT:solr + news ¶

Let’s assume there is a TYPO3 installation with a search and news. The search is provided by EXT:solr, and news are implemented using the “Custom Page Type approach™”, see (Blog Post: Everything is Content, that can be served via Solr and TYPO3 Documentation Page Types). The news should be displayed by using EXT:solr, as there is no need for another extension, in this use case.

A typical use case would be to display a list of recent news, e.g. the five recent news on the startpage. Maybe also some pre filtered news should be displayed on sub pages, e.g. only news regarding new products or news regarding the company. All those use cases are solved by using EXT:solr.

Of course one could now add TypoScript to pages to configure EXT:solr to start in filter mode instead of search mode. Also filters can be added to only show news records from these categories. This is not that flexible. The editor is not able to add new “News listings” to further pages, as TypoScript is involved.

It would be better if the integrator can add a new plugin “news” within the “Sitepackage™” of the installation. This plugin duplicates the existing plugin, provided by EXT:solr.

Benefits of this approach would be:

• Instead of keeping the result action none cacheable, it can define that this action should be cacheable.
• Also a new plugin allows to add a different FlexForm to this plugin. This FlexForm can provide a drop down with possible categories, or allows an editor to define how many news should be displayed. Thanks to Extbase conventions, all options available within TypoScript settings section can be used within FlexForms. Due to a different plugin signature, the plugin can be configured differently via TypoScript.

This new Plugin speeds up the delivery of the page, as it’s fully cached. Also an editor can now add a “news” content element and select the specific category and number of news to display. He does not need to understand that solr is used.

Example 2 - EXT:news ¶

In case of EXT:news, one might want to add “recent news” to the pages. This might contain a configurable number of news entries and different layouts, like “list” or “slider”. This is another example where custom plugins for existing 3rd party extensions might be useful. One can create those content elements and plugins.

Another benefit of this example: One can add “recent news” on a news detail page without thinking about any limitations. Due to being another plugin with a different signature, no arguments might create trouble. Also links created between those plugins can make use of the Extbase setting:

plugin.tx_news_recentnews {
    features {
        skipDefaultArguments = 1
    }
}

This can also be enabled for the whole extension:

plugin.tx_news {
    features {
        skipDefaultArguments = 1
    }
}

Or the whole installation / page:

config.tx_extbase {
    features {
        skipDefaultArguments = 1
    }
}

A link between those plugins can look like this, assuming to link from “Recent News” to “Detail News” custom plugin:

<f:link.action pageUid="11"
   pluginName="Details"
   arguments="{news: news}"
>
   <h4>{news.title}</h4>
</f:link.action>

As each plugin has its own default Controller-Action-Combination, there is no need to add them to the URL generation. Also thanks to the configuration of skipDefaultArguments, these will not be added to the url, resulting in an URL like this with CMS v9:

/?news_details%5Bnews%5D=1785&cHash=1f740d5404dddcf84b2c8bebc985deb9

How to add a new TYPO3 plugin ¶

To add a new plugin, first of one API call is necessary. After this was done, the plugin is already available to the frontend. Next, the content element can be created in the preferred way, which depends on the agency and developer.

Afterwards the optional FlexForm and TypoScript configuration can be added.

For further information, take a look at Real world example.

Conclusion for Extbase controller ¶

Each controller within an Extbase extension consists of actions, which should only do a single task each. By providing fine grained actions for single tasks, the Integrator is able to configure installation specific plugins, with new combinations of existing controllers and actions.

A contrary example was developed by myself and our team during my training. There we created a single controller with nearly 10 actions, all doing the same. The reason for those actions was to provide 10 different template variants. Today one could use ten custom plugins. Or even better use a setting like the layout field within the content element, together with an f:render call within Fluid to switch the rendering. But this will not be covered here. Just make sure, actions and controllers are written in a clean, reusable way.

Real world example ¶

The following example demonstrates the concept based on EXT:news and a new content element to display recent news. The editor can configure how many news should be displayed.

Register plugin within ext_localconf.php:

\TYPO3\CMS\Extbase\Utility\ExtensionUtility::configurePlugin(
    'GeorgRinger.news',
    'Recent',
    [
        'News' => 'list',
    ],
    [],
    \TYPO3\CMS\Extbase\Utility\ExtensionUtility::PLUGIN_TYPE_CONTENT_ELEMENT
);

Configure TCA for content element within Configuration/TCA/Overrides/tt_content_news_recent.php:

(function ($tablename = 'tt_content', $contentType = 'news_recent') {
    \TYPO3\CMS\Core\Utility\ArrayUtility::mergeRecursiveWithOverrule($GLOBALS['TCA'][$tablename], [
        'ctrl' => [
            'typeicon_classes' => [
                $contentType => 'content-recent-news',
            ],
        ],
        'types' => [
            $contentType => [
                'showitem' => implode(',', [
                    '--div--;LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:general',
                        '--palette--;;general',
                        'pi_flexform',
                    '--div--;LLL:EXT:frontend/Resources/Private/Language/locallang_ttc.xlf:tabs.appearance,--palette--;;frames,--palette--;;appearanceLinks,',
                    '--div--;LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:language,--palette--;;language,',
                    '--div--;LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:access,
                      --palette--;;hidden,
                      --palette--;;access,
                    --div--;LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:categories,
                         categories,
                    --div--;LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:notes,
                         rowDescription,
                    --div--;LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:extended,'
                ]),
            ],
        ],
        'columns' => [
            'pi_flexform' => [
                'config' => [
                    'ds' => [
                        '*,' . $contentType => 'FILE:EXT:sitepackage/Configuration/FlexForms/ContentElements/RecentNews.xml',
                    ],
                ],
            ],
        ],
    ]);

    \TYPO3\CMS\Core\Utility\ExtensionManagementUtility::addTcaSelectItem(
        $tablename,
        'CType',
        [
            'Recent News',
            $contentType,
            'content-recent-news',
        ],
        'textmedia',
        'after'
    );
})();

Optionally, add and register FlexForm.

Registration is happening in TCA, see above example, line 27-35. Please note that there must not be any whitespace in the array key when registering the FlexForm: '*,' . $contentType.

The FlexForm itself can look like the following Configuration/FlexForms/ContentElements/RecentNews.xml.:

<T3DataStructure>
   <sheets>
      <sDEF>
            <ROOT>
               <TCEforms>
                  <sheetTitle>LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_tab.settings</sheetTitle>
               </TCEforms>
               <type>array</type>
               <el>
                  <!-- Limit Start -->
                  <settings.limit>
                        <TCEforms>
                           <label>LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_additional.limit</label>
                           <config>
                              <type>input</type>
                              <size>5</size>
                              <eval>num</eval>
                           </config>
                        </TCEforms>
                  </settings.limit>

                  <!-- Offset -->
                  <settings.offset>
                        <TCEforms>
                           <label>LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_additional.offset</label>
                           <config>
                              <type>input</type>
                              <size>5</size>
                              <eval>num</eval>
                           </config>
                        </TCEforms>
                  </settings.offset>

                  <!-- Category Mode -->
                  <settings.categoryConjunction>
                        <TCEforms>
                           <label>LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categoryConjunction</label>
                           <config>
                              <type>select</type>
                              <renderType>selectSingle</renderType>
                              <items>
                                    <numIndex index="0" type="array">
                                       <numIndex index="0">LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categoryConjunction.all</numIndex>
                                       <numIndex index="1"></numIndex>
                                    </numIndex>
                                    <numIndex index="1">
                                       <numIndex index="0">LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categoryConjunction.or</numIndex>
                                       <numIndex index="1">or</numIndex>
                                    </numIndex>
                                    <numIndex index="2">
                                       <numIndex index="0">LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categoryConjunction.and</numIndex>
                                       <numIndex index="1">and</numIndex>
                                    </numIndex>
                                    <numIndex index="3">
                                       <numIndex index="0">LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categoryConjunction.notor</numIndex>
                                       <numIndex index="1">notor</numIndex>
                                    </numIndex>
                                    <numIndex index="4">
                                       <numIndex index="0">LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categoryConjunction.notand</numIndex>
                                       <numIndex index="1">notand</numIndex>
                                    </numIndex>
                              </items>
                           </config>
                        </TCEforms>
                  </settings.categoryConjunction>

                  <!-- Category -->
                  <settings.categories>
                        <TCEforms>
                           <label>LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.categories</label>
                           <config>
                              <type>select</type>
                              <renderMode>tree</renderMode>
                              <renderType>selectTree</renderType>
                              <treeConfig>
                                    <dataProvider>GeorgRinger\News\TreeProvider\DatabaseTreeDataProvider</dataProvider>
                                    <parentField>parent</parentField>
                                    <appearance>
                                       <maxLevels>99</maxLevels>
                                       <expandAll>TRUE</expandAll>
                                       <showHeader>TRUE</showHeader>
                                    </appearance>
                              </treeConfig>
                              <foreign_table>sys_category</foreign_table>
                              <foreign_table_where>AND (sys_category.sys_language_uid = 0 OR sys_category.l10n_parent = 0) ORDER BY sys_category.sorting</foreign_table_where>
                              <size>15</size>
                              <minitems>0</minitems>
                              <maxitems>99</maxitems>
                           </config>
                        </TCEforms>
                  </settings.categories>

                  <!-- Include sub categories -->
                  <settings.includeSubCategories>
                        <TCEforms>
                           <label>LLL:EXT:news/Resources/Private/Language/locallang_be.xlf:flexforms_general.includeSubCategories</label>
                           <config>
                              <type>check</type>
                           </config>
                        </TCEforms>
                  </settings.includeSubCategories>
               </el>
            </ROOT>
      </sDEF>
   </sheets>
</T3DataStructure>

Configure PageTSconfig for the content element to add it to the new content element wizard:

mod {
    wizards.newContentElement.wizardItems.common {
        elements {
            news_recent {
                iconIdentifier = content-recent-news
                title = Recent News
                description = Displayes recent news
                tt_content_defValues {
                    CType = news_recent
                    pi_flexform (
                        <?xml version="1.0" encoding="utf-8" standalone="yes" ?>
                        <T3FlexForms>
                            <data>
                                <sheet index="sDEF">
                                    <language index="lDEF">
                                        <field index="settings.limit">
                                            <value index="vDEF">4</value>
                                        </field>
                                    </language>
                                </sheet>
                            </data>
                        </T3FlexForms>
                    )
                }
            }
        }
        show := addToList(news_recent)
    }
    web_layout.tt_content.preview.news_recent = EXT:sitepackage/Resources/Private/Templates/ContentElementsPreview/RecentNews.html
}

Configure TypoScript for rendering of content element: (This example assumes EXT:fluid_styled_content is used)

plugin.tx_news_recent {
    settings {
        orderBy = datetime
        orderDirection = desc
    }
    view {
        templateRootPaths {
            10 = EXT:sitepackage/Resources/Private/Templates/Plugins/News/RecentNews/
        }
        pluginNamespace = news_recent
    }
}

Add fluid template accordingly to configured paths.

Optionally, register Icon for content element within ext_localconf.php:

$icons = [
    'content-recent-news' => 'EXT:news/Resources/Public/Icons/Extension.svg',
];
$iconRegistry = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance(\TYPO3\CMS\Core\Imaging\IconRegistry::class);
foreach ($icons as $identifier => $path) {
    $iconRegistry->registerIcon(
        $identifier,
        \TYPO3\CMS\Core\Imaging\IconProvider\SvgIconProvider::class,
        ['source' => $path]
    );
}

Acknowledgements to pietzpluswild GmbH and KM2 >> GmbH who allowed me to dive into the topic and to implement a solution for their customers.

Also thanks to Josef Glatz for proof reading and contributing to the Blog post. He also motivated me to finish this post.

• FlexForm data structure
• Configuring The Plugin
• newContentElement.wizardItems
• TCA Reference
• Icon API
• Adding your own content elements

As caching is a huge topic, this post will only cover the mentioned small part. Do not expect a complete explanation of all parts. The caching framework itself, useful in custom PHP Code is already explained in Caching Framework Architecture. Also Plugins are not part of this post. Instead this post will focuses on content elements and proper cache invalidation.

The example ¶

This post will use the following example:

A page using a file reference, e.g. an Textmedia or Uploads content element where an image is rendered. This image does not have any property set in the reference, e.g. no title or description. Also the original file which is referenced does not contain any information yet.

Still the template will render those information if available. Now the editor will add those information to the file via Filelist. Still the information is not displayed in frontend, until the cache is cleared.

This is default TYPO3 behaviour, which might be “fixed” in the future. Till now, this works as an example how to adjust TYPO3 to clear cache for all pages using this file, to display the new information immediately.

Understanding existing caching ¶

At the end of serving a rendered page, TYPO3 will cache the rendered markup. Some parts might be replaced with a marker, but those are not part of this post. If anything on the page is changed, e.g. the editor is saving a content element, TYPO3 will clear the cache for this page. This way the next visitor will trigger a new rendering and see the results. Also the new generated markup is added to the cache again.

How does TYPO3 invalidate, aka clear, the cache?

Two PHP Classes are involved here, one is the \TYPO3\CMS\Core\DataHandling\DataHandler which is THE API to change any data inside of TYPO3. The second class is TYPO3\CMS\Core\Cache\CacheManager, which is the API to all caches. The DataHandler receives the call to update a certain content element and will generate some Cache Tags out of that record. These tags are then send for clearing to the CacheManager.

TYPO3 already assigns necessary cache tags like pageId_6 to each rendered page cache entry. This way it’s possible to clear cache entries for a single page by clearing the accordingly cache tag.

DataHandler also generates these cache tag, e.g. for each record that is updated, the DataHandler will look up the page where the record is persisted, which is the pid column of each record. This cache tags are then cleared, leading to an invalidated generated page markup.

This way TYPO3 works out of the box. But not all circumstances are handled yet. E.g. the above example with file references does not work.

Concrete cache clearing example ¶

Let’s follow a concrete example:

There is one page with uid 10 and a single Textmedia content element with uid 20 on page 10. This content element displays some text and an image. The image is a sys_file_reference with uid 30 referencing sys_file with uid 40.

Once the content element is updated, DataHandler will generate the following cache tags:

• pageId_10
• tt_content
• tt_content_20
• sys_file_reference
• sys_file_reference_30

Everything is working as expected. But as an editor updates the file itself, the record sys_file_metadata with uid 40, the following cache tags will be generated:

• pageId_0
• sys_file_metadata
• sys_file_metadata_40

There is no connection to page with uid 10, leading to a non updated page, not displaying the updated title, which is inherited by sys_file_reference with uid 30 if no title is defined there.

Some more notes about the tags for sys_file_metadata: Saving the file updates not the file itself, but only the associated sys_file_metadata which are relevant through out the system. The file itself will only be changed if it’s replaced, e.g. by new upload. Also pageId_0 is generated, as most sys_ records are stored on the non existing page 0.

Possible approaches ¶

One now has to associate either cache tag sys_file_metadata_40 when rendering the page, or to add sys_file_reference_30 cache tag when updating the sys_file_metadata_40.

Both approaches are completely valid and the latter might look easier first. As we can use a Hook to extend the generated cache tags to clear, check for the already available cache tags, fetch all associated references, and add them.

Still, this approach will not work, as this cache tag is not associated to the stored page. The page does not know about the internals of rendered content elements. It does not know or understand that Textmedia fetches sys_file_reference with uid 30 which in turn is based on sys_file_metadata with uid 40. These might be added through a DataProcessor, which does not add any cache tags to the rendered page. Maybe this might change in the future.

Adding further cache tags to generated page ¶

Adding further cache tags to pages is possible via TypoScript, and PHP. This way one can collect information and attach further tags based on this information. TYPO3 itself will already generate necessary tags when records are updated, leading to auto clearing of necessary pages.

Those tags can be added via addPageCacheTags property of stdWrap.

This property can either add static strings, e.g.:

addPageCacheTags = pagetag1, pagetag2, pagetag3

This property also implements stdWrap again, adding the possibility to use a preUserFunc or postUserFunc:

tt_content.uploads.stdWrap {
    addPageCacheTags {
        postUserFunc = Codappix\CdxSite\Caching\ContentElementCaching->generateTags
    }
}

Now it’s up to the PHP implementation to add further tags.

Example PHP solution ¶

The concrete implementation depends on the concrete use case. For the above example the following implementation would be one working solution:

1. Extend existing FilesProcessor to “save” provided file references.
2. Make this processor a singleton. The processor is state less but becomes state full as all file references need to be remembered until cache tags are added to the page.
3. Replace existing processor in TypoScript with custom one.
4. Add custom processor also as postUserFunc via stdWrap

The TypoScript might look like:

tt_content.uploads {
    dataProcessing {
        10 = Codappix\CdxSite\Caching\FilesProcessorAddingCacheTagsToPage
    }
    stdWrap {
        addPageCacheTags {
            postUserFunc = Codappix\CdxSite\Caching\FilesProcessorAddingCacheTagsToPage->generateTags
        }
    }
}

While EXT:cdx_site/Classes/Caching/FilesProcessorAddingCacheTagsToPage.php might look like:

<?php

namespace Codappix\CdxSite\Caching;

use TYPO3\CMS\Core\Resource\FileReference;
use TYPO3\CMS\Core\SingletonInterface;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer;
use TYPO3\CMS\Frontend\DataProcessing\FilesProcessor;

/**
 * Singleton, in order to keep instance between filling in information and accessing information.
 *
 * Use as data processor replacement to add file references for content element.
 * Then add as userfunc to content element within TypoScript, to add cache tags.
 *
 * Example usage within TypoScript:
 *
 *   stdWrap.addPageCacheTags {
 *       postUserFunc = Codappix\CdxSite\Caching\FilesProcessorAddingCacheTagsToPage->generateTags
 *   }
 *
 */
class FilesProcessorAddingCacheTagsToPage extends FilesProcessor implements SingletonInterface
{
    private $tags = [];

    public function process(
        ContentObjectRenderer $cObj,
        array $contentObjectConfiguration,
        array $processorConfiguration,
        array $processedData
    ) {
        if (isset($processorConfiguration['if.']) && !$cObj->checkIf($processorConfiguration['if.'])) {
            return $processedData;
        }

        $processedData = parent::process(
            $cObj,
            $contentObjectConfiguration,
            $processorConfiguration,
            $processedData
        );

        $targetVariableName = $cObj->stdWrapValue('as', $processorConfiguration, 'files');
        foreach ($processedData[$targetVariableName] as $fileReference) {
            $this->tags[] = 'sys_file_metadata_'
                . $fileReference->getOriginalFile()->_getMetaData()['uid'];
            $this->tags[] = 'sys_file_reference_'
                . $fileReference->getUid();
        }

        return $processedData;
    }

    public function generateTags(string $content = '', array $configuration = null)
    {
        return implode(',', array_unique(array_filter(array_merge(
            GeneralUtility::trimExplode(',', $content),
            $this->tags
        ))));
    }
}

Acknowledgements to pietzpluswild GmbH who allowed me to dive into the topic and to implement a solution for their customer Stadtwerke Bonn.

• TypoScript reference about adding cache tags to rendered page: addPageCacheTags
• TypoScript reference about using and defining user functions: postUserFunc

Assumptions ¶

This Blog post makes the following assumptions about the TYPO3 installation, otherwise the solution might not work.

1. Page layout is rendered using FLUIDTEMPLATE.
2. EXT:fluid_styled_content is used to render content elements.
3. Plugins are always using Extbase.

Page Layout ¶

FLUIDTEMPLATE provides two ways to provide information to the template. One can use variables or settings. As the page uid does not need TypoScript stdWrap but is a fix value, and fits more into “settings”, we will use settings here:

page {
   10 {
      settings {
         cdx {
            pageUids {
               search = 10
            }
         }
      }
   }
}

The above example assumes that page.10 is of type FLUIDTEMPLATE and contains the existing setup to render page layout. We now add settings which is a plain PHP Array passed to the template.

As good practice namespaces are introduces, in above example cdx.pageUids where cdx is the company vendor, also used within PHP classes. pageUids is just a grouping, as we might also provide paths or further information.

With above example we could access the page uid using {settings.cdx.pageUids.search} within the template, layout and any partial, as settings are always passed to all further template files within Fluid.

One could now use the uid like:

<f:form pageUid="{settings.cdx.pageUids.search}" action="search">
   <!-- search form -->
</f:form>

Content Elements ¶

All content elements are rendered by inheriting lib.contentElement, which is an FLUIDTEMPLATE. Knowing this, we can re use the above knowledge from page layout, to add the necessary information to all content elements:

lib.contentElement {
   settings {
      cdx {
         pageUids {
            search = 10
         }
      }
   }
}

This way all content elements have the same information available and links can be created to important pages.

Thanks to the namespacing, there is no risk that an setting already in use might be replaced.

Plugins ¶

Plugins using Extbase are working nearly the same as content elements, all inherit an global “environment” from config.tx_extbase. This “environment” is merged with more specific further “environments” from TypoScript and Flexforms.

Knowing this, the general information can be added like this:

config.tx_extbase {
   settings {
      cdx {
         pageUids {
            search = 10
         }
      }
   }
}

Cleanup ¶

Right now the page uid for page “search” is defined three times, which is a bad practice. Therefore a constant can be used instead:

pageUids {
   search = 10
}

Which is then used in all three places:

page {
   10 {
      settings {
         cdx {
            pageUids {
               search = {$pageUids.search}
            }
         }
      }
   }
}

lib.contentElement {
   settings {
      cdx {
         pageUids {
            search = {$pageUids.search}
         }
      }
   }
}

config.tx_extbase {
   settings {
      cdx {
         pageUids {
            search = {$pageUids.search}
         }
      }
   }
}

The benefits ¶

Using the above approach, a new site can easily be added to the TYPO3 installation. To make the new site work, the constant has to be adjusted, that’s all one has to do.

Also replacing parts of the site is more easy to achieve. E.g. a single part of page tree is relaunched, one only has to change the constant again.

There is no need for “search and replace”.

• FLUIDTEMPLATE especially settings.
• Right now there is no documentation about config.tx_extbase.
• TYPO3 Core API Using site configuration in TypoScript.
• Changelog entry Feature: #91080 - Site settings as TypoScript constants and in TSconfig.

What’s mbox? ¶

Accordingly to Wikipedia:

Mbox is a generic term for a family of related file formats used for holding collections of email messages, […]

All messages in an mbox mailbox are concatenated and stored as plain text in a single file. […]

[…] the format used for the storage of email has never been formally defined through the RFC standardization mechanism […] In 2005 finally, the application/mbox media type was standardized as RFC 4155, and hints that mbox stores mailbox messages in their original Internet Message (RFC 2822) format, […]

—https://en.wikipedia.org/wiki/Mbox

So mbox does not introduce any dependency at all, it’s just a plain text file inside the filesystem. Stored mail can be checked with any editor one feels comfortable with. Beside a plain editor, thanks due the RFC existing Mail clients are able to read the stored mails. This way it’s possible to open mails with an existing desktop GUI for inspection.

Setup TYPO3 ¶

TYPO3 provides an API to deliver emails. It’s a small wrapper around the Swift Mailer library. Every email send through this API is delivered accordingly to the configuration of TYPO3. This makes it possible to send all mails via an SMTP or some other mail system. But it’s also possible to use mbox as an alternative.

To use mbox as mail delivery within TYPO3, the following two options have to be set, e.g. within AdditionalConfiguration.php:

$GLOBALS['TYPO3_CONF_VARS']['MAIL']['transport'] = 'mbox';
$GLOBALS['TYPO3_CONF_VARS']['MAIL']['transport_mbox_file'] = '/path/to/mbox/file';

Within typo3/sysext/core/Configuration/DefaultConfigurationDescription.yaml the option transport_mbox_file is documented:

The file where to write the mails into. This file will be conforming the mbox format described in RFC 4155. It is a simple text file with a concatenation of all mails. Path must be absolute.

Once the options are configured, one could use the “Environment” backend module to “Test Mail Setup”, which was located inside the Install Tool prior CMS v9. The Mail should be send and the file should be created by TYPO3 containing the example mail.

Setup E-Mail client ¶

Not every client works with mbox files:

Command Line ¶

For command line there is the “mail” utility, e.g. within the mailutils debian package or on macOS. Call mail /path/to/mbox/file to open all available mails.

Also on command line available is the mail client mutt and neomutt. Call the program with path to the mbox file.

For desktop clients, Thunderbird will do the job. ¶

Create a symlink to the mbox file within the Thunderbird profile. E.g. under Ubuntu 18.04 this can be done by calling:

ln -s "/path/to/mbox/file" "/home/<username>/.thunderbird/<profileFolder>/Mail/Local Folders"

An concrete example:

ln -s ~/Projects/mbox/typo3-example.mbox "/home/daniels/.thunderbird/j30pjbfz.default/Mail/Local Folders"

After the link was created, restart Thunderbird and one should see the mbox file in the bottom left of available mail accounts inside “Local Folders”.

Some further notes regarding mbox within Thunderbid:

1. The local folders are not synced. Once a mail is send, one needs to refresh them. This can be achieved by right click on the mbox file within Thunderbird. Select “Properties” and click “Repair Folder”. This will refresh the content within Thunderbird.

2. Thunderbird will not update the mbox file. E.g. if mails are deleted within Thunderbird, this will not be reflected within the file. So one might truncate the file content from time to time.

I didn’t take a deeper look at this, but it looks like Thunderbird uses an internal database beside the mbox for indexing, leading to the above workarounds.

There might be further clients available I’m not aware of, just contact me and I’ll add them.

Further thoughts ¶

Since mbox is just plain text one could easily integrate this for functional testing within a test suite, without the need to run any other software.

Also as the file can be opened with desktop mail clients, Frontend developers are able to “real world” test the output and styling. Contrary to solutions like MailHog or MailCatcher the rendering happens inside the client and not a browser.

Added information about refreshing Thunderbird and clearing contents of mbox. Thanks to @garbast for the hint to add these information.

• TYPO3 documentation about emails: https://docs.typo3.org/typo3cms/CoreApiReference/ApiOverview/Mail/Index.html
• Reading an mbox file with Thunderbird - Berkeley Lab Commons: https://commons.lbl.gov/display/~jwelcher@lbl.gov/Reading+an+mbox+file+with+Thunderbird

Danger ¶

This approach is only for a local development environment. Do never use this in a production or publical available environment!

This auto login automatically authentificates the configured user in. Please read the complete article and do nothing you do not understand.

Background knowledge ¶

To implement the feature we need to understand how TYPO3 processes the login, so we can provide our auto login. The process and implementation is documented at Authentication section in TYPO3 Core API Reference.

TYPO3 will fetch all registered services to detect authentications in TYPO3 backend. By default the registered services will only be called if certain conditions are met, e.g. credentials were submitted in the current request. As we do not want to submit anything, we have to configure TYPO3 to try authentication all the time.

Once we have configured TYPO3 to process authentication all the time, we have to register a new service for authentication to automatically login the specified user.

With these information we can check the example implementation which will to both.

Implementation ¶

This is the working implementation, which is explained afterwards:

<?php

namespace Codappix\CdxAutoLogin {

    use TYPO3\CMS\Core\Utility\ExtensionManagementUtility;
    use TYPO3\CMS\Core\Authentication\AbstractAuthenticationService;

    /**
     * Auto login the configured user.
     */
    class AutoAuthenticationTypo3Service extends AbstractAuthenticationService
    {
        public function getUser()
        {
            $record = $this->fetchUserRecord('dsiepmann');
            if (is_array($record)) {
                return $record;
            }

            return $this->fetchUserRecord('daniel.siepmann');
        }

        public function authUser(array $user)
        {
            return 200;
        }
    }

    if ((TYPO3_REQUESTTYPE & TYPO3_REQUESTTYPE_CLI) === 0) {
        ExtensionManagementUtility::addService(
            'sitepackage',
            'auth',
            AutoAuthenticationTypo3Service::class,
            [
                'title' => 'Auto User authentication',
                'description' => 'Auto authenticate user with configured username',
                'subtype' => 'authUserBE,getUserBE',
                'available' => true,
                'priority' => 100,
                'quality' => 50,
                'className' => AutoAuthenticationTypo3Service::class,
            ]
        );

        $GLOBALS['TYPO3_CONF_VARS']['SVCONF']['auth']['setup']['BE_alwaysFetchUser'] = true;
        $GLOBALS['TYPO3_CONF_VARS']['SVCONF']['auth']['setup']['BE_alwaysAuthUser'] = true;
    }
}

The first thing you might see is the namespace declaration using curly braces. This feature was introduced in PHP 5.3.0 to enable multiple namespace declarations in one file. This is a bad practice as already mentioned in official php docs. We still use it here, to define a new scoped namespace for our code. This way we can separate the code from the global namespace, without the need of an additional file.

Inside of the namespace we define the service as a class. This class extends the AbstractAuthenticationService class of TYPO3. Thanks to the namespace we can import the namespace of the extended class. The service itself is very small, as we do not have any logic.

We just define that our service had authenticated the user and no further checks should be processed. This is the return value of 200 inside of the method authUser.

The service also returns the user which was logged in. In our case the username has to be placed in the method call fetchUserRecord.

Once the service exists, we can register the service. The registration is done using the addService method of ExtensionManagementUtility. The important part is that the service is available and has a higher priority to be called first. Also the quality has to be equal or higher then the required quality. The configured subtype defines which features are provided by the service. In the above example the service authenticates the user, which was the 200, and provides the user, which was done inside getUser method.

All registered services are added to $GLOBALS['T3_SERVICES'] which can be inspected using the Configuration module inside of TYPO3 backend. This way you can fetch the necessary information for the registration of the service.

Last but not least we have to force the authentication process even if no credentials were provided. This is done with this configuration:

$GLOBALS['TYPO3_CONF_VARS']['SVCONF']['auth']['setup']['BE_alwaysFetchUser'] = true;
$GLOBALS['TYPO3_CONF_VARS']['SVCONF']['auth']['setup']['BE_alwaysAuthUser'] = true;

For further information about the above configuration refer to Advanced Options section of the TYPO3 Core Reference.

The registration and configuration are wrapped with a condition to check that we are not in CLI mode. As we do not need and want our code to run in CLI context. We only need this in backend and frontend mode.

Usage ¶

The above implementation can be pasted into AdditionalConfiguration.php. Only the username has to be inserted on line 15.

Multiple php namespaces in one file ¶

You should read the official documentation about multiple php namespaces in one file, otherwise you might run into issues with above implementation, depending on your AdditionalConfiguration.php.

If you already have code inside of your AdditionalConfiguration.php you should wrap that code with:

namespace {
   // Existing code ...
}

// Above implementation

As the namsepace has to be the first statement after any declare statements in a PHP file. Without the global namespace-scope you would receive a fatal error.

Simple “Solution” ¶

Instead of a “full blown” auto login, you could also just raise the session timout. This way your session lasts longer and you do not have to login so often. Add the following configuration to AdditionalConfiguration.php:

$GLOBALS['TYPO3_CONF_VARS']['BE']['sessionTimeout'] = 60 * 60 * 24;

The value is in seconds, so 60 * 60 * 24 is a whole day. You could also add *7 for a week.

Going further ¶

As everything here, this post and solution is open source. If you have ideas based upon this solution go ahead. E.g. write a small extension and use require-dev to fetch the extension. Make the user name configurable or check a cookie or IP address to activate this solution and detect the user name. That’s why all necessary resources are referenced and explained.

But always think about security.

Thanks to Tim Schreiner the condition for CLI requests was added which will prevent the code from execution in CLI context.

Thanks to Felix Althaus from undkonsorten these solution is available as a composer package undkonsorten/typo3-auto-login.

• PHP documentation about multiple namespaces in one file.
• Authentication in TYPO3 Core API Reference.
• Advanced Options section of the TYPO3 Core Reference.
• Composer package version: undkonsorten/typo3-auto-login

Requirements ¶

All requirements are already documented at Setup in the official contribution workflow. All requirements which are specific to acceptance tests, are already configured inside of composer.json of TYPO3 and therefore installed. Typically the requirements for acceptance tests consist of:

WebDriver ¶

As acceptance tests are run inside a browser, most of the time, we need a browser and access to control the browser from the outside.

WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform- and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of web browsers.

—https://www.w3.org/TR/webdriver/

Chromedriver ¶

Also we need a browser that supports WebDriver, or a driver that will add the support. As Chrome is nowadays a browser that supports headless mode, Chrome is the de-facto-standard for acceptance tests nowadays. To add the capabilities to use WebDriver with Chrome, one has to install Chromedriver.

PHPUnit ¶

To write and execute tests, there is always PHPUnit as a framework. Even if you are using another framework it’s very likely that PHPUnit is a dependency and used under the hood.

Codeception ¶

To make writing acceptance tests easier, TYPO3 CMS is using Codeception as a testing framework on top of PHPUnit. Codeception has much easier to use API to make use of WebDriver.

But all of these are installed through composer. So just take a look at the require-dev section of composer.json to get these information.

Finding the truth ¶

Once everything is installed, we need to know how to execute tests. There is a single point of truth for that, its called Bamboo. As we do not have access rights to read the configuration inside of the web UI of https://bamboo.typo3.com, we have another way. Luckily the whole configuration is right in our hands, inside the TYPO3 CMS repositories /Build/bamboo folder.

Don’t be afraid, Bamboo is written in Java, and so are the configuration files at time of writing this Blog post. But we do not have to change anything, we just need to grab information, so that’s not a problem, you are a developer, right?

So let’s take a look at the file /Build/bamboo/src/main/java/core/AbstractCoreSpec.java, which is shown with only necessary lines and context here:

/**
 * Job acceptance test installs system on mysql
 *
 * @param Requirement requirement
 * @param String requirementIdentfier
 */
protected Job getJobAcceptanceTestInstallMysql(Requirement requirement, String requirementIdentifier) {
    return new Job("Accept inst my " + requirementIdentifier, new BambooKey("ACINSTMY" + requirementIdentifier))
        .description("Install TYPO3 on mysql and create empty frontend page " + requirementIdentifier)
        .pluginConfigurations(this.getDefaultJobPluginConfiguration())
        .tasks(
            this.getTaskGitCloneRepository(),
            this.getTaskGitCherryPick(),
            this.getTaskComposerInstall(),
            this.getTaskPrepareAcceptanceTest(),
            new CommandTask()
                .description("Execute codeception AcceptanceInstallMysql suite")
                .executable("codecept")
                .argument("run AcceptanceInstallMysql -d -c " + this.testingFrameworkBuildPath + "AcceptanceTestsInstallMysql.yml --xml reports.xml --html reports.html")
                .environmentVariables(this.credentialsMysql)
        )

If you search for the string “acceptance” in this file, you will first find line 2 from above example. It’s like PHPDoc showing us that the installation procedure for MySQL is tested as an acceptance test here. Take a deeper look and you will see that different tasks are defined from line 12 to 20. First some basic tasks are executed like cloning, composer install and stuff like that. Afterwards getTaskPrepareAcceptanceTest is called, which sounds interesting. Let’s take a look at this method afterwards. Right after getTaskPrepareAcceptanceTest a command is defined, which already has a nice description “Execute codeception AcceptanceInstallMysql suite”. We can see the executable and the arguments in line 18 and 19.

So we know that Codeception is used to execute the acceptance test. We also know which reports are generated and which configuration is loaded. The configuration is prefixed with this.testingFrameworkBuildPath which is also defined in the same file:

protected String testingFrameworkBuildPath = "vendor/typo3/testing-framework/Resources/Core/Build/";

So our configuration is looked up in /vendor/typo3/testing-framework/Resources/Core/Build/AcceptanceTestsInstallMysql.yml.

Also in line 20 some environment variables are added, which are defined in this.credentialsMysql:

protected String credentialsMysql =
    "typo3DatabaseName=\"func\"" +
    " typo3DatabaseUsername=\"funcu\"" +
    " typo3DatabasePassword=\"funcp\"" +
    " typo3DatabaseHost=\"localhost\"" +
    " typo3InstallToolPassword=\"klaus\"";

So we know everything, except what happens inside getTaskPrepareAcceptanceTest, so let’s take a look at the last piece:

/**
 * Task to prepare an acceptance test starting selenium and others
 */
protected Task getTaskPrepareAcceptanceTest() {
    return new ScriptTask()
        .description("Start php web server, chromedriver, prepare environment")
        .interpreter(ScriptTaskProperties.Interpreter.BINSH_OR_CMDEXE)
        .inlineBody(
            this.getScriptTaskBashInlineBody() +
            "php -n -c /etc/php/cli-no-xdebug/php.ini -S localhost:8000 >/dev/null 2>&1 &\n" +
            "echo $! > phpserver.pid\n" +
            "\n" +
            "./bin/chromedriver --url-base=/wd/hub >/dev/null 2>&1 &\n" +
            "echo $! > chromedriver.pid\n" +
            "\n" +
            "mkdir -p typo3temp/var/tests/\n"
        );
}

We can see in the description, which is in line 6, that a web server and Chromedriver is started. Taking a look at the inlineBody we see that the local PHP server is started with some configuration, together with a Chromedriver and some configuration.

Playing around with the truth ¶

Looks like the following is necessary to execute the tests:

Create a database and user with access to it, and access to create “sub database”. See: https://wiki.typo3.org/Acceptance_testing#Acceptance_Testing_since_TYPO3_v8

Start a web server which listens to localhost on port 8000:

php \
   -d memory_limit=128M \
   -d max_execution_time=240 \
   -d xdebug.max_nesting_level=400 \
   -d max_input_vars=1500 \
   -S localhost:8000

This looks different from the Java-File. Yes, cause our local environment is different from Bamboos PHP environment; Once you start php -S localhost:8000 and tests, you will see typical PHP errors in install routine. This errors tell you to configure PHP, that’s what we did in above example.

Start the Chromedriver which was installed via composer:

./bin/chromedriver --url-base=/wd/hub

Execute Codeception to run the test:

typo3DatabaseName="typo3_acceptance" \
   typo3DatabaseUsername="typo3_acceptance" \
   typo3DatabasePassword="typo3_acceptance" \
   typo3DatabaseHost="localhost"
   typo3InstallToolPassword="klaus" \
   ./bin/codecept run AcceptanceInstallMysql -d \
      -c vendor/typo3/testing-framework/Resources/Core/Build/AcceptanceTestsInstallMysql.yml \
      --html reports.html

In above example the database, user and password are typo3_acceptance, because I liked it that way on my locale machine. Just place your values in there.

That’s what I’ve tried locally, which worked on my Ubuntu 18.04 setup with CMS 9 commit bc5dcaacef26cecb29e89554e5b1775dc839a4ae.

As above commands start long running processes, you have either to send them to background, or to start a new shell for each of the last three commands.

Getting results ¶

Codeception will tell you where you can find the generated reports, they are inside /typo3temp/var/tests/AcceptanceReportsInstallMysql/reports.html in our above example. You can open the report with any web browser.

Executing further tests ¶

TYPO3 does not only provide the above test, but a larger range of acceptance tests.

If you search further for “acceptance” inside the /Build/bamboo/src/main/java/core/AbstractCoreSpec.java-file, you will find the method getJobsAcceptanceTestsMysql. There is a bit “magic” that will split the tests to parallelize execution inside of Bamboo, which we can ignore for local testing for now.

To execute all further acceptance tests of TYPO3s core, run:

./bin/codecept run Acceptance -d \
   -c vendor/typo3/testing-framework/Resources/Core/Build/AcceptanceTests.yml \
   --html reports.html

We just exchange the argument after run and the configuration file to use. At the moment of this Blog post, this will execute 77 acceptance tests.

Where are the tests? ¶

So far we covered how to execute tests, and how to find information how to execute the tests. What’s still missing is the information where to find the actual tests. You can check out the configuration for Codeception, that’s the file we provide after the -c option in the above examples. For all acceptance tests this is /vendor/typo3/testing-framework/Resources/Core/Build/AcceptanceTests.yml.

If we take a look at this file we find out all we need to know:

actor: Tester
paths:
# @todo clean up here: https://forge.typo3.org/issues/79097 
  tests: ../../../../../../typo3/sysext/core/Tests
  log: ../../../../../../typo3temp/var/tests/AcceptanceReports
  data: Configuration/Acceptance/Data
  support: Configuration/Acceptance/Support
  envs: Configuration/Acceptance/Envs
settings:
  colors: true
  memory_limit: 1024M
extensions:
  enabled:
    - Codeception\Extension\RunFailed
    - Codeception\Extension\Recorder
    - TYPO3\TestingFramework\Core\Acceptance\AcceptanceCoreEnvironment
groups:
  AcceptanceTests-Job-*: Configuration/Acceptance/AcceptanceTests-Job-*

Line 4 is the important one. This tells us where tests can be found. All tests are available at /typo3/sysext/core/Tests. Also note the comment which points to https://forge.typo3.org/issues/79097 and covers the architectural problem of dependencies evolving from this structure.

The last two lines are for splitting up the acceptance tests for parallelization inside Bamboo.

Further information ¶

Acceptance tests are known to be fragile. Currently TYPO3 is not executing the acceptance tests inside of bamboo, as they are broken from time to time. As TYPO3 is GPL and Codeception MIT, both are joining forces to cover this issues in the future.

The above mentioned Java-Files are the single point of truth. They are used by Bamboo to execute the pre-merge and nightly plans.

• https://github.com/TYPO3/TYPO3.CMS/tree/master/Build/bamboo Bamboo files inside of TYPO3 CMS repository.
• https://wiki.typo3.org/Acceptance_testing#Acceptance_Testing_since_TYPO3_v8 Old documentation for TYPO3 CMS 8.
• https://github.com/TYPO3/testing-framework The testing framework dependency which is used as a base in TYPO3 CMS.
• https://codeception.com/docs/modules/WebDriver WebDriver documentation of Codeception, covering configuration and test API.

TypoScript linting ¶

Martin Helmich has created a TypoScript parser and TypoScript linter which already provides basic rules. The concepts are inherited from PHP Code Sniffer with different Sniffs which can be configured.

You can install the linter as dev dependency using composer:

composer require --dev helmich/typo3-typoscript-lint

The linter is pre-configured with a default configuration, which can be found at GitHub: https://github.com/martin-helmich/typo3-typoscript-lint/blob/v1.4.6/tslint.dist.yml

Configuration is written as yaml. You can extend the default configuration. Most linters provide the .dist way to provide a distributed configuration which can be overwritten by a more specific configuration. We are using yaml as file extension, which is the preferred accordingly to FAQ of yaml. Our configuration looks like the following tslint.yaml:

paths:
    - localPackages/sitepackage/Configuration/PageTSConfig/
    - localPackages/sitepackage/Configuration/TypoScript/
    - localPackages/sitepackage/Configuration/UserTSConfig/

sniffs:
    - class: Indentation
      parameters:
          indentConditions: true
    - class: RepeatingRValue
      disabled: true

We define paths containing TypoScript to check. Also we adjust the default sniffs a bit. We force indentation of one level inside of conditions, as we do in PHP and JavaScript. Also we disable the RepeatingRValue sniff as we check this rule ourselves.

To actually lint anything, you will call the linter with the configuration:

./vendor/bin/typoscript-lint -c tslint.yaml

Which will generate something like the following if everything is fine:

..................................................   [ 50 / 106,  47%]
..................................................   [100 / 106,  94%]
......                                               [106 / 106, 100%]

Complete without warnings

If errors or warnings exist, the following output will be generated:

...............................W..................   [ 50 / 106,  47%]
........W................................W........   [100 / 106,  94%]
......                                               [106 / 106, 100%]

Completed with 9 issues

CHECKSTYLE REPORT
=> localPackages/cdx_site/Configuration/TypoScript/Setup/Plugins/SearchCore.typoscript.
   7 No whitespace after object accessor.
  10 Accessor should be followed by single space.
  11 Value of object "plugin.tx_searchcore.view.templateRootPaths.10" is overwritten in line 12.

SUMMARY
9 issues in total. (9 warnings)

You can also output in checkstyle format which might be useful for some CI environments and editors.

yaml linting ¶

As we didn’t find any useful yaml linter written in PHP, we use one written in Python. Most php linters just call Symfony yaml parser and catch exceptions, which results in a single parsing error.

So we decided to use yamllint, which is no problem using Docker and GitLab-CI. I’ll not describe how to install locally, refer to GitLab-CI in next section.

As the TypoScript linter, this linter also provides a default of rules to check, which can be found at GitHub: https://github.com/adrienverge/yamllint/blob/v1.10.0/yamllint/conf/default.yaml There is also a “relaxed” version of the configuration. You can “inherit” one of the configurations in your own, which is provided as yamllint.yaml in your project root:

extends: default

rules:
    line-length: disable
    document-start: disable
    braces:
        min-spaces-inside-empty: 1
        max-spaces-inside-empty: 1
    brackets:
        min-spaces-inside-empty: 1
        max-spaces-inside-empty: 1
    comments:
        level: error
        min-spaces-from-content: 1
    comments-indentation:
        level: error
    empty-lines:
        max: 1
    empty-values:
        forbid-in-block-mappings: true
        forbid-in-flow-mappings: true
    indentation:
        spaces: 4

We adjust some rules, some become more strict, some are disabled completely.

You can call the linter using the following command:

yamllint -c yamllint.yaml localPackages/ yamllint.yaml .gitlab-ci.yml tslint.yaml

Nothing will be printed if everything is fine. Errors and warnings are reported like:

localPackages/cdx_site/Configuration/Forms/Base.yaml
  7:21      error    duplication of key "10" in mapping  (key-duplicates)

tslint.yaml
  10:28     error    empty value in block mapping  (empty-values)

Integrate linting in GitLab-CI ¶

As we now know how to configure the linter and how to execute them, we need to integrate them into our GitLab-CI through our .gitlab-ci.yml. Poorly GitLab does not allow yaml as file extension for that file.

The task for TypoScript looks like the following:

lint:typoscriptcgl:
    image: composer:1.6
    stage: lint
    script:
        - composer install --no-progress --no-ansi --no-interaction
        - ./vendor/bin/typoscript-lint -c tslint.yaml

The task for Yaml looks like the following:

lint:yaml:
    image: python:alpine3.7
    stage: lint
    before_script:
        - pip install yamllint==1.10.0
    script:
        - yamllint -c yamllint.yaml localPackages/ yamllint.yaml .gitlab-ci.yml tslint.yaml

As both linter will provide proper exit codes, we are finished. Output is generated in a human friendly way, so everyone can take a look right into the log what goes wrong and can fix the issues.

By default, both tools will return an exit code 0 if only warnings were found, which is considered best practice, as warnings are just warnings and those okay.

typoscript-lint will return an exit code 2 if an issue was found. This is only true for errors, not warnings. To enable exit code 2 also if warnings were found, add the option --fail-on-warnings.

yamllint will return 1 if errors were found, 0 otherwise. To return 2 if an warning was found, enable strict mode via --strict. For further information see Errors and warnings.

Updated on 20 Apr, 2019

Added further details about exit codes for TypoScript linter by Martin Helmich.

Thanks Twitter user spooner_web for a hint that these information might be helpful.

• GitLab
• GitLab-Ci
• TypoScript linter
• yamllint
• Blog post Integrate Typoscript linter into VIM.

Install dependencies ¶

First of all you need syntastic or ale to be installed properly, refer to their docs for installation guideline.

Also you need to have the linter installed. We prefer composer:

composer require --dev helmich/typo3-typoscript-lint

That will install the linter package with all dependencies into your project. By default the binary will be installed into vendor/bin/typoscript-lint. But the concrete path depends on your composer settings. Also it’s possible to install the package globally.

Configure vim plugin syntastic ¶

After all dependencies are installed, we need to add the syntax checker for TypoScript. The file has to be located at syntax_checkers/typoscript/lint.vim inside your runtimepath, e.g. ~/.vim/syntax_checkers/typoscript/lint.vim.

The file has the following content:

if exists('g:loaded_syntastic_typoscript_lint_checker')
    finish
endif
let g:loaded_syntastic_typoscript_lint_checker = 1

let s:save_cpo = &cpo
set cpo&vim

function! SyntaxCheckers_typoscript_lint_GetLocList() dict
    let makeprg = self.makeprgBuild({
        \ "exe": self.getExec(),
        \ "args": '--format=checkstyle',
        \ })

    let errorformat = '%f:%t:%l:%c:%m'

    return SyntasticMake({
        \ 'makeprg': makeprg,
        \ 'errorformat': errorformat,
        \ 'preprocess': 'checkstyle',
        \ 'postprocess': ['guards'] })
endfunction

call g:SyntasticRegistry.CreateAndRegisterChecker({
    \ 'filetype': 'typoscript',
    \ 'name': 'lint'})

let &cpo = s:save_cpo
unlet s:save_cpo

" vim: set sw=4 sts=4 et

Also add the path to your installed executable, e.g. inside of ~/.vimrc like so:

let g:syntastic_typoscript_lint_exec='./vendor/bin/typoscript-lint'

If you have installed the binary on a per project base with default paths. Otherwise adjust accordingly. If your path is different on a project level, take a look at Folder specific settings in Vim.

Use ale instead ¶

ale is another plugin for vim/neovim that integrates linters. You can find my pull request to integrate the linter at https://github.com/dense-analysis/ale/pull/4673.

• syntastic
• Ale
• TypoScript linter

Install dependencies ¶

First of all you have to install the filp/whoops package. Also I recommend to install symfony/var-dumper. Only with both packages you will see arguments in stack traces.

To install both run the following in your terminal:

composer global require filp/whoops
composer global require symfony/var-dumper

Configure TYPO3 ¶

Afterwards you can configure TYPO3 to use the new exception handler. Therefore insert the following in your typo3conf/AdditionalConfiguration.php. Of course you have to update line 4 to match your installation path:

call_user_func(function ($exceptionHandling = 'whoops') {
    // Use whoops error handler for errors.
    if ($exceptionHandling === 'whoops') {
        require_once '/Users/siepmann/.composer/vendor/autoload.php';
        $handler = null;
        if (defined('TYPO3_cliMode') && TYPO3_cliMode) {
            // $handler = new \Whoops\Handler\PlainTextHandler();
        } else {
            $handler = new \Whoops\Handler\PrettyPageHandler();
            $handler->setApplicationPaths([
                'web' => realpath(PATH_site . '../web'),
                'typo3' => realpath(PATH_site . '../vendor/typo3/cms'),
                'typo3conf' => realpath(PATH_site . 'typo3conf'),
            ]);
        }
        if ($handler !== null) {
            $whoops = new \Whoops\Run;
            $whoops->pushHandler($handler);
            $whoops->register();
        }
    }
    if ($exceptionHandling === 'xdebug' || $exceptionHandling === 'whoops') {
        // Disable original handler to use whoops or xdebug
        $GLOBALS['TYPO3_CONF_VARS']['SYS']['productionExceptionHandler'] = '';
        $GLOBALS['TYPO3_CONF_VARS']['SYS']['debugExceptionHandler'] = '';
        $GLOBALS['TYPO3_CONF_VARS']['SYS']['errorHandler'] = '';
    }
});

With this setup you will get the new exception handler for web requests.

On CLI I recommend to use typo3cms which will use Symfony stack traces anyway.

Configuration options ¶

To switch to the handler of xdebug, exchange whoops on line 1 with xdebug. To use the original TYPO3 handler insert something else, e.g. TYPO3.

Also you might want to adjust the application paths to your setup. All paths listed as values in that array will be filterable as application in stack trace.

• http://filp.github.io/whoops/

The following steps are necessary:

1. Write custom element as PHP Class, extending base element.
2. Define custom element as prototype in yaml-Configuration, which inherits existing configuration of a select.
3. Use the new element in the form.

The PHP Class ¶

As we want to provide custom functionality, we need to create a PHP Class which will contain this functionality. In our example we want to provide sys_category records based on a parent sys_category as possible options. Therefore we need to provide one option, the uid of the parent system category. Also we need to fetch the records from database using Doctrine and provide them as options for the element, e.g. select in our case.

The implementation is done with the following class:

<?php
namespace DS\ExampleExtension\Domain\Model\FormElements;

use TYPO3\CMS\Core\Database\ConnectionPool;
use TYPO3\CMS\Core\Database\Query\Restriction\FrontendRestrictionContainer;
use TYPO3\CMS\Core\Utility\GeneralUtility;
use TYPO3\CMS\Form\Domain\Model\FormElements\GenericFormElement;
use TYPO3\CMS\Frontend\Category\Collection\CategoryCollection;

class SystemCategoryOptions extends GenericFormElement
{
    public function setProperty(string $key, $value)
    {
        if ($key === 'systemCategoryUid') {
            $this->setProperty('options', $this->getOptions($value));
            return;
        }

        parent::setProperty($key, $value);
    }

    protected function getOptions(int $uid) : array
    {
        $options = [];

        foreach ($this->getCategoriesForUid($uid) as $category) {
            $options[$category['uid']] = $category['title'];
        }

        asort($options);
        return $options;
    }

    protected function getCategoriesForUid(int $uid) : array
    {
        $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
            ->getQueryBuilderForTable('sys_category');
        $queryBuilder->setRestrictions(
            GeneralUtility::makeInstance(FrontendRestrictionContainer::class)
        );

        return $queryBuilder
            ->select('*')
            ->from('sys_category')
            ->where(
                $queryBuilder->expr()->eq(
                    'parent',
                    $queryBuilder->createNamedParameter($uid, \PDO::PARAM_INT)
                )
            )
            ->execute()
            ->fetchAll();
    }
}

First of all we extend the setProperty method, which receives all options. If the current option is the configured systemCategoryUid, we hook into and add the options. In all other situations we just call the original method.

Based on the configured uid, we fetch the records from database in our getCategoriesForUid method.

Afterwards we iterate over the results and prepare them to be used by the select, therefore we need a label and identifier. We use the saved title and uid. The result is set as the options for select element.

The class itself does not contain any relation to the specifics of a select-element. It should also be possible to use the same code for radio or checkboxes, as long as they make use of the options property.

Define custom element ¶

Once our functionality is provided, we need to create a new form element to be available to our forms. Therefore we define the new element:

TYPO3:
  CMS:
    Form:
      prototypes:
        standard:
          formElementsDefinition:
            SingleSelectWithSystemCategory:
              __inheritances:
                10: 'TYPO3.CMS.Form.prototypes.standard.formElementsDefinition.SingleSelect'
              implementationClassName: 'DS\ExampleExtension\Domain\Model\FormElements\SystemCategoryOptions'
              renderingOptions:
                templateName: 'SingleSelect'

On line 7 we define the identifier of our new element, under which we can use the element in our forms. We inherit the existing configuration of the select element and exchange the concrete php class for implementation. As the path of fluid template is generated from the name, we define to use the same template as for the select element.

That’s all we have to do, to define a new select with different implementation.

Use element ¶

We are now able to use the defined element in our forms:

type: Form
identifier: Example
label: 'Example - Form'
prototypeName: standard
renderingOptions:
  submitButtonLabel: Submit
renderables:
  -
    type: Page
    identifier: page1
    renderingOptions:
      previousButtonLabel: 'previous page'
      nextButtonLabel: 'next page'
    renderables:
      -
        type: SingleSelectWithSystemCategory
        identifier: jobTitle
        label: Job Title
        properties:
          systemCategoryUid: 5
          prependOptionLabel: 'please choose'
          fluidAdditionalAttributes:
            required: required
        validators:
          -
            identifier: NotEmpty

We define our own SingleSelectWithSystemCategory element to be used and define our systemCategoryUid to be used. Everything else is exactly the same as for any other select, as we use the same template.

File Collection example ¶

I was able to build another example, thanks to our customer werkraum_media, following the same approach to use a selected File Collection instead. The concept is the same, you can have a look at the code at:

• https://git.daniel-siepmann.de/Customers/form_file_collection
• https://github.com/werkraum-media/form_file_collection/

Check out the official doc sections:

• Custom form element implementations
• __inheritances operator
• Prototypes

The new form framework already provides a SaveToDatabase finisher to persist submitted information to a database. Therefore it’s easy to create a fe_user registration. But there is one reason you can’t: passwords are stored in plain text as they get no special handling. Therefore you can write a custom finisher to crypt the password before saving to database.

The following steps are necessary:

1. Write custom finisher as PHP Class.
2. Register custom finisher as prototype in yaml-Configuration.
3. Use Finisher in your form before saving to database.
4. Use modified data by finisher instead of submitted data while saving to database.

Write custom finisher as PHP Class ¶

Just create a new PHP class that extends \TYPO3\CMS\Form\Domain\Finishers\AbstractFinisher, configure the possible options with their defaults and implement the executeInternal abstract method.

In our example we provide the option to configure a single field to crypt. Inside of our implementation we check that this field was submitted and that salted passwords are enabled in frontend.

If so, we crypt the password and add it as a new variable at Crypt.<fieldname>. Where Crypt is the shortFinisherIdentifier and <fieldname> the configured field to crypt.

<?php
namespace DS\ExampleExtension\Domain\Finishers;

use TYPO3\CMS\Form\Domain\Finishers\AbstractFinisher;
use TYPO3\CMS\Saltedpasswords\Salt\SaltFactory;
use TYPO3\CMS\Saltedpasswords\Utility\SaltedPasswordsUtility;

/**
 * This finisher for form framework will crypt the configured field with salted
 * passwords, if enabled for frontend.
 */
class CryptFinisher extends AbstractFinisher
{
    /**
     * @var array
     */
    protected $defaultOptions = [
        'field' => '',
    ];

    protected function executeInternal()
    {
        $fieldName = $this->parseOption('field');
        $formValues = $this->finisherContext->getFormValues();
        if (!isset($formValues[$fieldName])) {
            return;
        }

        if (SaltedPasswordsUtility::isUsageEnabled('FE')) {
            $this->finisherContext->getFinisherVariableProvider()->add(
                $this->shortFinisherIdentifier,
                $fieldName,
                SaltFactory::getSaltingInstance(null)->getHashedPassword($formValues[$fieldName])
            );
        }
    }
}

It’s nearly the same as a Fluid ViewHelper. We configure options instead of arguments and can provide defaults. Then we add new variables to the container, in this case the FinisherVariableProvider.

Register custom finisher ¶

New finishers are not registered out of the box, we have to register them manually. Therefore we add the following to our yaml configuration which is defined in TypoScript:

TYPO3:
  CMS:
    Form:
      prototypes:
        standard:
          finishersDefinition:
            CryptFinisher:
              implementationClassName: DS\ExampleExtension\Domain\Finishers\CryptFinisher

This way we can prevent naming collisions as we define the name of the finisher to use.

We just register the existing PHP class under CryptFinisher and can now use the finisher.

Use Finisher in own form ¶

We register the new custom finisher before other finishers, so he can crypt the plain text password and we are able to use the crypted version while saving to database.

We just tell the form framework to run our finisher and provide the necessary options:

finishers:
   -
     identifier: CryptFinisher
     options:
       field: password

Use modified data by finisher ¶

As the finisher has added new data, we can access the new data and save this instead of the original submitted plain text password. That’s done on line 18.

We use the curly braces as already known by Fluid. Inside of the braces we use the short identifier of the finisher, which is the class name without namespace and Finisher suffix. So for \DS\ExampleExtension\Domain\Finishers\CryptFinisher this would be Crypt. Then, as already known by Fluid, we separate the path with dots and access the added data as documented in our finisher.

finishers:
   -
     identifier: SaveToDatabase
     options:
       1:
         table: 'fe_users'
         mode: insert
         databaseColumnMappings:
           pid:
             value: 140
           disable:
             value: 1
           usergroup:
             value: 1
           description:
             value: 'Registered via form'
           password:
             value: '{Crypt.password}'

That’s it. We now save the crypted password instead of the original plain text submitted value. If, for any reason, multiple fields need to be crypted we can add the same finisher with different options multiplet times and don’t need to bloat the implementation itself.

Check out the official doc sections:

• Write a custom finisher
• implementationClassName
• TYPO3\CMS\Form\Domain\Finishers\AbstractFinisher

Note ¶

Since TYPO3 v10, Dependency Injection is integrated in the whole system. It is no longer limited to Extbase and his ObjectManager.

When working with TYPO3 v10 or newer, please do not read this blog post, but the official documentation at docs.typo3.org. There are also two blog posts at usetypo3.com which I highly recommend: Dependency Injection in TYPO3 and Usecase: Caching, DI and Events.

As TYPO3 uses the Symfony component, I would also recommend to read their documentation at symfony.com.

Possible inject methods ¶

There are three different ways to make use of dependency injection:

Via annotation ¶

Annotate a class property with @inject to enable Extbase to resolve the dependency through reflection:

/**
 * @var \TYPO3\CMS\Extbase\Utility\ArrayUtility
 * @inject
 */
 protected $arrayUtility;

Via inject method ¶

You can provide a method that’s reflected by the framework to inject a dependency. The main benefit is that you are able to initialize the dependency. A typical use case it to inject the logger through a method, as you inject the LogManager and fetch the concrete Logger inside the method:

/**
 * @var \TYPO3\CMS\Core\Log\Logger
 */
protected $logger;

public function injectLogger(\TYPO3\CMS\Core\Log\LogManager $logManager)
{
    $this->logger = $logManager->getLogger(__CLASS__);
}

Another example is injection of settings through the ConfigurationManager, see Inject TypoScript Settings.

Via constructor ¶

Extbase will also reflect the __construct method and inject dependencies not provided during construction:

public function __construct(ConfigurationContainerInterface $configuration)
{
    $this->configuration = $configuration;
}

Differences for methods ¶

With each method comes some difference:

With annotation ¶

You are not able to access the injected dependencies in your __construct but the special method initializeObject.

Also you have to use the FQCN (=Fully qualified class name), so import statements will not work here.

With inject method ¶

Like with annotation, you are not able to access the injected dependencies in your __construct but the special method initializeObject.

It’s possible to use use statements. As only the signature is reflected, no comment is needed at all.

The method has to be public and start with inject. The special method injectSettings is blocked and will not work.

Also this will be supported by TYPO3 Version 10 LTS out of the box accross all code, not only Extbase.

With constructor ¶

It’s possible to use use statements. As only the signature is reflected, no comment is needed at all.

Also this will be supported by TYPO3 Version 10 LTS out of the box accross all code, not only Extbase.

When injection is not working ¶

As dependency injection is part of Extbase and not TYPO3 Core, it will not work if you instantiate new instances through new or \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance(). You have to use \TYPO3\CMS\Extbase\Object\ObjectManager::get(). The ObjectManager will also take care of all sub dependencies.

Therefore you should use hooks and further stuff only as proxies, connecting your logic to the system through its API. This way you are able to load your logic through the ObjectManager resolving all dependencies, and are also able to reuse the logic in different places.

While calling get you can provide constructor arguments. You have to provide them in the way they are defined in the method signature. All arguments left undefined will be resolved through dependency injection. This way it’s possible to create a new instance and inject different dependencies:

class MyOwnClass
{
    public function __construct(
        ArrayUtilityInterface $arrayUtility,
        AnotherInterface $anotherDepdendency
    ) {
        // ...
}

class MyOwnArrayUtility implements ArrayUtilityInterface
{
    // ...
}

$customArrayUtility = $this->objectManager->get(MyOwnArrayUtility::class);
$this->objectManager->get(MyOwnClass::class, $customArrayUtility);

Just make sure to extend the original class or implement the expected interface. Therefore it’s much better to define interfaces and to use them in your signatures, then concrete class implementations.

This will change with TYPO3 Version 10 LTS. I'll try to update the blog post in the future.

Configuring dependencies ¶

Once you make use of dependency injection, you might want to exchange one resolved dependency for some reason, e.g. in a 3rd party or core Extension.

There are two ways you can configure dependencies to be resolved. One is TypoScript and the other is PHP.

TypoScript ¶

You have to configure the dependencies the following way:

config.tx_extbase {
    object {
        TYPO3\CMS\Extbase\Persistence\Storage\BackendInterface {
            className = DS\ExampleExtension\Persistence\Storage\Backend
        }
    }
}

The above example will inject our own implementation \DS\ExampleExtension\Persistence\Storage\Backend whenever \TYPO3\CMS\Extbase\Persistence\Storage\BackendInterface is required.

The downside of this approach is, that Extbase bootstrapping has to be run to initialize the ObjectManager with this configuration. But in TYPO3 there are enough situation when this is not the case, e.g. in Hooks.

The benefit is, you can also configure different dependencies per extension, plugin or module:

plugin.tx_exampleextension {
    object {
        TYPO3\CMS\Extbase\Persistence\Storage\BackendInterface {
            className = DS\ExampleExtension\Persistence\Storage\PluginSpecificBackend
        }
    }
}

module.tx_exampleextension {
    object {
        TYPO3\CMS\Extbase\Persistence\Storage\BackendInterface {
            className = DS\ExampleExtension\Persistence\Storage\ModuleSpecificBackend
        }
    }
}

PHP ¶

The other way is to directly configure the ObjectManager:

\TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance(\TYPO3\CMS\Extbase\Object\Container\Container::class)
    ->registerImplementation(
        \Codappix\SearchCore\Connection\ConnectionInterface::class,
        \Codappix\SearchCore\Connection\Elasticsearch::class
    );

You should place this inside of a ext_localconf.php. This way the configuration is available no matter which context is used. Therefore this should be preferred. Still this will always configure globally.

If nothing is configured, Extbase will remove the trailing Interface of the dependency and try to inject the class name, so for Vendor\Extension\Utility\ExampleUtilityInterface Extbase will try to provide Vendor\Extension\Utility\ExampleUtility.

Caching ¶

As reflection is a bit expensive, Extbase will cache the information. Therefore you have to clear cache once you add a new dependency injection, no matter which method you are using. Otherwise you will see method calls to non objects, or non working instantiations, as they are not injected. The used cache is extbase_object.

How to use in tests ¶

Another big benefit of the flexibility is used inside of tests. Compare the “old way” vs. the new way:

Old:

class SomeClass
{
    protected $exampleUtility;

    public function __construct()
    {
        $this->exampleUtility = GeneralUtility::makeInstance(ExampleUtility::class);
    }
}

New:

class SomeClass
{
    protected $exampleUtility;

    public function __construct(ExampleUtilityInterface $exampleUtility)
    {
        $this->exampleUtility = $exampleUtility
    }
}

The new one makes it very easy to pass a mocked version of a dependency inside of our tests, that’s true for all the methods, enabling us to mock the behaviour and create a unit test for a single class. For annotation and method there is a helper method you might use inside of your tests to inject the dependency.

The helper method is part of BaseTestCase and is called inject:

$testSubject = new ClassToTest();
$this->inject($testSubject, 'exampleUtility', $this->getMockBuilder(ExampleUtilityInterface::class)->getMock());

Which method should you use? ¶

I would prefer the _construct approach, as it’s not only working with Extbase, but also the only way to really define dependencies. Everyone still can create instances through makeInstance or new. But they still have to provide the dependencies. Also they can not be altered once an instance exists.

Also the construct and inject versions will be supported from TYPO3 version 10 LTS accross the code base, not only in Extbase.

• \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance()
• \TYPO3\CMS\Extbase\Object\ObjectManager::get()
• Also you might find the post Inject TypoScript Settings useful.
• Github gist in a form of a Blog post about this topic https://gist.github.com/NamelessCoder/3b2e5931a6c1af19f9c3f8b46e74f837
• German Blog post about this topic https://www.typo3tiger.de/blog/post/extbase-dependency-injection.html

The idea ¶

PHP_CodeSniffer already is able to parse PHP code. It can be used as a framework to check code against any possible rule, as you can implement new rules using PHP.

Beside the check, it’s also possible to use the cli tool phpcbf, which comes with PHP_CodeSniffer, to auto fix the found issues. This way we can provide own rules to find code that needs migration and provide a fix for each occurrence.

As PHP_CodeSniffer already parses the source code for us, it’s very easy to implement new rules and fixes.

By providing an easy to use command line interface, we are able to include this migrations into existing workflows like pre-commit hooks or continuous integrations. Also we can run them on our local dev environments without complex installation instructions.

IDEs like PHPStorm and editors like Vim or Sublime Text already integrate PHP_CodeSniffer directly, or through plugins, and make it easy to warn about outdated code while writing new code. Also you can fix the issues from within your IDE / Editor. That makes PHP_CodeSniffer the perfect tool for automated code migrations.

Benefits ¶

• Integration into IDE / Editor
• Automation on CLI through CVS hooks / continuous integrations
• Easy to use
• Easy to extend
• A lot of available examples

The basics ¶

PHP_CodeSniffer comes with two command line tools phpcs for “PHP_CodeSniffer” which will check code against configured rules, and phpcbf for “PHP Code Beautifier and Fixer” which will adjust the code accordingly.

To provide your own migration, you need a new “Coding Standard” that can be used with these tools. Creating a new standard basically consists of creating a new folder, and a ruleset.xml which configures the standard. Also you can provide further Sniffs, which are PHP files including further rules. A “Coding Standard Tutorial” can be found at github.

Afterwards you need to “install” your new standard to enable the cli tools to execute the standard, that’s it. Installation of a new standard is also explained at Github, see “Setting the installed standard paths”. Also you can always provide the path to the standard, like:

$ phpcs --standard=/path/to/MyStandard test.php

How to ¶

As already a tutorial is provided by the project itself, I’ll provide further information not covered by the tutorial.

Own standard ¶

There is not much to say, just follow the tutorial and “create” a standard. As documented on github in the Annotated ruleset.xml it’s possible to include existing standards. E.g. gather existing sniffs from other projects, here are some examples:

• Magento PHP_CodeSniffer Coding Standard to check for best practices in Magento development.
• compatibility of the code with PHP versions to check for your PHP version.
• Drupal.
• TYPO3 Sniffs

Chances are that your cms / framework already provides a basic standard. At least PSR-2 which is already included in PHP_CodeSniffer installation is available.

Own Sniffs fixing code ¶

Once you configured the standard to include some existing standard, you need to add custom Sniffs as documented in the tutorial.

Most of the time your sniff will gather the tokens for current file and check some parts before or after the current stack to match certain conditions. E.g. check out the small examples at our own project in the LegacyClassnames folder at Github.

Make sure you make use of $phpcsFile->addFixableError whenever possible, to allow phpcbf to fix the issues. Otherwise it’s not about automated code migration, but just a check providing you with a list of violations.

Allowing to fix an error instead of only reporting the error is done by the following code:

$fix = $phpcsFile->addFixableError(
    'Legacy classes are not allowed; found "%s", use "%s" instead', // Error message
    $stackPtr, // Stack pointer
    'legacyClassname', // identifier inside the sniff
    [$classname, $this->getNewClassname($classname)] // Arguments used for replacement in error message
);

// Check whether fixing is active
if ($fix === true) {
    // Execute code to modify the tokens to fix the violation
    $phpcsFile->fixer->replaceToken($stackPtr, 'new token content');
}

You add the error as usual but using a different method. This method will return true if phpcbf is run and fixes should be done. If fixes should happen, use the replaceToken method of the PHP_CodeSniffer_Fixer class to adjust the code.

$stackPtr in the above example is no longer the provided $stackPtr from PHP_CodeSniffer, but the token that contains the violation. So if you register T_NEW but the classname afterwards contains the violation, $stackPtr is the token of the classname.

Further help for new sniffs ¶

While writing own sniffs, some information might be handy, that are:

Where do I find the tokens I can return inside of the register method? ¶

The first step is to check out the official php tokens at php.net Also check out the additional tokens of PHP_CodeSniffer itself inside the Tokens.php Also note that Tokens.php contains some collections you can reuse, e.g.:

/**
 * Tokens that are comments.
 *
 * @var array(int)
 */
public static $commentTokens = array(
                                T_COMMENT                => T_COMMENT,
                                T_DOC_COMMENT            => T_DOC_COMMENT,
                                T_DOC_COMMENT_STAR       => T_DOC_COMMENT_STAR,
                                T_DOC_COMMENT_WHITESPACE => T_DOC_COMMENT_WHITESPACE,
                                T_DOC_COMMENT_TAG        => T_DOC_COMMENT_TAG,
                                T_DOC_COMMENT_OPEN_TAG   => T_DOC_COMMENT_OPEN_TAG,
                                T_DOC_COMMENT_CLOSE_TAG  => T_DOC_COMMENT_CLOSE_TAG,
                                T_DOC_COMMENT_STRING     => T_DOC_COMMENT_STRING,
                            );

How do I run only one sniff, the one I’m working on right now? ¶

Just provide the --sniffs option during CLI calls:

phpcbf -p --colors -s --sniffs=Typo3Update.LegacyClassnames.DocComment Classes/Controller.php

How do I get the sniff name of a sniff? ¶

1. Coding Standard name (Typo3Update)
2. Folder name (LegacyClassnames)
3. File name (DocCommentSniff.php -> DocComment)

Also they are displayed by running phpcs with option -s, like:

$ ./vendor/bin/phpcs -s <path>
8 | ERROR | [x] Legacy classes are not allowed; found
  |       |   backend_toolbarItem
  |       |   (Typo3Update.LegacyClassnames.Inheritance.legacyClassname)

Make parts configurable through ruleset.xml ¶

All public properties of sniffs are configurable through the ruleset.xml. So all you have to do, is to provide a public property as an option. The properties are configured on a sniff base. So extending a class with a public option makes the option available to all sniffs, same goes for traits.

The configuration will look like the following:

<rule ref="Typo3Update.LegacyClassnames.DocComment">
    <properties>
        <property name="allowedTags" type="array" value="@param,@return,@var,@see,@throws"/>
    </properties>
</rule>

You have to define the rule to configure, followed by Tag properties that contain each property you want to configure as a tag inside.

You can also take a look at Customisable Sniff Properties.

REPL your sniffs ¶

I prefer to use psysh nowadays and it makes it easy to “discover” your code and write your sniffs interactively. It’s an Symfony Cli App you can call from within your code by including the following line:

require_once('~/bin/psysh');eval(\Psy\sh());

Like an xdebug_break() the execution will halt and you are inside the app and can play around.

Result ¶

The result is a check like:

$ ./vendor/bin/phpcs -p --colors -s <path>
E


FILE: <path>
----------------------------------------------------------------------
FOUND 5 ERRORS AFFECTING 5 LINES
----------------------------------------------------------------------
 8 | ERROR | [x] Legacy classes are not allowed; found
   |       |   backend_toolbarItem
   |       |   (Typo3Update.LegacyClassnames.Inheritance.legacyClassname)
14 | ERROR | [x] Legacy classes are not allowed; found TYPO3backend
   |       |   (Typo3Update.LegacyClassnames.DocComment.legacyClassname)
16 | ERROR | [x] Legacy classes are not allowed; found TYPO3backend
   |       |   (Typo3Update.LegacyClassnames.TypeHint.legacyClassname)
48 | ERROR | [x] Legacy classes are not allowed; found t3lib_extMgm
   |       |   (Typo3Update.LegacyClassnames.StaticCall.legacyClassname)
61 | ERROR | [x] Legacy classes are not allowed; found t3lib_div
   |       |   (Typo3Update.LegacyClassnames.StaticCall.legacyClassname)
----------------------------------------------------------------------
PHPCBF CAN FIX THE 5 MARKED SNIFF VIOLATIONS AUTOMATICALLY
----------------------------------------------------------------------

Time: 35ms; Memory: 5Mb

And of course the auto migrated code.

History ¶

We are currently using PHP_CodeSniffer to auto migrate TYPO3 Extensions in a 6.2 installation, to be compatible with the latest LTS release. Due to massive namespace changes in versions between the original writing of the extensions, we make heavy use of PHP_CodeSniffer to auto migrate the code.

Before we did some small research how TYPO3 migrated the code itself and how Neos / Flow does the job. But plain regular expressions are not enough for us. Also regular expressions are not as well integrated into IDEs and editors as PHP_CodeSniffer.

You can check out the source code at my git instance: DanielSiepmann/automated-typo3-update.

• PHP_CodeSniffer at Github
• PHP_CodeSniffer documentation (wiki) at Github
• PHP_CodeSniffer documentation at php.net
• DanielSiepmann/automated-typo3-update

The idea ¶

The basic idea is to use the HMENU like all other solutions, but instead of using the optionSplit we are using data to inject the values from a language file.

The TypoScript ¶

tmp.language = HMENU
tmp.language {
    wrap = <ul>|</ul>

    special = language
    special {
        value = 0, 1, 2, 3
    }

    1 = TMENU
    1 {
        NO = 1
        NO {
            allWrap = <li>|</li>

            ATagTitle {
                data = LLL:EXT:example/Resources/Private/Language/Frontend.xlf:languageMenu.title.{field: _PAGES_OVERLAY_LANGUAGE}
                data {
                    insertData = 1
                }
            }

            stdWrap {
                cObject = COA
                cObject {
                    1 = TEXT
                    1 {
                        data < tmp.language.1.NO.ATagTitle.data
                    }
                }
            }
        }

        ACT < .NO
        ACT {
            allWrap = <li class="active">|</li>
        }

        USERDEF1 = 1
        USERDEF1 {
            doNotShowLink = 1
        }
    }
}

On Line 5 to 8 we define the menu as you normally would. With one exception, we add all sys_language_uid’s, not only the one we want on the current site. Via NO all existing languages that are not currently active are rendered. With ACT the current active language is rendered and via USERDEF1 we define to not show links to languages which are not available for the current site, depending on your configuration.

By Using USERDEF1 we don’t have to adjust the set of languages for each page.

Inside of data, the field: _PAGES_OVERLAY_LANGUAGE contains the uid of the current sys_language to render.

Using the data we can render the content of a language file. This file can be different for each language. This way we can adjust the labels for each language depending on the language.

The language file ¶

The language file for above example might look like the following.

<?xml version="1.0"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
    <file original="en" datatype="plaintext">
        <body>
            <trans-unit id="languageMenu.title.">
                <source>Default</source>
            </trans-unit>
            <trans-unit id="languageMenu.title.1">
                <source>Deutsch</source>
            </trans-unit>
            <trans-unit id="languageMenu.title.2">
                <source>Nederlands</source>
            </trans-unit>
            <trans-unit id="languageMenu.title.3">
                <source>English</source>
            </trans-unit>
        </body>
    </file>
</xliff>

The result ¶

The output will look like the following:

This solution was “invented” by Justus Moroni and myself during one project, as we thought that option split and adjusting the settings for each site is not the best way. Also we were just to lazy, you know programmer?, to adjust the configuration for each multisite.

And further resources to TYPO3 documentation which are used in this example:

• HMENU
• special = language
• TMENUITEM
• data
• cObject

What are Hooks in TYPO3? ¶

TYPO3 has a lot of processes like evaluating data, authenticating users, displaying content and so on. All of this processes are handled inside of TYPO3. In some use cases you want to hook into the process and manipulate the process. E.g. you want to modify data inserted into the backend, before they get persisted into the database.

This can be achieved by using a hook to modify the process. The hook allows you to include your custom PHP into the process, which will get executed.

The execution of Hooks is typically implemented like the following:

if (is_array($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_div.php']['devLog'])) {
     $params = array('msg' => $msg, 'extKey' => $extKey, 'severity' => $severity, 'dataVar' => $dataVar);
     $fakeThis = false;
     foreach ($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_div.php']['devLog'] as $hookMethod) {
         self::callUserFunction($hookMethod, $params, $fakeThis);
     }
 }

So your configured method will be called with some arguments, on line 5 in above example. Most of the time the arguments will be references enabling you to modify them.

Hooks differ in two ways:

1. Some hooks need your class to implement a specific interface, or method. Others, like above will just call the function or method you have provided, like a userfunc.

2. Also some hooks will provide arguments via reference and others via copy.

A simple example ¶

Let’s assume the following example: You want to provide latitude and longitude to frontend users, auto generated based on their address. That can be achieved by using a hook inside of \TYPO3\CMS\Core\DataHandling\DataHandler.

Configure the hook in ext_localconf.php of your extension like:

$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_tcemain.php']['processDatamapClass'][$_EXTKEY]
    = 'DanielSiepmann\FeuserLocations\Hook\DataMapHook';

This will register the class DanielSiepmann\FeuserLocations\Hook\DataMapHook to be processed. Inside of the class we have the following method, called during the process:

/**
 * Hook to add latitude and longitude to locations.
 *
 * @param string $action The action to perform, e.g. 'update'.
 * @param string $table The table affected by action, e.g. 'fe_users'.
 * @param int $uid The uid of the record affected by action.
 * @param array $modifiedFields The modified fields of the record.
 *
 * @return void
 */
public function processDatamap_postProcessFieldArray( // @codingStandardsIgnoreLine
    $action,
    $table,
    $uid,
    array &$modifiedFields
) {
    if(! $this->processGeocoding($table, $action, $modifiedFields)) {
        return;
    }

    $geoInformation = $this->getGeoinformation(
        $this->getAddress($modifiedFields, $uid)
    );

    $modifiedFields['lat'] = $geoInformation['geometry']['location']['lat'];
    $modifiedFields['lng'] = $geoInformation['geometry']['location']['lng'];
}

This method will get called for all data changed through DataHandler before they are processed by the DataHandler.

How to find hooks ¶

Hooks are always configured through $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS'], so finding hooks is as easy as a search for SC_OPTIONS. Of course you need to understand the surrounding code and where your hook should be executed to find the right place.

E.g. execute the following in your shell:

grep -n -C 5 "SC_OPTIONS" -r vendor/typo3/cms

Beside the core, also extensions might provide hooks, so adjust the path, vendor/typo3/cms, to search inside an extension.

Also all registered hooks can be found inside the backend “Configuration” module. Just select the TYPO3_CONF_VARS in dropdown and search for SC_OPTIONS. By registed I mean hooks that are already in use, it’s not a full list of available hooks.

Signal Slots ¶

Beside the concept of hooks, TYPO3 also provides the concept of Signal Slots. I will not document that concept here, there are already some blog posts about the topic:

• English blog post by Felix Oertel. No longer available.
• English blog post at usetypo3.com
• German blog post at typo3blogger.de
• Official Documentation of Signal Slots in Flow Framework

In general it’s the same idea, just implemented in an object oriented way.

Checkout the official documentation at Events, Signals and Hooks.

Also check out examples for userfunctions.

Also you can check how other developers make usage of hooks, e.g. in the example extension wv_feuser_locations.

Sphinx ¶

First let’s start with Sphinx to get a first documentation rendered on the local machine.

Install ¶

To use Sphinx, we need to install it. Up to date instructions can be found at Installing Sphinx.

Basically you can install via:

pip install -U Sphinx

Start documenting ¶

To start your documentation sphinx provides a CLI tool to create a new project:

sphinx-quickstart

It’s interactive and will ask all necessary information.

An example setup will look like:

Welcome to the Sphinx 1.4.3 quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Enter the root path for documentation.
> Root path for the documentation [.]: example-documentation

You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]: y

Inside the root directory, two more directories will be created; "_templates"
for custom HTML templates and "_static" for custom stylesheets and other static
files. You can enter another prefix (such as ".") to replace the underscore.
> Name prefix for templates and static dir [_]:

The project name will occur in several places in the built documentation.
> Project name: Example for Blogpost
> Author name(s): Daniel Siepmann

Sphinx has the notion of a "version" and a "release" for the
software. Each version can have multiple releases. For example, for
Python the version is something like 2.5 or 3.0, while the release is
something like 2.5.1 or 3.0a1.  If you don't need this dual structure,
just set both to the same value.
> Project version: 1.0.0
> Project release [1.0.0]:

If the documents are to be written in a language other than English,
you can select a language here by its language code. Sphinx will then
translate text that it generates into that language.

For a list of supported codes, see
http://sphinx-doc.org/config.html#confval-language.
> Project language [en]:

The file name suffix for source files. Commonly, this is either ".txt"
or ".rst".  Only files with this suffix are considered documents.
> Source file suffix [.rst]:

One document is special in that it is considered the top node of the
"contents tree", that is, it is the root of the hierarchical structure
of the documents. Normally, this is "index", but if your "index"
document is a custom template, you can also set this to another filename.
> Name of your master document (without suffix) [index]:

Sphinx can also add configuration for epub output:
> Do you want to use the epub builder (y/n) [n]: n

Please indicate if you want to use one of the following Sphinx extensions:
> autodoc: automatically insert docstrings from modules (y/n) [n]: n
> doctest: automatically test code snippets in doctest blocks (y/n) [n]: n
> intersphinx: link between Sphinx documentation of different projects (y/n) [n]: y
> todo: write "todo" entries that can be shown or hidden on build (y/n) [n]: y
> coverage: checks for documentation coverage (y/n) [n]: n
> imgmath: include math, rendered as PNG or SVG images (y/n) [n]: n
> mathjax: include math, rendered in the browser by MathJax (y/n) [n]: n
> ifconfig: conditional inclusion of content based on config values (y/n) [n]: n
> viewcode: include links to the source code of documented Python objects (y/n) [n]: n
> githubpages: create .nojekyll file to publish the document on GitHub pages (y/n) [n]: n

A Makefile and a Windows command file can be generated for you so that you
only have to run e.g. `make html' instead of invoking sphinx-build
directly.
> Create Makefile? (y/n) [y]: y
> Create Windows command file? (y/n) [y]: n

Creating file example-documentation/source/conf.py.
Creating file example-documentation/source/index.rst.
Creating file example-documentation/Makefile.

Finished: An initial directory structure has been created.

You should now populate your master file example-documentation/source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

As Sphinx is written in Python and used to document Python modules, most extensions can be omitted for your documentation, until you are working with Python code.

I definitely recommend to enable todo and intersphinx all the time. Also ifconfig can be helpful. But it’s just the kickstart and you can add extensions later on inside the configuration.

Also do yourself a favour and create the Makefile for easier usage.

Sphinx will setup a structure like:

.
└── example-documentation
    ├── Makefile
    ├── build
    └── source
        ├── _static
        ├── _templates
        ├── conf.py
        └── index.rst

5 directories, 3 files

You now can render the documentation by calling:

make html

Sphinx will generate the full HTML and write it to build/html. You can open the documentation using:

open ./build/html/index.html

The first output will look like in Figure i2.

You can now start writing the documentation, following the Sphinx Documentation, and adjust the look and feel, e.g. change the theme using one of the builtin Themes.

Github ¶

To make integration with Read the docs easier, we will publish our documentation as a Git repository to Github.

First of all you need to initialize a new Git-Repository, of course you can also use Mercurial or something else. Do so by running:

echo "build" > .gitignore && git init && git add . && git commit -m "First version"

Next sign up at Github if you don’t have an account yet. Create a repository, which is only possible if you are logged in. Github should redirect you to your new repository with a URL scheme like <UserName>/<RepositoryName>. Add the repository at Github to your local repository by running:

git remote add origin https://github.com/<UserName>/<RepositoryName>.git && git push --mirror

If you reload the web Gui of Github you should see a first commit.

Github provides full documentation at https://help.github.com/ if something is not clear or you need further help setting everything up.

Read the docs ¶

To host our documentation without the need to setup the rendering or web space, we will use Read the docs.

Therefore register at Read the docs, and connect the account to your Github account. You can now see all your Github repositories and select the created one to automatically render the documentation on new commits. Have a look at figure i6 and i4.

You are now ready to go, Read the docs should already render your documentation. You have an overview at https://readthedocs.org/dashboard/ where your project should appear. Navigate to the project by clicking the title and you should the Last Built on the right mentioning whether everything worked. Also at top you have a navigation. Go to Builds and you can get an detailed view what was going on and where something broke.

To setup further branches in your repository to render, head to Versions and set them up.

The green Button View Docs will bring you to your generated documentation. It’s already online and all you have to do in the future is to do a:

git commit -m "Made changes" && git push

Read the docs will detect the changes and render your documentation.

Plantuml ¶

Everything is working now. Let’s add some sugar with nice looking UML diagrams to explain the structure of our project, or some complex workflows.

To provide nice looking UML diagrams like Figure i5.

We will use PlantUml. As it’s not available as a Debian package yet, Read the docs doesn’t provide rendering for it. So you have to render the images on your local machine and provide them to Read the docs.

Install ¶

First of all you need to install Java and Graphviz to draw the diagrams. Head over to http://plantuml.com/starting and http://plantuml.com/faq-install to follow the installation.

Provide wrapper ¶

After PlantUml is on your local system, make your live easier by providing the following shell script inside your $PATH to just call plantuml in the future anywhere on your CLI:

#!/usr/bin/env sh -e
java -Djava.awt.headless=true -jar $HOME/Applications/plantuml.jar -tsvg -failfast2 "$@"

Adjust the path according to your location of plantuml.jar. This wrapper will run PlantUml without the GUI, and generate SVGs as default for all provided PlantUml source files.

Integration into Documentation ¶

To integrate PlantUml into your Sphinx documentation, you can setup the following structure:

.
└── example-documentation
    ├── Makefile
    ├── build
    └── source
        ├── uml
        │   └── example.uml
        ├── _static
        ├── _templates
        ├── conf.py
        └── index.rst

6 directories, 4 files

And adjust your Makefile to render all PlantUml files for you.

Add the following entry to your Makefile:

plantuml:
    plantuml -psvg -o ../images/uml/ ./source/uml/*.uml

You now can call:

make plantuml

That will create a new folder with generated images:

.
└── example-documentation
    ├── Makefile
    ├── build
    └── source
        ├── images
        │   └── uml
        │       └── example.svg
        ├── uml
        │   └── example.uml
        ├── _static
        ├── _templates
        ├── conf.py
        └── index.rst

8 directories, 5 files

To include the diagram into your documentation, use the image or figure directive of rst.

To ease workflow, adjust your Makefile further to run plantuml also for html and such by using:

html: plantuml

Adjust look ¶

At the moment we will get the default styling of PlantUML which is not nice in our Template. You can adjust the styling by providing the a file called plantuml.cfg with the following content:

skinparam backgroundColor white

skinparam note {
    BackgroundColor #F1FFFF
    BorderColor #2980B9
}

skinparam activity {
    BackgroundColor #BDE3FF
    ArrowColor #2980B9
    BorderColor #2980B9
    StartColor #227BC6
    EndColor #227BC6
    BarColor #227BC6
}

skinparam sequence {
    ArrowColor  #2980B9
    DividerBackgroundColor  #BDE3FF
    GroupBackgroundColor    #BDE3FF
    LifeLineBackgroundColor white
    LifeLineBorderColor #2980B9
    ParticipantBackgroundColor  #BDE3FF
    ParticipantBorderColor  #2980B9
    BoxLineColor    #2980B9
    BoxBackgroundColor  #DDDDDD
}

skinparam actorBackgroundColor #FEFECE
skinparam actorBorderColor    #A80036

skinparam usecaseArrowColor   #A80036
skinparam usecaseBackgroundColor  #FEFECE
skinparam usecaseBorderColor  #A80036

skinparam classArrowColor #A80036
skinparam classBackgroundColor    #FEFECE
skinparam classBorderColor    #A80036

skinparam objectArrowColor    #A80036
skinparam objectBackgroundColor   #FEFECE
skinparam objectBorderColor   #A80036

skinparam packageBackgroundColor  #FEFECE
skinparam packageBorderColor  #A80036

skinparam stereotypeCBackgroundColor  #ADD1B2
skinparam stereotypeABackgroundColor  #A9DCDF
skinparam stereotypeIBackgroundColor  #B4A7E5
skinparam stereotypeEBackgroundColor  #EB937F

skinparam componentArrowColor #A80036
skinparam componentBackgroundColor    #FEFECE
skinparam componentBorderColor    #A80036
skinparam componentInterfaceBackgroundColor   #FEFECE
skinparam componentInterfaceBorderColor   #A80036

skinparam stateBackgroundColor #BDE3FF
skinparam stateBorderColor #2980B9
skinparam stateArrowColor #2980B9
skinparam stateStartColor black
skinparam stateEndColor   black

More about styling can be found at http://plantuml.com/skinparam , http://plantuml.com/sequence-diagram .

And adjust your Makefile to provide this file to PlantUML:

plantuml:
    plantuml -config plantuml.cfg -psvg -o ../Images/Uml/ ./Uml/*.uml

You should now be able to write basic documentation with hosting at Read the docs. The following links can be startpoints to get further:

• Sphinx autobuild will detect changes and autogenerate documentation. Also comes with server and autoreload.
• Sphinx intersphinx allow linking between sphinx projects without need to know urls.
• Sphinx builtin Themes

Also the following links as a collection:

• Sphinx
• Github
• Read the docs
• Plantuml