Over the past 12 months, our teams have completed numerous Drupal upgrades. We would like to share our experiences and knowledge with anyone who has yet to undergo this process to help make it smoother for you.
Background
From Drupal 8, upgrades became a lot easier with several tools to semi-automate the process.
In addition, if you, like CTI Digital, have a number of sites to upgrade and different developers working on each of them, you could easily duplicate effort as many contributed modules are used in most sites - e.g. Webforms and Paragraphs, just to name two. Therefore, this article also contains the approach that CTI Digital took in managing the latest upgrades to Drupal 10 of around 50 sites (some still on Drupal 8) to ensure knowledge transfer within the team.
This article looks at the tools available and the lessons we learned.
Why is it easier since Drupal 8
The approach taken by the Drupal core team with Drupal 8+ has made things a lot easier. That approach is to deprecate hook functions, classes, class methods, etc., while introducing their replacements. Then, a list of deprecated features that will be removed in the next major version is produced and fixed at a specified minor version of the current Drupal major version (8.9 for D8 and 9.5 for D9). Drupal's main dependency, Symfony, takes the same approach.
This approach clarifies what is deprecated in Integrated Development Editors (IDEs) and allows various tools to semi-automate the process. It enables them to find deprecated code, advise on what changes are needed and even make many of them. Finally, the Drupal CI systems automatically run tools to spot deprecated code on contributed modules and produce patches, which are then attached to a new issue created for each module. These can then be tested, modified and approved by the community.
Tools
So, what are these tools? There are command line tools for picking up and making changes, but there are also contributed modules (often using those command line tools). The ones we at CTI Digital found useful are listed here.
The three tools we used are:
-
Upgrade status - a Drupal module that identifies what needs to be done
-
Rector - a Drupal command line module that changes code automatically
PHP Codesniffer - a tool to find code issues by rule; in this case, PHP 8.1 compatibility
Upgrade Status
This is the first one that should be installed on a development copy of each site to identify the work that needs to be done. It is a contrib module that can be installed with composer:
composer require drupal/upgrade_status
This tool provides a report through your site's UI (Reports/Upgrade status admin/reports/upgrade-status). This report details everything you need to do - down to lines of code that need changing per module or theme. This detail includes what to change each line to and if Rector - see next section - can make the change automatically.
It will tell you the following:
-
Environmental factors - such as PHP, database and Drush versions required. For Drupal 10 it will also include a list of invalid permissions and deprecated core modules that you have installed
-
Modules/themes that are not installed
-
Contributed modules/themes that have an upgrade available (including if that is compatible)
-
Contributed modules/themes that have nothing available (with a link to an issue queue search for issues containing the text ‘Drupal 10’)
-
Modules/themes that have changes that could be made by Rector
-
Modules/themes that have changes that need to be fixed manually
Each module/theme can be scanned for detailed depreciations, which will also categorise each as one that can be done with Rector or one that needs to be done manually. This can be run where there is not a compatible version or patch already available.
For all contributed modules/themes, you have the following options:
-
Upgrade to a more recent version
-
Apply a readily available patch on its issue queue
-
Upgrade and patch
-
Write a patch from scratch (using Rector where possible)
-
Remove an unused module/theme from the code
-
Replace the module/theme
It should be noted that removing or replacing a module/theme that is currently installed requires two deployments. The first uninstalls the module/theme, and the second removes it from the code base. If you do both in one go, you will get issues with Drush commands in the deployment.
Drupal Rector
This command line tool can automatically make many of the necessary changes. It can be installed in a project with:
composer require palantirnet/drupal-rector --dev
Then, a file called rector.php needs to be copied from the vendor/palantirnet/drupal-rector folder to the Drupal root directory (web in the examples here).
You can then run this tool on any module/theme with:
vendor/bin/rector process web/[modules or themes]/[SUB_FOLDER]/[YOUR_MODULE] --dry-run
This will find what needs to be changed, and if you are happy with the changes, removing the –dry-run option will, of course, allow it to do its thing.
PHP Codesniffer
This command line tool is not specific to Drupal core upgrades. It looks for particular code patterns, and you can install add-ons to look for anything, including PHP versions. Since Drupal upgrades often include PHP upgrades, it is at least worth running this tool on all custom code.
In order to test for PHP 8.1 ( required for Drupal 10), the following will install the tools you need:
composer require --dev phpcsstandards/phpcsutils:"^1.0@dev"
composer require --dev phpcompatibility/php-compatibility:dev-develop
You can confirm that this worked by running:
vendor/bin/phpcs -i
This command will list what sniffs are installed and will include PHPCompatibility.
Then the following can be run to test all your custom modules:
vendor/bin/phpcs -p web/modules/custom --standard=PHPCompatibility --runtime-set testVersion 8.1 --extensions='php, module,inc,install,theme'
This will test for PHP 8.1 compatibility, specifically in all PHP code files. You can do the same for any custom themes.
Deprecations not spotted by the tools
There are some deprecations that the tools do not spot. They are services, libraries and hook functions. In the case of services, these are the calls to \Drupal::service('<service name>'). If this call is assigned to a variable and that variable is given the relevant class (/* @var */), then that class will also be deprecated and picked up. Also, if you inject a service into a class, the service's related class will be picked up.
The only solution we found was to create a text file with one service, library or hook function per line and use grep to search the custom code:
grep -r -f [textfile] web/modules/custom
And the same for custom themes.
The upgrade steps
Apart from upgrading and patching contrib modules and themes, fixing custom modules and themes, and removing unused code, there are two extra steps for D9 to D10 upgrades.
The first is to fix an old issue of invalid permissions. In D8, modules/themes were not required to delete permission allocations to user roles when they deleted a permission that they created. One of the minor versions of D9 firmed this up, and a core database update removed any such orphans. However, this did not always work, and the Upgrade status report lists the user roles with non-existent permissions that must be deleted - manually from the configuration yml files.
The second is core modules and themes that have been deprecated and are being deleted from D10. The Upgrade status report will list the core modules and themes the site uses. All of these have a contributed version that can be added to a project if needed. Detailed recommendations are also available here.
The final upgrade to core
This should simply be a case of running the Composer command once everything else is done:
composer require drupal/core-* –with-all-dependencies
This presupposes that the site is built with the standard drupal/core-recommended and related packages.
However, you will often find that Composer finds some requirements clashing with D10 or its dependencies.
The main reason is that a module/theme is patched to D10, including the change to the info.yml file's core_version_requirement and the composer.json file. Composer will have an issue with this. This is because Composer is using the files in the repository to determine compatibility, not the patch file changes. However, there is a solution with 'lenient'. This Composer add-on allows you to ask Composer to be lenient on the constraints of individual packages.
The command to add the lenient package to the site is:
composer require mglaman/composer-drupal-lenient
The command to add a list of modules and themes to the allowed list is:
composer config --merge --json extra.drupal-lenient.allowed-list '["drupal/YOUR_MODULE1", "drupal/YOUR_MODULE2" …]'
In addition, some projects have drupal-composer/drupal-scaffold as a dependency. This is deprecated, and you will get a notice about that. It is to be replaced with drupal/core-composer-scaffold. Finally drupal/console is incompatible with D10 and needs to be removed.
Managing upgrades for a large number of sites
As mentioned at the top of this article, there could be a lot of duplicate effort and different approaches taken if steps are not taken to organise the upgrade of a large number of sites.
The solution we at CTI Digital came up with is simply maintaining a spreadsheet of information about contributed modules/themes. This would contain recommendations (version, available patches) and any complications or special instructions associated with that module or theme's upgrade, etc. It is maintained as more sites are audited and then upgraded with findings as we go to assist later sites. This can also be applied to any custom modules/themes that are shared with more than one site.
As each site is audited for the effort required, the spreadsheet is referred to and added to so that effort is not duplicated and the spreadsheet is kept up to date with findings from each site. Also, any approaches that should be followed on all sites are kept here - e.g. we replaced all sites using Swiftmailer with Symfony Mailer for mail management.
The other primary approach was to consolidate effort patching contributed modules/themes that are not ready. CTI Digital's work to improve a contributed module/theme is always contributed back to drupal.org. A link to this new patch is stored in the database for the following site that uses this module/theme.
What about Drupal 8 to Drupal 10
CTI Digital still had a few sites on Drupal 8, which needed to be upgraded to Drupal 10 rather than two separate upgrades.
There are factors that affect this:
-
Upgrade status will only give what needs to be done to get to D9
-
Drush does not cope with a deployment that goes from 8 to 10 in one go
-
A significant number of contributed modules/themes have versions that are 8 and 9 compatible and versions that are 9 and 10 compatible, rarely one version compatible with 8, 9 and 10
-
A few contributed modules/themes have the newest version drop support of D9
This means you have to do an 8-9 audit (with the dev copy on D8.9 minimum), upgrade the dev copy to D9 and then do a new audit from 9-10. You will be able to assess the complete upgrade requirement for all the modules/themes identified by the first audit, but the second audit will find modules/themes currently compatible with D9 but not D10 that are missed by the first. The second audit will also supply invalid permissions and core modules/themes that are removed from D10.
You must also upgrade as two deployments minimum (one to D9 and the second to D10). Some modules/themes will be upgraded in the D8 site before upgrading to D9, and some in the D9 site before upgrading to D10. Some will have to be upgraded twice. Some will need to be upgraded when the site is on D10 (occasionally at the same time as the core upgrade). It will depend upon the module's upgrade path and the version the D8 site is on.
What is next?
You may be wondering if anything can make things easier for D11. Although the overall effort can not be reduced, it can be spread out.
If you regularly upgrade contributed modules and themes to the most recent version, you will reduce the work when it's time to upgrade to D11.
If you also run regular Upgrade Status reports on your D10 site, you can create work dockets for changes to your custom code. It can also be used to identify contributed modules and themes that you use that have not yet been upgraded. You could create work dockets for replacing these modules or contributing patches to upgrade them to D11. These can be spread out between now and the need to upgrade to D11.
The ultimate goal of these approaches is that you only have to upgrade the core when it is time to upgrade to D11.
To get ahead, speak to one of our Drupal experts and book a 30-minute session today to discuss a migration or upgrade for your business.
