History (through Jisse's pink glasses)

First, let's consider the history. When 2.0.0 came out, it was full of bugs. You could say that Magento as a company was solely accountable for this, but I don't think that's fair: Since the alpha versions came out, Magento was pushing extension providers and technology vendors to start testing things. But when 2.0.0 Stable was released, it seemed to me as if the entire community was just waiting for Magento to deliver them a stable, yet completely new product ... for free. (I'm pretending that enterprise packages were never there.)

Only with 2.0.0, people started to play with things. And this led to numerous issues, found in the wild, being reported on GitHub and remaining pending for months (because - I guess - the Magento engineers were flooded with work). When 2.1 came out, it seemed a lot of issues were fixed. But it was no easy fix, because core engineers found various things that needed to be refactored (duplicate dependency injection (which was later undone again), DI compile issues, broken JavaScript, etc).

With 2.2 and 2.3, the amount of issues gradually decreased - more issues were fixed, less breaking changes were introduced. But while Magento sticks more and more to semantic versioning and testing to try to help developers determine what goes wrong where, still too many upgrades become messy. Due to ... what?

History ... in short

In short, too many times, agencies spent a lot of time on troubleshooting failed upgrades. With every major upgrade however, some kind of failure is too be expected - that's why it is a major upgrade and not a minor upgrade. But it either seems as if the expectations are wrong or as if the upgrade is not working as expected or the major upgrade is too major. Either way, something with expectations.

Why upgrade to 2.4?

Upgrades should be considered a good thing. For instance, Magento 2.4 brought new things. Like ... well, what did it actually bring? It brought us a breaking change with MySQL Full Text Search, which is gone now and you should use ElasticSearch instead. It brought controller decompositioning but this is only useful for new code.

As I consider it, 2.4.0 brought a breaking change with search. And it patched a few bugs. And it brought a couple of new things like a full blown module to create a simple security.txt file, Two Factor Authentication, some MSI performance, PHPUnit upgrade, a new media gallery. I found the breaking change with MySQL to weigh more than those small addons.

If you don't actually need the new features, if you are just upgrading for the sake of upgrading, upgrading to 2.4.0 could be a big (expensive) thing. Magento is now helping you stay on an older release, by adding patch releases (2.3.5-p2) on top of the already complex semantic versioning. To allow for security fixes to be implemented and not the rest.

Breaking changes between Magento 2.4 or 2.3?

Still, all this talk about pros and cons of new releases doesn't explain to me what actually is breaking when you attempt an upgrade to Magento 2.4. Disclaimer, I'm not a full time developer working for an agency. I experiment a lot, I talk to agencies quite a bit, but I don't have a first-hand frustration with these upgrades. I've helped along with some 2.4 upgrades with little trouble. And this makes me wonder what the major changes are within Magento 2.4.0 - besides the ones being mentioned in marketing blogs. And it makes me wonder what those issues are, that I hear about.

The good thing is that Magento is sticking to semantic versioning, so that their versioning reflects the type of changes being made to modules. Ideally, this means that - if we are depending in the correct way (whatever that means) on the Magento core - we don't need to take notice of the bug fixes and newly added features, but we should take note of the breaking changes only. Hmmm, if only.

Magento 2.4.0 versus 2.3.5-p2

So I've used a command composer show | grep magento/ > deps.txt to list all composer packages (with their versioning) of a Magento 2.4.0 site and compared this with a similar list of 2.3.5-p2. The outcome is that the Adobe modules heavily changed, there were some PayPal changes, some MSI modules were added, etcetera. But the major bulk of changes seems to be with the following major bumps:

• magento/framework 102 to 103
• magento/framework-bulk 100 to 101
• magento/magento-coding-standard 3 to 5
• magento/module-backend 101 to 102
• magento/module-catalog 103 to 104
• magento/module-catalog-search 101 to 102
• magento/module-cms 103 to 104
• magento/module-customer 102 to 103
• magento/module-elasticsearch 100 to 101
• magento/module-sales 102 to 103
• magento/module-url-rewrite 101 to 102

The listing shows that Magento Framework received a major bump: The Laminas migration, cache refactoring, new interfaces and for the major part, the drop of MySQL as a search engine. Similar changes are there in the other modules as well.

Obviously, this could lead to bugs - hence the major change. But with my own upgrades (disclaimer: with silly, manageable shops that are not that complex) or with my 50+ shared extensions, I found no issues at all (except for this search thing, upgrading PHPUnit and bumping new dependency versions)!

(Note: Don't use my blog for upgrading, use the official release notes. Especially, see the document on breaking changes.)

The pitfalls of semantic versioning

Semantic versioning can be a good thing. This is my opinion and I know that others have a different opinion. Communicating software changes into three versions (major-minor-patch) is nice and it allows for a better communication, if people stick to the same standard. Creating changelogs becomes easier, knowing what kind of package upgrade contains what kind of changes is something I consider useful.

As I see it, the issue with semantic versioning is not really related to the semantics of the versioning itself. It more has to do with the practical implementation. If we write down a dependency with package A as ^1.0, then an upgrade to 2.X would be a major upgrade and actually a tool like composer would prevent this upgrade from taking place automatically. Instead, a manual switch from ^1.0 to ^2.0 would be needed, by editing the composer.json file. In practice, few people do this, so that everybody is still stuck on version 1.X with all of its security issues.

But this flaw deals more with the package maintainer not maintaining older majors anymore, than it has to do with semantic versioning itself. And the problem could be downsized if the maintainer simply keeps maintaining older majors as well, when it comes to important issues like security fixes.

On the web, you can find references to sentimental versioning as well. I love that: Semantic versioning is not neccessarily a bad thing, it simply requires a bit of empathy.

Testing is still key

While semantic versioning tries to solve an issue, it will not solve everything. To guarantee that certain functionality still works after an upgrade, testing is key. The Magento core ships with unit tests, integration tests, functional tests, API tests, static tests and a lot more.

Third party extension should be required to follow this pattern as well, by adding integration tests to the least, functional tests and unit tests where possible.

And in a project, ideally there would be end-to-end tests and functional tests to proof the sum of everything still works in the way expected as well: Codeception, Cypress, the likes.

Interpreting composer upgrade errors

All of this talk on semantic versioning and testing is needed, because Magento is committing to it. By studying the changes made to versioning, hopefully, you as a developer have a better understanding on the impact that such a version has upon your own code, your own extension, your own project. By testing things, you can know for sure. Testing might show things are breaking, semantic versioning might explain why.

Unfortunately, semantic versioning makes things harder as well. Because of the 100s of dependencies that composer needs to manage, the errors when upgrading via composer are sometimes mystical and scary. Numerous times per week, composer is requesting that I downgrade PHP back to 5.6 and Magento back to version 2.0, because of some dependency calculation going wrong in one of my development packages.

First off, it is just proof that composer is doing a complex task. But most of the time, the issues I get are not due to Magento core packages, PHP dependencies or composer itself. No, most of the time, I fail to upgrade due to third party extensions.

Third party extensions and bundled extensions

When reviewing shops with bad performance, weird issues or installations that fail to upgrade, about 90% of the time (or more), these issues are caused by third party extensions. Maybe it is due to the fact that that other 10% (the Magento core issues) are well known issues that require less troubleshooting.

But fact remains that the most horrific Magento situations I have encountered are with third party extensions. And more than often, the involved extension has issues that could have been prevented by sticking to semantic versioning. It is actually not about the versioning at all, but about the code. But the versioning allows you to make sense of the code changes. The versioning keeps track of the dependencies.

With the latest 2.4.0 release, I became a bit more angry than usual, because the core also shipped with third party extensions - bundled extensions - and they actually contained flaws as well: Flaws that prevented me to install modules, flaws that prevented me to run tests, flaws that led to bugs in production. Some of these bugs should have been prevented, if the tests of Magento include a simple composer installation like all agencies would do and a simple integration test run like all extension developers (should) do. I don't want to bash anyone specifically (I've done that already earlier), but I want to point out that these kind of bugs are harming the trust of agencies.

Where does this all lead?

I started writing this blog to try to come to some kind of explanation on why Magento upgrades have become so hard. For instance, Magento seems to take up its own responsibility to release a product that has both new features and breaking changes (not a goal on its own, but a sensible thing to do if you want to move forward). Sometimes, their release mechanism contains flaws - introducing new bugs by fixing old bugs, shipping untested code. But all in all, I think it is more than fair to say that Magento has improved all of this hugely over the years. And therefore, an agency should also pick up on their responsibility - testing as a base, semantic versioning if it is reasonable, picking only extensions that are reliable.

Another main conclusion to make is that if there are third party extensions used in your project, not using composer, not sticking to semantic versioning and not shipping with any kind of tested code, this means you are waiting for disaster to happen. No wonder that major upgrades fail. Disclaimer, this does not mean that projects with 100% tested code and semantic versioning to the max have zero issues. But you will definitely experience less issues, much less issues.

Ok, semantic versioning in the core and third party extensions are good, testing is good. But what kind of conclusions can be make on the current release of Magento 2.4.0?

Some conclusions

First of all, Magento should test any release by not relying upon automated tests within the GitHub repository only, but also by running composer installs, like regular developers do. Some bugs have come to rise because of composer leading to different outcome than a regular GitHub clone installation.

Second, bundled extensions should live up to even higher standards then the rest of the extensions out there. This means automated tests all over the place, transparent releases, splitting up modules where possible, etcetera. Numerous developers have become frustrated with issues caused by bundled extensions. So living up to higher expectations is a natural thing. You don't get the love of developers by just shipping your extension with the Magento core by throwing money at it - no, you will the most appreciation by setting a superb technical example.

Third, the community needs to be acting as an open source community, not a bunch of leechers. The Magento product is not free. We are making money from it, so let's collaborate on making it better: The extension quality still needs to be increased. So installing extensions of bad quality will lead to more issues. Any project should therefore ideally check up upon things like testability, code quality, code compliance, deprecated issues and more. And the same counts for the Magento core project: Issues should not be reported on GitHub just to sit and wait for Magento to do its job. No, instead, issues should be filed and followed up by an even bigger investment by the reporter to come up with a Pull Request to fix things. The true nature of open source. Not free beer, but free speech.

Final words

This blog is long and it could grow even longer the more I think about this topic. But I think it is important to realize that Magento is not a static product delivered by some evil company that makes huge money from us, it is a community product, it is something I personally use for free, it is something that we all need to invest in.

So if agencies do not want to upgrade to a new Magento 2.4.0 because of reasons, I understand and accept that. But those reasons actually point to an unhealthy situation that requires fixing. Let's not wait only for Magento to provide the fix. Let's fix it, because that's the actual meaning of open source: We can fix it, because it is open source.