Steady State Update™ process

Introduction

For the last 10 years, I have been working as a freelancer for high-profile Drupal agencies. During this time, I have developed, migrated, and handled DevOps for numerous Drupal websites.

But just recently I came to a new insight. An insight that allows for efficient Drupal 7 to Drupal 10 upgrades, with way less risk than the industry-standard “let’s build a new site” approach.

The insight is that it is possible to capture and compare the state of a web application with automated tools. One example is a tool that takes screenshots, compares these screenshots and is able to compare screenshots that were taken at different moments. Other tools capture the application from different perspectives. When all relevant perspectives are combined, the automated tool will tell if the application behaves the same after the upgrade as it did before.

Enter “Steady State Update™”

A “Steady State Update” is a upgrade where the state before and after the upgrade is … steady; the same. Nothing changes.

The opposite of a “Steady State Update” is a “rebuild”. In a rebuild a new site will be built. And when we’re at it, the site gets a new design, a new sitemap and new visual components to create pages.

Human vs Automated Validation

In a “rebuild” we need a human being that validates if everything was done correctly. We need a human being, because computers are notoriously bad in comparing things that are similar, but not equal. On the other hand, computers are very good at comparing things that should be equal. And even if just one letter on the website is misspelled the computer will tell us.

When doing regular updates developers and web agencies call unplanned changes “regressions”. We have our tool boxes filled with automations that tell us if a website has regressions. We prefer to not bother clients with regressions, but instead find and fix them as early as possible.

Why “Steady State Updates” are more efficient

But when doing major upgrade projects, there is a tendency to opt for “rebuilds” instead of “Steady State Updates”. I contend this reflex and will explain why “Steady State Updates” are one or more orders of magnitude more efficient than “rebuild upgrades”.

Typical projects that leverage manual testing, will have a workflow like this.

1. Client & agency & developer talk and decide on features.
2. Client & agency provides a new design.
3. Developer builds new site, implements the design and migrates content.
4. Client tests site, provides feedback, several rounds of testing.
5. Website has the go-live.

Compare this with a typical ‘steady-state’ upgrade:

1. Scan website and formalize “steady state”.
2. Developer builds new site, implements the design and migrates content.
3. Client tests site, provides feedback, several rounds of testing.
4. Website has the go-live.

The amount of steps are similar. But let’s go over the steps again and ask yourself; how risky is this? (i.e. new unanticipated costs?) and how much time does this take (minutes/hours or days/weeks). Looking at it from this angle there are 2 steps I want to point out to you:

• Definition of done.
• Test & feedback loop.

Definition of Done

When the task at hand is to rebuild a website the definition of done is probably a mix of visual guidance (design), written specifications (word documents, meeting notes) and references to existing functionality (the drupal 7 website). It takes time to create this definition of done. It is also highly likely that people will misunderstand the definition of done. Imagine a scenario where a client asks ‘more’ and the agency/developer reads ‘less’. This will lead to friction and in many cases unanticipated costs (extra hours paid or unpaid) for all parties involved.

With a “steady state” upgrade a computer program is used to capture the definition of done. Examples are:

• Sitemap before and after the upgrade.
• Homepage before and after the upgrade.
• Order placement Ux before and after the upgrade.

Test & Feedback Loop

Then the testing. A computer program can execute above comparisons in minutes. At any point the developer can execute the comparisons and iterate on the instant feedback. The developer is not dependent on client feedback while doing the job. We clearly have a short and effective feedback loop. The opposite is true when doing rebuilds. The feedback is not instant, but it will probably take days or weeks. The client feedback must be interpreted by the developer. And only after the developer made the changes, the client can test again. There is always somebody waiting on someone else. And then the inevitable might happen… After several feedback rounds, time & money are up. There are discussions about the definition of done, the timeline and the budget. Aargh!

In the following diagram there are 2 lines. Each line represents the progress (impact) versus the time. The ‘roundabout’ symbol stands for a feedback (iteration) round. Both lines have 3 finished iterations. But when looking at the time axis it took way longer on the orange line.

With the above in mind it should be clear that whenever possible, we should try to fit our upgrade projects in a mall that allows for fast iterations. Fast iterations that are supported by automated testing tools. This is exactly how we work.

Enter Our Approach for Drupal 7 to Drupal 10 upgrades

Back in 2012, Drupal 7 was one of the world’s top-3 CMS’es. The web was in a different state then it is today. Most Drupal websites were young, really young. Business used Drupal 7 for their first online presence. In the decade after it these websites matured. Content was added, month after month, year after year. And also functionality was added and improved. Not always by a fulltime development team, but rather in once-in-a-year sprints. All together over the years big investments are done to those sites.

Doing a upgrade from Drupal 7 to Drupal 10 is, unfortunately, not simple. The shift from Drupal 7 to “modern” Drupal introduced multiple non backwards compatible changes.

The fact is that there are actually multiple upgrades… at once.

• Convert the Drupal 7 theme to the Drupal 10 theme. We have ongoing research to automate this step. Currently this is a labor intensive task.
• Select Drupal 10 modules that replace Drupal 7 modules (not a 1:1 mapping)
• Setup Drupal 10 hosting
• Convert configuration (i.e. the structure of the website)
• Copy content (i.e. pages, files, ‘entities’)
• Rewrite and update custom code
• And finally validate that this was all done correctly.

Our approach is to use the “Steady State Update” philosophy.

1. First we capture the definition of done with automation.
   1. We have ‘no-code & no-config’ automations that work extremely well for content-rich websites with many (thousands) pages.
   2. We have ‘low-code & low-config’ automations that work for workflows like “newsletter subscribe” or “place order”.
   3. If required, our consultants will help to create automated tests to capture complex business logic like “discount rules XZY”.
2. We create 2 environments; a ‘source’ Drupal 7 environment and a ‘target’ Drupal 10 environment. These environments can be inspected by the client.
3. We run our upgrade automations which can already finalize the upgrade to 80% - 99%.
4. We do a Gap analysis which clearly outlines what the automation was not able to finish. This outline will be converted to developer tasks.
5. At this point a Drupal developer will take over and finalize the upgrade.
6. When we know that the Drupal 10 Steady State Update™ is successful, we provide the proof to the client. After the client approves, we do the go-live. After the go-live the website is served with our regular maintenance service.