We are excited to announce the launch of Maestro, an intelligent layer atop Prodvana’s Managed Delivery.

Designed to sequence complex releases, Maestro is like a conductor who knows every note of the symphony and perfectly times each section's entrance. 

Note: Release in this context refers to all deployment steps across all environments, including intermediate steps like blue-green deployments, canary, etc.

The Problem: Stateful System Releases

Stateful systems, known for their intricate release processes, face unique challenges:

1. High cost of mistakes (e.g., data corruption).

2. Complex rollbacks (e.g., are schema changes both forward- and backward-compatible?).

3. A large number of environments (stateful systems tend to be replicated).

4. Delayed issue detection (e.g., write-path bugs in a read-heavy system).

Consider Dropbox’s Magic Pocket, which powers the storage giant and serves exabytes of data. Magic Pocket has a 4-weeks-long release cycle. Owners of stateful systems like this often face a dilemma:

(1) They can choose to schedule their releases sequentially, with one release fully finishing before another can begin. This approach will work out of the box on most deployment systems but has the drawback of infrequent releases. In the Magic Pocket example, the sequential approach would mean a release at most once every 4 weeks.

(2) They can run multiple releases concurrently in a release train. Take a very simple stateful system with a single staging environment and two production environments, with the promotion between environments taking a week. Instead of releasing sequentially, which would mean a release every two weeks, one can start a new release on staging after the previous release has been promoted to the first production environment, allowing for weekly releases. 

The impact of release trains grows proportional to the number of environments and pending releases.

While release trains lead to more frequent releases and likely higher development velocity, most deployment systems do not have the release train concept built in. Implementing release trains on existing deployment systems results in a distributed system problem simply to do releases, or they end up doing so manually and relying on a human to do the coordination.

That is, until now.

Maestro and the Release Train Strategy

Maestro is an intelligent conductor of releases. It operates on a queue of releases and computes the next set of desired states given a release strategy. 

A significant challenge in a deployment system is having a single system and interface handle simple and sophisticated use cases. Maestro solves this by allowing any service in Prodvana to have its release strategy updated without any other changes as the service matures. Today, Maestro is launching with two release strategies out of the box:

The Immediate Release Strategy

The immediate release strategy promotes the latest release immediately. With the immediate strategy, Maestro makes the latest release the desired state and skips any pending releases in between.

The immediate release strategy replaces the base case and is equivalent to how Prodvana operated without Maestro. The immediate release strategy enables us to ship Maestro to all our users without increasing complexity. At the same time, it sets up our users to be able to upgrade to a more sophisticated strategy transparently.

The Release Train Strategy

The release train strategy promotes a release once the previous one has converged for each Release Channel. With the release train strategy, Maestro computes the effective desired state from the set of pending releases.

The release train strategy solves the long-running releases of stateful systems outlined above. It is also an excellent solution for scenarios where code changes are faster than deployments can occur. For example, we have talked with users who want to break up commit ranges into multiple batches to be released and validated separately,

The release train strategy significantly reduces the end-to-end release time and, more importantly, makes the release experience fire and forget. Just queue up the releases and let Maestro take care of it.

The Maestro UI

Additionally, the UI has been updated  to support release strategies. The UI now shows the pending release queue and the desired states, giving a clear and elegant waterfall representation of releases.

The release creation UI has also been updated to show how the new release will impact the queue. There is also a selector for release strategy, allowing just that release to switch to the immediate release strategy if needed. This can be useful for error recovery.

Enabling Maestro

Maestro will be rolling out to all Prodvana customers in the coming weeks with the immediate release strategy being the default. 

To switch release strategies, make a two-line config change to your Prodvana Service configuration.

service:

  name: my-service

  ...

  maestro:

    strategy

Conclusion

With Maestro, Prodvana is now able to handle all kinds of architectures ranging from day 0 setups to complex, long-running releases. Maestro with the release train strategy is available today, and we are excited to bring more powerful release strategies to Maestro in the future. Sign up for Prodvana today to try Maestro.