Continuous Development of our Mobility Platform
April 22, 2020 | Technical Blog
#3 — Backward compatible by default
This is probably one of the most important rules, and also one of the costliest. When changes are applied to the code, developers need to identify any breaking change in terms of function signature (REST API e.g.) but also behavior.
At Bestmile, as development is happening fast, endpoints are all versioned independently. When a breaking change is necessary, the endpoint’s versions are usually “bumped”, and the compatibility with lower versions is guaranteed as much as possible.
As a rule of thumb:
There are some corner cases, but all developers at Bestmile are aware of and follow backward compatibility principles
#4 — Dissociate deployments (to production) from deliveries (to clients)
Continuous Deployment of code means that the code is reaching production before the full readiness of an end-to-end feature. It’s especially true in bigger, more complex systems.
Some features might require multiple sprints to be fully ready for a customer to use. In a more traditional deployment format, features used to be ready-to-use when the code was deployed to production.
With a Continuous Deployment approach, some components might be ready in sprints, if not months in advance, while the full feature is not yet usable.
#5 — Feature flags
Feature flags are a mechanism that allows the user (or an admin) to enable/disable the usage of the feature for customers.
Linked to rule #4 above, this allows us to orchestrate the development of a feature at different speeds between teams. That way, even if the full feature is not available yet, we can already start using part of it, or we can enable it earlier to some preselected and trusted customer. This allows faster feedback loops. The drawback of this is that you need to account for these flags in advance, and document them.
Customer facing teams need to understand the orchestration of delivery vs. deployment to know when to communicate what to customers. On the other end, we found that product management teams have a big role to play in that orchestration.
Some leading companies in the software field call that the BusProdDevOps (Business+Product+Development+Operations) mentality, related to what was known in software as DevOps (Development+Operations)
This highlights the need for business and product to be aligned with how the software is built.
The details of what works for Bestmile at the moment are going to be shared in a separate article. Below is a summary (and spoiler!).
Bestmile leverages Kubernetes and Docker container technologies to orchestrate 0-downtime, Continuous Deployments using out-of-the-box rolling updates strategies.
But installing Kubernetes is only the first step. We had to prepare all our services to support this kind of deployments, and shape the tools that would fit our team processes.
Stay tuned for more details of how we use and orchestrate those technologies in the follow-up part of this article.
The last point proves to be wrong, as smaller incremental deployments tend to break less, are more stable, are easier to roll back, bring feedback sooner, and are also easier to fix (in case a rollback is not an option).
Stability keeps improving, as engineers get better at deployment and build habits and tools, spread the mindset, the skills and the processes around this deployment practice.
Issues still happen, but they are less critical, less frequent (or at least not more), and last for a shorter amount of time.
As of the publication of this article our Platform and Dashboard now totals