Postmortems

an analysis or discussion of an event after it is over.

Merriam-Webster

The backstory

Earlier this year, on one of our half-yearly IRL’s, we had a discussion about whether or not we should proceed with a massive upgrade process that had been going on for the past year or so. The team that was working on it, myself included, had been facing numerous problems on a daily basis where it felt like if we plugged one leak it started leaking from a different hole so we were starting to get demotivated and it felt like no solution was in sight.

The discussion went on for a few hours with a lot of strong opinions on both sides, for and against proceeding, but in the end, we decided that it was time to stop beating on the dead horse and start fresh with a new one. Nejc didn’t feel like the right decision had been made, however, and decided to reinforce the team, and even join himself, to give it one last attempt with everything we had.

After just two-three weeks with the reinforced team working on it we finally saw a light at the end of the tunnel, and most of our issues were either already solved or had been identified. Now, a full quarter later, we’re certain we made the right choice to continue with and finish what we had started on.

The postmortem

Smallbones, CC BY-SA 3.0, via Wikimedia Commons

Realizing that we had almost made a big mistake if we had followed through with the decision to start fresh, we scheduled a postmortem to discuss and identify what had led us to lose hope and want to abandon ship.

Before the meeting, we all watched Who Destroyed Three Mile Island? to get into the right mindset of not setting out to assign blame but to understand what led to the wrong decision being made, learn from it, and strengthen our routines and systems to prevent it from happening again.

Lessons learned

The postmortem made us realize that we should have done more to follow the list of guidelines and best practices below. It’s important and helpful to keep this in mind when starting with a new project.

  • Systems Design is important
    When doing big architectural changes it’s recommended to have frequent catchups with seniors for reviews, bouncing ideas off of, or for general rubber ducking.
  • Be conscious of your feelings
    When working on a project we can sometimes get emotionally involved. Be it negative feelings like anger or frustration because of bugs, problems, or the level of difficulty, or emotional attachment in a defensive way if someone questions your design or approach. Humans are not robots. Feelings affect our decisions whether we like it or not. It’s important to be aware of them to know if they are doing the talking instead of your logical objective self.
  • Change as few things at once as possible
    If you’re going to create a proof-of-concept that aims to replace an existing system make sure to compare them fairly. For example, if you swap out your Android phone for an iPhone and at the same time you start using Apple Music instead of Spotify. It wouldn’t be a fair comparison if you blame the iPhone for not having the music you like.
  • You aren’t gonna need it (YAGNI)
    Don’t let any potential future plans influence your current plans. Focus on the task at hand and solve it as quickly and cleanly as possible. Any future additions can be dealt with when the time comes.
  • Avoid keeping personal notes
    There’s nothing wrong with keeping personal notes but if you find yourself searching for them often they should go into the codebase.
  • Don’t rely on discipline
    Don’t expect yourself or your coworkers to stay disciplined enough to always read the documentation thoroughly before and during every task. An excellent way to ensure this is to improve the code/tooling so we won’t have to use the documentation at all by e.g. printing an error message explaining what you need to do instead if you do something wrong. Using good comments in the code is also important.
  • Best practices sooner rather than later
    Time is almost always of the essence, so it’s important to move fast but at the same time stick to best practices.
    For example, when prototyping it’s fine to leave out tests to keep up the momentum but they should be added as soon as the code changes less frequently. If you are more than a month into a project it’s time to start adding them. The same goes for other best practices like linters, documentation, version control, and more.
    Here’s a nice list of best practices from opensource.com.
  • The world is malleable
    Where there’s a will, there’s a way, and in software development it’s definitely true. Imagining that you’re working with clay rather than stone can help you free your mind of any self-enforced limitations.
  • Other good practices that we should always try to follow
    • DRY (Don’t repeat yourself).
    • VCS (Version control system).
    • TDD (Test-driven development).

Summary

We’ll continue to make mistakes, make no mistake about it, but taking the time to analyze why it happened without having to worry about pointing fingers can be extremely valuable both as humans and as a company.

Bjørnar Myrheim

Bjørnar is an up-and-coming full-stack engineer, tackling Python, Nix, DevOps and everything in between.

See other posts »