We all know that the beautiful world of software is like a basketball balanced on a Jenga game. It's fine while it's fine but all it takes is a semi colon in the wrong place, a bad database failover or some unlucky road worker to cut a cable and the whole world comes crashing down. Fortunately we're all good people and when the pieces are on the floor we pull together and get it all balanced again. But then what?
It’s not good enough to just forget about it. Sweep it under the run and pretend it never happened. Production incidents are part of life. More than that, they are great learning opportunities. By examining what went wrong and identifying the contributing factors, we can not only learn how to prevent this happening again, but also pre-empt other issues within our system.
This talk looks at the strategies for Post Incident Review starting with the common place practices including the Fishbone Diagrams, Sticky Note Brainstorming and the Five Whys methods of root cause analysis. This talk will go beyond these practices to show how leveraging modern monitoring practices along side ChatOps we can the contributing factors, not just a single cause.
These contributing factors can be used to find actionable outcomes that not only help prevent the same thing happening again but feed back into the incident life cycle to make the team better prepared to Detect, Respond to, Remediate and Analyse the next incident.
Klee is a software engineer who is interested in the art and craft of software development over any particular technology.
He's one of the organisers of the Newcastle Coders Group working along side Peter Drew and John Roach to push the Newcastle development community forward.
Tickets start at $1195, for two in-depth days with the world's leading design experts.
Australia's conference for JS developers and front-end engineers.
For over a decade, we've worked hard to create inclusive, fun, inspring and safe events for the Web Industry.
As part of our commitment to these values, we've adopted a code of conduct for all involved: ourselves, our speakers, our partners and our audience.
If you have any concern or feedback, please don't hesitate to contact us.