One of my favourite apps had a security incident this week. Buffer is a social media management tool that allows you to schedule posts into a “buffer” that posts on a predetermined schedule so you aren’t bombarding people with all your content at once. Its a great app and I’ve been using it for over a year.
One of the things that differentiates the buffer experience from competing products is that their management team are AMAZING communicators. They haven’t lost touch with who they are as things have grown and this weeks experience is no exception. Buffers handling of what could be a fatal experience for a startup is an awesome example of why they are going to be very very successful in the future.
So lets look at what happened (Full details here.):
- Oct 26 12pm – Buffer announces that it has experienced a security incident that has caused spam to be posted to some customers social media accounts.
- 1pm – Buffer lets customers know that spam is no longer being sent.
- 3pm – Customers are informed that there was an issue with how Buffer has stored Twitter tokens and it has been locked down.
- 5.30pm – Buffer and Facebook are working together to help clean up Facebook spam.
- 8pm – Services fully restored.
- Oct 27 9am – Notice that Buffers team have monitored all posts overnight and there have been no issues.
- Oct 28 3pm – Notice that the origin of the breach has been identified.
- Oct 29 2pm – Disclosure of the origin as being an external database provider, Buffer took responsibility for the issue affecting their customers and apologised to customers.
Some awesome points come out of this:
- Communication & honesty with all stakeholders makes or breaks a security incident.
- Cloud has completely changed the attack surfaces available to attackers.
- Buffer is awesome.
- Buffers team are amazing communicators.
There is nothing like a crisis squeezing on a team to see what its culture is like. Buffers response to this incident should become the benchmark for handling of security incidents. Their honesty and proactive communication turned a disaster into an easy to deal with experience.
How can your team become security crisis pimps like Joel and the team at Buffer?
- Plan for when (not if) you experience a security incident & practice it regularly!
- Create a stakeholder communications strategy and have someone dedicated to keeping everyone in the loop.
- Build applications with an assumption of poor security, especially when using external cloud providers like AWS or MongoHQ.
- Develop great relationships with your providers (Like Buffer has with Facebook).