`On 29 June 2017, from 05:00 to 07:43 CDT, Cloud Load Balancers encountered a licensing issue with a 3rd party vendor. Engineers worked with the vendor to resolve the licensing issue. During the time of impact, customers would have been unable to access their Cloud Services, Cloud products via their control panel, as well as unable to access ticketing.`
I think it's a case of developers saying "what're the chances of that happening?". I've been using Rackspace load balancers for > 5 years and never had an issue like this before.
It is seriously frustrating though. This is the busiest weekend of the year for my business. 2+ hours of downtime is not good.
Somebody isn't keeping an eye on the calendar... licenses need to be tracked and renewed, just like domain registrations and SSL certificates, otherwise things break.
Unfortunately, I've seen this before, more than once. I've come to believe that a high-availability product should keep the places where human error can creep in to an absolute minimum. That means no artificial breaking points.
Ask vendors whether their products are susceptible to this failure mode and factor the answer into your purchasing decisions. Also recognize that if they want to they can give you a perpetual license file even if business terms mandate renewal.
I had a very expensive Cisco IronPort start blackholing email when a license expired. From what I'm seeing this license enforcement approach is not unheard of in enterprise products.
I think any CI/CD system should be set up to also validate that any license used in the system has at least a month to go, and start complaining if not...
12 comments
[ 3.1 ms ] story [ 27.3 ms ] thread`On 29 June 2017, from 05:00 to 07:43 CDT, Cloud Load Balancers encountered a licensing issue with a 3rd party vendor. Engineers worked with the vendor to resolve the licensing issue. During the time of impact, customers would have been unable to access their Cloud Services, Cloud products via their control panel, as well as unable to access ticketing.`
It is seriously frustrating though. This is the busiest weekend of the year for my business. 2+ hours of downtime is not good.
Flag vendor? Fireworks manufacturer?
Unfortunately, I've seen this before, more than once. I've come to believe that a high-availability product should keep the places where human error can creep in to an absolute minimum. That means no artificial breaking points.
Ask vendors whether their products are susceptible to this failure mode and factor the answer into your purchasing decisions. Also recognize that if they want to they can give you a perpetual license file even if business terms mandate renewal.
Who provides Cloud Load Balancers?