10 Cases of Certificate Outages Involving Human Error

As we begin this new year and review our data protection strategy for the months ahead, it’s crucial not to overlook one of the leading causes of certificate outages: human errors. Enterprises are exposed to increasing security and compliance challenges with no margin for error. Every expired certificate opens a backdoor for rogue hackers to steal critical data from the organization and leave behind hefty bills of financial loss. Even one certificate outage can cost enterprises an average of 15 million dollars to recover. These expired certificates that fall through the cracks are causing certificate breaches and massive outages to some of the most prominent companies throughout the world. Around 81% of companies have experienced a certificate-related outage in the past two years, and these cases just corroborate the claims.

CISCO

The latest on the list is Cisco, who recently warned its clients on Twitter about an expired hardware certificate in its SD-WAN devices, eventually posing risks to the attached SD-WAN environments. In their released statement, they confirmed that their team is actively working to resolve the device failure that affected several of their routers, including VEdge 100, 1000, and 2000. These routers are responsible for security and multi-cloud connectivity to the enterprise. They have advised their customers to stay away from the go-to solution of restarting their devices as it leads to a complete loss of service. While the team at Cisco is actively working to resolve the issue, the incident is pushing enterprises to take a second look at the status of their digital certificates.

MICROSOFT

The WinGet package failure from expired SSL certificates at Microsoft raised many concerns about the potential risks of certificate outages. The open-source Windows package manager users started reporting issues in installing or updating apps via WinGet as they received “InternetOpenUrl() failed” errors. While the tech giant was quick to release a workaround to provide a solution to their users, the users continued to share screenshots of the error on GitHub, questioning the credibility of the brand for missing out on the renewal of their SSL/TLS Certificate.

SPOTIFY

A certificate expiration took down the prominent streaming service Spotify for over an hour leading to a massive tweetstorm by their music-deprived users. Despite the company not making an official statement releasing the cause of the outage, experts were quick to point out expired certificates to be the reason behind it. The lapsed certificates caused another disruption two years later that lasted for 9 hours affecting the platform’s biggest producers. Listeners were unable to access and download podcasts from Megaphone-hosted publishers due to an expired SSL certificate.

ERICSSON

The certificate expiration at Middlebury Ericsson proved how one certificate outage could cause massive disruptions in over 11 countries and affect millions of people, including all the company stakeholders. The expired certificate led to over 32 million people in the UK losing their 4G and SMS signals. After a public apology, the company immediately decommissioned the software that was responsible for managing its certificates. This incident is still one of the biggest certificate outages that the world has seen and highlights the need for a robust certificate management solution.

LINKEDIN

LinkedIn, our go-to social platform for business networking, has faced certificate outages twice in 2 years due to SSL expiration. The first incident impacted millions as they were unable to log into their accounts. The second breach only affected the desktop users after several users started receiving the SSL connection error message caused by LinkedIn’s link shortener, lnkd.in. Their team was quick to install a new certificate, but these two incidents highlight the issue that many large enterprises face of certificate visibility, creating the need for a centralized location for all the digital certificates in a certificate management solution.

GOOGLE VOICE

Google Voice experienced a global outage from February 15th to 16th, 2021, lasting over four hours due to an expired TLS certificate. Google’s Root Cause Analysis revealed that the issue stemmed from a failure to update certificate configurations, unintentionally causing the active certificate to expire. During the outage, users were unable to initiate or receive VoIP calls. The incident highlights the recurring problem of organizations, even giants like Google, grappling with certificate expirations, often due to faulty alerting systems or manual renewal processes. Expired certificates have been implicated in various high-profile outages and data breaches.

MICROSOFT TEAMS

On February 3, 2019, Microsoft Teams experienced a three-hour outage due to an expired authentication certificate. The incident left its 20 million daily users unable to access essential collaboration tools. Microsoft acknowledged the problem on Twitter, assuring a swift certificate renewal. However, users expressed frustration, some even contemplating a switch to Slack. The incident highlighted concerns about Microsoft’s oversight of certificate management, especially given the critical role Teams plays in modern workplaces. Critics questioned why a company like Microsoft lacked automated solutions to prevent such certificate-related issues.

EQUIFAX

The Equifax data breach, revealing 9,000 queries on 48 databases and 265 instances of unauthorized data access, was attributed to an unnoticed certificate expiration on Equifax’s monitoring device. Due to a lapsed security certificate for 19 months, the inactive device allowed attackers to transfer personally identifiable information unnoticed. Similar to the recent Ericsson outage, emphasizing the necessity of automated enterprise-wide certificate management, the incidents underscore the risks posed by expired certificates. During the breach, Equifax had 324 expired SSL certificates, including 79 critical domain monitoring devices. The importance of proactive identification and renewal of certificates is crucial to prevent potentially catastrophic failures.

AMAZON WEB SERVICE

On December 7, 2021, a significant outage in Amazon Web Services (AWS) disrupted various online services, including Amazon’s delivery operations and third-party sellers. Colleges had to delay exams due to reliance on AWS-powered software. The outage, originating in the US-East-1 region, affected fulfilment centres and delivery operations, leading to stranded packages and delayed orders during the peak holiday season. The incident exposed the vulnerability of relying on a few major cloud service providers. Notably, Whole Foods, Amazon Flex drivers, and third-party merchants suffered disruptions, causing financial losses for businesses and inconvenience for customers.

APPLE

In April 2023, a widespread outage affected Apple services, including the App Store, Apple Music, and Apple News. Users encountered errors when attempting to download or update apps, and SSL certificate issues were identified as the cause. Apple has since resolved the problem, but in the preceding two weeks, various Apple services, such as the Weather app and the Apple Developer website, experienced reliability issues raising concerns.

What Steps Can You Take to Safeguard Your Organization from Certificate Outages for 2024? 

• Moving away from the traditional use of patchwork of spreadsheets and internal PKI interface to a secure certificate management solution that provides enhanced visibility of all your certificates and automates the complete lifecycle of certificates.
• Conduct regular assessments to identify vulnerability issues and monitor your certificates for expiration.
• Checking certificate transparency logs to track rogue certificates and mitigate security risks.
• Implementing automated alerting and reporting systems to get prior notifications of any certificate reaching its end of life.
• Securing ownership to designate the responsibility of tracking and monitoring certificates to ensure accountability and avoid confusion.

How can Encryption Consulting’s CertSecure Manager help?

Encryption Consulting’s Certificate Management Solution plays a crucial role in addressing and mitigating the risks associated with certificate expiry. CertSecure Manager offers a range of comprehensive features designed to proactively manage certificates throughout their lifecycle. Here’s how it can be beneficial:

• Certificate Monitoring and Alerting

  Consistently monitoring the expiration dates of digital certificates across an organization’s infrastructure. It promptly sends alerts and notifications to administrators, providing ample time for necessary renewal actions.

• Automated Renewal

  Renewing certificates on time can be challenging, especially with a large number of certificates. CertSecure Manager automates the renewal process, reducing the risk of oversight and ensuring the timely renewal of critical certificates.

• Centralized Management

  Managing certificates across various systems and applications can be complex. CertSecure Manager offers a centralized platform, allowing administrators to view, track, and manage all certificates from a unified interface, enhancing efficiency and control.

• Policy Enforcement

  Organizations can define and enforce certificate policies with CertSecure Manager. This ensures that certificates are created with proper configurations and adhere to security best practices, minimizing the chances of misconfigurations that could lead to vulnerabilities.

• Risk Assessment

  CertSecure Manager conducts regular assessments of the certificate landscape, identifying potential vulnerabilities arising from expired or misconfigured certificates. This enables proactive remediation before malicious actors can exploit them.

• Reporting and Auditing

  Detailed reports are generated, offering insights into certificate statuses, renewals, and potential risks. These reports are valuable for compliance requirements, audits, and demonstrating a robust security posture.

• Integration with Existing Systems

  CertSecure Manager seamlessly integrates with various certificate authorities, ensuring smooth integration with existing certificate issuance processes.

• Reduced Downtime

  By preventing disruptions caused by certificate expiration, CertSecure Manager ensures uninterrupted services, reducing downtime and enhancing customer trust and satisfaction.

• Enhanced Security Posture

  CertSecure Manager contributes to an organization’s overall security posture by ensuring certificates are always up-to-date, thus preventing security gaps that attackers could exploit.

Conclusion

The digital certificates that connect 64 billion users across the continental borders hold tremendous pressure to run smoothly. These incidents of certificate outages cause massive disruptions and hamper the integrity and trust of the users in the company.

By automating the complete certificate management lifecycle, you don’t just avoid certificate outages but also provide an additional security layer and meet all compliance requirements to enhance the organization’s security posture. Our world’s first Microsoft PKI native certificate management solution automates the complete certificate lifecycle management process that provides automatic certificate renewal to ensure zero certificate outages