Security Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Security Magazine logo
  • NEWS
    • Security Newswire
    • Technologies & Solutions
  • MANAGEMENT
    • Leadership Management
    • Enterprise Services
    • Security Education & Training
    • Logical Security
    • Security & Business Resilience
    • Profiles in Excellence
  • PHYSICAL
    • Access Management
    • Fire & Life Safety
    • Identity Management
    • Physical Security
    • Video Surveillance
    • Case Studies (Physical)
  • CYBER
    • Cybersecurity News
    • More
  • BLOG
  • COLUMNS
    • Cyber Tactics
    • Leadership & Management
    • Security Talk
    • Career Intelligence
    • Leader to Leader
    • Cybersecurity Education & Training
  • EXCLUSIVES
    • Annual Guarding Report
    • Most Influential People in Security
    • The Security Benchmark Report
    • The Security Leadership Issue
    • Top Guard and Security Officer Companies
    • Top Cybersecurity Leaders
    • Women in Security
  • SECTORS
    • Arenas / Stadiums / Leagues / Entertainment
    • Banking/Finance/Insurance
    • Construction, Real Estate, Property Management
    • Education: K-12
    • Education: University
    • Government: Federal, State and Local
    • Hospitality & Casinos
    • Hospitals & Medical Centers
    • Infrastructure:Electric,Gas & Water
    • Ports: Sea, Land, & Air
    • Retail/Restaurants/Convenience
    • Transportation/Logistics/Supply Chain/Distribution/ Warehousing
  • EVENTS
    • Industry Events
    • Webinars
    • Solutions by Sector
    • Security 500 Conference
  • MEDIA
    • Videos
      • Cybersecurity & Geopolitical Discussion
      • Ask Me Anything (AMA) Series
    • Podcasts
    • Polls
    • Photo Galleries
  • MORE
    • Call for Entries
    • Classifieds & Job Listings
    • Continuing Education
    • Newsletter
    • Sponsor Insights
    • Store
    • White Papers
  • EMAG
    • eMagazine
    • This Month's Content
    • Advertise
  • SIGN UP!
CybersecurityManagementSecurity Enterprise ServicesSecurity Leadership and ManagementLogical SecuritySecurity & Business ResilienceSecurity Education & TrainingCybersecurity News

How a layered defense strategy protects organizations from security incidents occurring at the seams

By Bren Briggs
DevOps-software-freepik678.jpg
November 1, 2021

DevSecOps is (appropriately) emerging as the de facto pattern for managing and deploying applications and managing infrastructure. Security controls, deployments, and virtually all other aspects of enterprise systems should integrate and, from the beginning, automate. However, we should take care of the trust we place in automation and realize that blind spots in DevSecOps are always shifting. Organizations rely on code and decision-making created by humans who are prone to making human mistakes. When security incidents occur, they tend to happen at the seams and cracks of an organization, where the automation is incomplete, observability is not omniscient, and humans are still in the loop.


I recently was involved in an incident that is unfortunately all too familiar to readers: API key exposure. Some time ago, we received an email from AWS informing us that an IAM keypair was exposed in a public git repo. As all security teams would, we immediately took action to mitigate and remediate. However, over the course of the investigation, some of the details didn’t make sense, particularly given the controls we had in place. We had automatic and mandatory secrets scanning on all of our repositories. We have our SIEM in place and had already caught similar things in the past, but it was quiet this time. We also had deprecated and removed our IAM users in favor of SSO and short-lived API keys. These three tools, when combined, should have ensured this never occurred in the first place. So, what actually happened?


The investigation confirmed what we already knew: that the keys were real, legitimate, and belonged to a moderately high-privilege user. The origin of the exposure was what confused us, though. It came from a repository in one employee’s personal namespace on Github, not our company namespace. Additionally, no developers appeared to be aware of its existence, including the person who owned it. To add a further wrinkle, the commit originated 26 days before the exposure was noticed, but logs showed a startling lack of abuse of those keys. In fact, they clearly showed when the keys were discovered by an automated scanning tool, validated, and then subsequently reported to Amazon, who automatically suspended them, all in the space of a few minutes.


It quickly became apparent that the issue was at the seams and gaps of our organization’s detection and automation. While the automation around secret scanning and IAM key management was in place, their effectiveness and correct function was assumed. If not for another fortuitous accident that eventually caused the exposure, it might have been much longer before this was detected.


How the key ended up in a public repository continued to baffle us until, on a hunch, one responder asked the developer who owned the repository if they used VS Code with any plugins that required authentication to Github. As it turned out, the ultimate step leading to exposure was when VS Code attempted to helpfully create a repo on the user’s behalf when it believed that the user did not have write access to push a commit back to the original repository. Instead, it created a new repository with a default name in that user’s namespace. Shortly after that, the now exposed credentials were detected by an automatic scanning service that runs on all public Github repositories and the keys were suspended by Amazon. Case closed? Well, nearly.


We observed that the secret scanning we believed to be functional and ubiquitous inside our own organization was failing silently in some cases — at the seams. Remember how I said we had deprecated all IAM users and were removing them? We had just one AWS account remaining to fix, and the key just so happened to originate from that one. The SIEM, however, performed exactly as expected and turned out to be the critical piece in completing the investigation. It just happens that there was nothing to detect.


What can we take away from an incident which initially appeared to be a straightforward key exposure?

●    Complex systems fail in complex ways. Nobody would have predicted the compound failure required to expose a key in this fashion.

●    Despite having robust controls and tooling, extremely fallible humans are still in the loop at nearly every stage of the development process. Therefore, care and attention must be paid to the control and continuously testing it to provide assurance that it remains effective.

●    All of the prevention in the world is no substitute for rapid response. The inverse is also true.


Remember: Incidents tend to happen at the seams and cracks of your organization, where the automation is incomplete, observability is not omniscient, and humans are still in the loop. Our blind spots are constantly evolving, and we must update our mental models of how to approach security accordingly.

KEYWORDS: cyber security DevOps risk management security vulnerability

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

Bren briggs

Bren Briggs is the VP of DevSecOps at Hypergiant, where he oversees the infrastructure and security operations of the Hyperdrive Platform and all client engagements. In his spare time he enjoys brewing beer, spending time with his wife and three children, and perfecting his barbeque technique.

Recommended Content

JOIN TODAY
To unlock your recommendations.

Already have an account? Sign In

  • Security's Top Cybersecurity Leaders 2024

    Security's Top Cybersecurity Leaders 2024

    Security magazine's Top Cybersecurity Leaders 2024 award...
    Top Cybersecurity Leaders
    By: Security Staff
  • cyber brain

    The intersection of cybersecurity and artificial intelligence

    Artificial intelligence (AI) is a valuable cybersecurity...
    Logical Security
    By: Pam Nigro
  • artificial intelligence AI graphic

    Assessing the pros and cons of AI for cybersecurity

    Artificial intelligence (AI) has significant implications...
    New Security Technology
    By: Charles Denyer
Subscribe For Free!
  • Security eNewsletter & Other eNews Alerts
  • eMagazine Subscriptions
  • Manage My Preferences
  • Online Registration
  • Mobile App
  • Subscription Customer Service

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Security audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Security or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • Crisis Response Team
    Sponsored byEverbridge

    Automate or Fall Behind – Crisis Response at the Speed of Risk

  • Perimeter security
    Sponsored byAMAROK

    Why Property Security is the New Competitive Advantage

  • Duty of Care
    Sponsored byAMAROK

    Integrating Technology and Physical Security to Advance Duty of Care

Popular Stories

Pills spilled

More than 20,000 sensitive medical records exposed

Laptop in darkness

Verizon 2025 Data Breach Investigations Report shows rise in cyberattacks

Coding on screen

Research reveals mass scanning and exploitation campaigns

White post office truck

Department of Labor Sues USPS Over Texas Whistleblower Termination

Computer with binary code hovering nearby

Cyberattacks Targeting US Increased by 136%

2025 Security Benchmark banner

Events

May 22, 2025

Proactive Crisis Communication

Crisis doesn't wait for the right time - it strikes when least expected. Is your team prepared to communicate clearly and effectively when it matters most?

September 29, 2025

Global Security Exchange (GSX)

 

View All Submit An Event

Products

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

See More Products

Related Articles

  • half open laptop with colorful screen

    How threat hunting secures organizations: A proactive security strategy

    See More
  • cyber-threat-freepik1170x658v56.jpg

    Cyber warfare: How to empower your defense strategy with threat intelligence

    See More
  • cyber security freepik

    When security and resiliency converge: A CSO’s perspective on how security organizations can thrive

    See More
×

Sign-up to receive top management & result-driven techniques in the industry.

Join over 20,000+ industry leaders who receive our premium content.

SIGN UP TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • eNewsletter
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Reprints
    • Market Research
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2025. All Rights Reserved BNP Media.

Design, CMS, Hosting & Web Development :: ePublishing