Security Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Security Magazine logo
  • NEWS
    • Security Newswire
    • Technologies & Solutions
  • MANAGEMENT
    • Leadership Management
    • Enterprise Services
    • Security Education & Training
    • Logical Security
    • Security & Business Resilience
    • Profiles in Excellence
  • PHYSICAL
    • Access Management
    • Fire & Life Safety
    • Identity Management
    • Physical Security
    • Video Surveillance
    • Case Studies (Physical)
  • CYBER
    • Cybersecurity News
    • More
  • BLOG
  • COLUMNS
    • Cyber Tactics
    • Leadership & Management
    • Security Talk
    • Career Intelligence
    • Leader to Leader
    • Cybersecurity Education & Training
  • EXCLUSIVES
    • Annual Guarding Report
    • Most Influential People in Security
    • The Security Benchmark Report
    • The Security Leadership Issue
    • Top Guard and Security Officer Companies
    • Top Cybersecurity Leaders
    • Women in Security
  • SECTORS
    • Arenas / Stadiums / Leagues / Entertainment
    • Banking/Finance/Insurance
    • Construction, Real Estate, Property Management
    • Education: K-12
    • Education: University
    • Government: Federal, State and Local
    • Hospitality & Casinos
    • Hospitals & Medical Centers
    • Infrastructure:Electric,Gas & Water
    • Ports: Sea, Land, & Air
    • Retail/Restaurants/Convenience
    • Transportation/Logistics/Supply Chain/Distribution/ Warehousing
  • EVENTS
    • Industry Events
    • Webinars
    • Solutions by Sector
    • Security 500 Conference
  • MEDIA
    • Videos
      • Cybersecurity & Geopolitical Discussion
      • Ask Me Anything (AMA) Series
    • Podcasts
    • Polls
    • Photo Galleries
  • MORE
    • Call for Entries
    • Classifieds & Job Listings
    • Continuing Education
    • Newsletter
    • Sponsor Insights
    • Store
    • White Papers
  • EMAG
    • eMagazine
    • This Month's Content
    • Advertise
  • SIGN UP!
CybersecuritySecurity Leadership and ManagementLogical Security

The rising threat of ChatGPT

By Sam Crowther
ChatGPT on computer

Image via Unsplash

July 25, 2023

OpenAI’s ChatGPT has started to become an indispensable tool for many people. According to a recent study, 46% of respondents are already using ChatGPT at work, with 80% describing it as a valuable work tool that will continue to shape the future of work.

Unfortunately, cybercriminals have also recognized the immense potential of ChatGPT as a productivity enhancer and have wasted no time in exploiting its security vulnerabilities. 

One of the most distressing developments is the emergence of a thriving marketplace on the dark web, where compromised ChatGPT accounts are being bought and sold. A cybersecurity research firm recently reported that more than 100,000 ChatGPT accounts have been compromised and are being traded on covert black markets. In addition to gaining unauthorized access to premium features, hackers are now able to engage and conduct various criminal acts such as disinformation campaigns, data breaches and phishing attacks on a scale not possible before.

The stolen ChatGPT Plus accounts are being sold online to individuals and businesses in regions where the platform is not permitted. These paid accounts offer higher message limits and eliminate waiting times, making them desirable for those who need unrestricted access to ChatGPT Plus. 

Cybercriminals use various techniques to exploit vulnerabilities in OpenAI systems. For example, Kasada's research team discovered that hackers leverage GitHub repositories to gain unauthorized access to ChatGPT's API. This is accomplished by circumventing security controls and integrating the language model into unauthorized applications. In addition, automated credential stuffing and brute force techniques can be used to identify and exploit weak account credentials quickly.

So exactly how does a cybercriminal hack a ChatGPT account and sell it on the black market? Let’s look at the lifecycle of a hacked ChatGPT account.

  1. Launch the account takeover attack: The first step is for cybercriminals to gain access to the accounts through techniques such as credential stuffing or jailbreaking to gain unauthorized access to ChatGPT Plus and GPT-4 APIs. By exploiting security weaknesses, they break through the barriers protecting these APIs.
  2. Evade bot detection: Next attackers exploit bot detection bypasses that are designed to protect against unauthorized access to the system. They leverage various open source and inexpensive tools, including using CAPTCHA-solving services like 2Captcha to bypass ChatGPT Plus account protections.
  3. Use compromised ChatGPT account for malicious purposes: Once inside premium-level accounts, fraudsters take advantage of the absence of guardrails for nefarious purposes. They generate fraudulent documents, forge accounts, and orchestrate scams. Sometimes these compromised accounts are directly sold to other individuals.
  4. Sell jailbroken ChatGPT accounts on the dark web or other forums: The final step of this lifecycle involves the trading of jailbroken ChatGPT Plus accounts on the dark web and other online forums. These platforms cater to users seeking unrestricted access to the premium features of ChatGPT. Kasada's threat intelligence has observed these compromised accounts being sold for as low as $5 per account, representing a substantial discount ranging from 50 to 75%.

The wider implications

The implications of compromised generative AI accounts extend beyond the immediate victims. Once hackers gain unauthorized access, they can use the query histories associated with these accounts, potentially exposing sensitive personal or corporate information. In addition, cybercriminals may change the account credentials to take over the account and potentially commit further fraud. 

The risks associated with compromised accounts are particularly concerning for companies that rely on CAPTCHA technology as part of their cybersecurity measures. Despite CAPTCHAs being widely used, AI-assisted CAPTCHA-solving services have emerged, enabling sophisticated bots to bypass these protections more quickly and easily.

Addressing the risks and rewards of ChatGPT

To effectively tackle the threats associated with ChatGPT while also being able to reap its benefits, it’s important that security leaders take collective responsibility, starting with OpenAI.

  • OpenAI has already taken significant steps by implementing robust security measures and adopting responsible data handling practices. OpenAI’s GPT and Safety Best Practices guides provide valuable information on protective measures, including the use of tools like the Moderation API, which can identify and warn against or block certain types of unsafe content.
  • Government bodies and industry leaders are increasingly recognizing their role in regulating and safeguarding against potential abuses of AI. Collaborative conversations and initiatives are already underway to establish frameworks that protect users and businesses from cybercriminals seeking to exploit AI technologies.
  • Enterprises, through their security teams, also bear responsibility for mitigating risks associated with ChatGPT. Educating employees about how the models work and the potential consequence of disclosing sensitive information is crucial. By providing training on data privacy and security practices, employees can better understand the implications of their actions and take steps to prevent inadvertent data breaches. Additionally, security teams can implement restrictions on the types of prompts that can be used with ChatGPT and bolster the overall security of ChatGPT accounts and APIs, further fortifying protection against potential vulnerabilities.

However, striking the right balance between preventing malicious activities and preserving the platform's legitimate use cases remains a significant challenge. As the adoption of ChatGPT continues to accelerate, it is important for users and organizations to maintain a proactive stance against potential security threats and various forms it can take. Through training, education and proactive measures, we can harness the immense potential of ChatGPT and other models while ensuring the highest standards of security and privacy for users and businesses alike.

KEYWORDS: artificial intelligence (AI) Artificial Intelligence (AI) Security bots data breaches vulnerability

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

 Sam Crowther, CEO and founder, Kasada

Recommended Content

JOIN TODAY
To unlock your recommendations.

Already have an account? Sign In

  • Security's Top Cybersecurity Leaders 2024

    Security's Top Cybersecurity Leaders 2024

    Security magazine's Top Cybersecurity Leaders 2024 award...
    Top Cybersecurity Leaders
    By: Security Staff
  • cyber brain

    The intersection of cybersecurity and artificial intelligence

    Artificial intelligence (AI) is a valuable cybersecurity...
    Cybersecurity
    By: Pam Nigro
  • artificial intelligence AI graphic

    Assessing the pros and cons of AI for cybersecurity

    Artificial intelligence (AI) has significant implications...
    Cybersecurity
    By: Charles Denyer
Subscribe For Free!
  • Security eNewsletter & Other eNews Alerts
  • eMagazine Subscriptions
  • Manage My Preferences
  • Online Registration
  • Mobile App
  • Subscription Customer Service

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Security audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Security or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • Crisis Response Team
    Sponsored byEverbridge

    Automate or Fall Behind – Crisis Response at the Speed of Risk

  • Perimeter security
    Sponsored byAMAROK

    Why Property Security is the New Competitive Advantage

  • Duty of Care
    Sponsored byAMAROK

    Integrating Technology and Physical Security to Advance Duty of Care

Popular Stories

Pills spilled

More than 20,000 sensitive medical records exposed

Laptop in darkness

Verizon 2025 Data Breach Investigations Report shows rise in cyberattacks

White post office truck

Department of Labor Sues USPS Over Texas Whistleblower Termination

Computer with binary code hovering nearby

Cyberattacks Targeting US Increased by 136%

Internal computer parts

Critical Software Vulnerabilities Rose 37% in 2024

2025 Security Benchmark banner

Events

September 29, 2025

Global Security Exchange (GSX)

 

November 17, 2025

SECURITY 500 Conference

This event is designed to provide security executives, government officials and leaders of industry with vital information on how to elevate their programs while allowing attendees to share their strategies and solutions with other security industry executives.

View All Submit An Event

Products

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

See More Products

Related Articles

  • AI-powered phishing

    Combating the rising threat of AI-powered phishing attacks

    See More
  • tap to pay device

    The rising threat of cybersecurity for retailers

    See More
  • SEC_Web_5Minutes-Breen.jpg

    The impact of ChatGPT and AI on the cybersecurity field

    See More

Events

View AllSubmit An Event
  • November 14, 2024

    Best Practices for Integrating AI Responsibly

    ON DEMAND: Discover how artificial intelligence is reshaping the business landscape. AI holds immense potential to revolutionize industries, but with it comes complex questions about its risks and rewards.
View AllSubmit An Event
×

Sign-up to receive top management & result-driven techniques in the industry.

Join over 20,000+ industry leaders who receive our premium content.

SIGN UP TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • eNewsletter
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Reprints
    • Market Research
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2025. All Rights Reserved BNP Media.

Design, CMS, Hosting & Web Development :: ePublishing