Menlo Security researchers have discovered an increase in traffic to generative AI (GenAI) websites. The research found these visits rose by 50%, from 7 billion in February 2024 to 10.53 billion in January 2025.

The research highlighted the risks of shadow AI, revealing that a majority of employees (68%) utilize free-tier AI tools (such as ChatGPT) through personal accounts. Out of these employees, 57% input sensitive data. 

Furthermore, the researchers found a corresponding increase in cyberattacks. Zero-hour phishing attacks saw a 130% year-over-year increase, with approximately 600 false GenAI sites identified.

Security Leaders Weigh In

Satyam Sinha, CEO and Co-Founder at Acuvity:

There has been a lot more information and awareness about the risks and threats with regards to AI provided over the last year. In addition, there are numerous regulations brought in by various governments. In our discussions with customers, it is evident that they are overwhelmed on how to prioritize and tackle the issues — there’s a lot that needs to be done. At the face of it, personnel seems to be a key inhibitor, however, this pain will only grow. GenAI has helped in multiple industries from customer support to writing code. Workflows that could not be automated are being handled by AI agents. We have to consider the use of Gen-AI native security products and techniques which will help achieve a multiplier effect on the personnel.

The field of AI has seen enormous leaps over the past two years, but it is evolving with new developments every month. The gap in confidence and understanding of AI creates a massive opportunity for AI native security products to be created which can ease this gap. In addition, enterprises must consider approaches to bridge this gap with specialized learning programs or certifications to aid their cybersecurity teams. 

GenAI has helped in multiple industries from customer support to writing code. Workflows that could not be automated are being handled by AI agents. We have to consider the use of GenAI native security products and techniques which will help achieve a multiplier effect on the personnel. This is the only way that we can solve this problem.

Nicole Carignan, Senior Vice President, Security & AI Strategy, and Field CISO at Darktrace:

Understanding the evolving threat landscape and the techniques adversaries are using to manipulate AI is key and critical for defenders to be able to test these use cases against their own models to effectively secure their AI systems and to defend against AI attacks. As AI systems become embedded into the tools and processes organizations depend on every day, cybersecurity plays a crucial role and is foundational to AI safety. Organizations must be focused on applying cybersecurity best practices to protect models and invest in safeguards to keep AI systems protected at all stages of the AI lifecycle, to avoid unintended behaviors or potential hijacking of the algorithms. 

Adoption of AI, specifically in response scenarios, requires transparency, explainability and control. To facilitate data-driven recommendations to autonomous action, trust needs to be built with AI’s decision-making capabilities. The recent 2025 State of AI Cybersecurity Report found that 78% of CISOs admit that AI-powered cyber-threats are having a significant impact on their organization. To combat emerging challenges from AI-driven attacks, organizations of all sizes must leverage AI-powered tools that can provide granular real-time environment visibility and alerting to augment security teams. Where appropriate, they should get ahead of new threats by integrating machine-driven response, either in autonomous or human-in-the loop modes, to accelerate security team response. Through this approach, the adoption of AI technologies — such as solutions with anomaly-based detection capabilities that can detect and respond to never-before-seen threats — can be instrumental in keeping organizations secure. 

The State of AI Cybersecurity 2025 Report also noted that the increase of Shadow AI has introduced greater risk with security teams progressively seeking ways to lock down their data. I expect that we will see an explosion of tools that use AI and GenAI within enterprises and on devices used by employees. In addition to managing AI tools that are built in-house, security teams will see an surge in the volume of existing tools that have new AI features and capabilities embedded, as well as a rise in Shadow AI. If it remains unchecked, this raises serious questions and concerns about data loss prevention as well as compliance concerns as new regulations start to take effect. This drives an increasing need for AI asset discovery, the ability for companies to identify and track the use of AI systems throughout the enterprise. It is imperative that CIOs and CISOs dig deep into new AI security solutions — asking comprehensive questions about data access and visibility.

Moving forward, it will take a growing arsenal of defensive AI to effectively protect organizations in the age of offensive AI. Defensive AI includes tools that can detect anomalous behavior at scale by leveraging deep insights and intelligence into an organization’s assets. Whether the attack is AI-powered, automated, or a sophisticated threat actor, AI that identifies and isolates anomalous, suspicious behavior specific to an organization's normal patterns can detect and defend in machine time. 

As adversaries double down on the use and optimization of autonomous agents for attacks, human defenders will become increasingly reliant on and trusting of autonomous agents for defense. Specific types of AI can perform thousands of calculations in real time to detect suspicious behavior and perform the micro decision-making necessary to respond to and contain malicious behavior in seconds. Transparency and explainability in the AI outcomes are critical to foster a productive human-AI partnership.

Mr. Jamie Boote, Associate Principal Security Consultant at Black Duck:

Today, businesses are facing a landscape made up of maturing AI technology that is offering more mature tools that enable their employees in their jobs. Just like any technological shift changes how businesses operate, from the migration cloud hosting forcing companies to revamp their vendor agreements and data policy, to mobile applications forcing companies to put programmed logic and code on a new architecture that they don’t own or control, AI will leave its mark on governance.

As AI impacts how software is built, and what software is, governance will need to adjust to cover a vision for security in an era where software writes software. Right now, there is guidance around security testing and vulnerabilities for AI, however, the technology is moving so fast that categorical best practices that can be coded into governance are still being rewritten almost as fast as they’re committed to paper. 

To close the gap, organizations should instead build a strong vision for what secure AI use and development would look like in their company. That’s not easy or fast right now, but it is necessary for security teams that want to embrace AI at scale with confidence and don’t want to play catch-up with the engineering organization.

Dave Gerry, CEO at Bugcrowd:

AI is top of mind these days, and the introduction of America's AI Action Plan, a foundational step for U.S. leadership in AI, has only brought more attention. We commend its focus on accelerating innovation by cutting unnecessary regulations, which is crucial for the private sector to develop and deploy AI, and ultimately position the U.S. as a global AI leader. Investments in grid and datacenter infrastructure are vital to sustain AI's rapid growth. The plan's commitment to open-source AI and empowering the American workforce further solidifies this goal, though the interplay of state versus federal regulatory climates will be key to watch.

Now, let’s face the facts: AI-generated exploits and misinformation are already here. The big risk isn’t just technical, it’s trust erosion.  The security community needs to zero in on model manipulation (prompt injection, fine-tuning abuse), model output validation, and adversarial testing of AI systems.

This is where crowd-led testing matters most. Static models don’t adapt well to evolving tactics, however, diverse human researchers do. Initiatives that bring together defenders, researchers, and builders, such as VDPs and responsible disclosure programs, are crucial. We need more incentives for constructive interference — helping the AI ecosystem improve by breaking things safely.

Krishna Vishnubhotla, Vice President, Product Strategy at Zimperium:

Before the arrival of GenAI, attackers were already proficient at rapidly creating new domains to bypass traditional phishing detection tools. The focus was on speed and creating domains quickly to elude detection and launch attacks. However, with the rise of GenAI, phishing attacks have become more sophisticated and automated, making traditional security tools increasingly ineffective, particularly on mobile browsers. Sophistication shows up in the form of highly realistic and personalized, well-written phishing content at scale across all mobile phishing (mishing) vectors, including audio, video, and voicemail. The automation aspect allows attackers to clone websites in seconds, making brand impersonation easier. 

Combining these capabilities with the form factor or mobile browsers, with their limited URL visibility and reliance on auto-login features, provides the perfect storm for attackers to steal credentials without raising suspicion. The situation will only worsen as GenAI models improve and attackers start sharing capabilities and establishing Phishing-as-a-Service models to make money. Organizations need to adopt real-time, AI-driven mobile security to detect and block phishing before users are compromised. Relying on outdated defenses is no longer enough — security must evolve as fast as the threats.

Ms. Kris Bondi, CEO and Co-Founder of Mimoto: 

The generative nature of Shadow AI, and its lack of guardrails, makes it potentially more powerful and more dangerous than traditional shadow IT. An unmonitored AI can quickly evolve into providing inaccurate or even offensive responses. Because these are unsanctioned, they are many times a free version of AI that could be accidentally or intentional poisoned. Additionally, while shadow IT is officially unsanctioned, it is often known to be in use within a department. Workers are injecting shadow IT into their companies’ products or potentially exposing critical data to the external world. With Shadow AI, individual workers may decide to use it without telling anyone and may even hide their use from their coworkers. Its stealth usage adds to the risk associated with it.

All industries are vulnerable to the use of Shadow AI, but there are three types of industries where it will do the most damage. These are industries that are regulated, health or infrastructure-related, or where accuracy is critical. As you can imagine, these often overlap. To support innovation while minimize the threat of Shadow IT, enterprises must take a three-pronged approach of educating on the dangers of unsupported, unmonitored AI tools, create company protocols for what is not acceptable use of unauthorized AI tools, and most importantly provide AI tools that are sanctioned. Explaining while one tool is sanctioned and another isn’t greatly increases compliance. It does not work for a company to have a zero-use mandate. In fact, this results in an increase in stealth use of Shadow AI.