Security Magazine logo
  • Sign In
  • Create Account
  • Sign Out
  • My Account
  • NEWS
  • MANAGEMENT
  • PHYSICAL
  • CYBER
  • BLOG
  • COLUMNS
  • EXCLUSIVES
  • SECTORS
  • EVENTS
  • MEDIA
  • MORE
  • EMAG
  • SIGN UP!
cart
facebook twitter linkedin youtube
  • NEWS
  • Security Newswire
  • Technologies & Solutions
  • MANAGEMENT
  • Leadership Management
  • Enterprise Services
  • Security Education & Training
  • Logical Security
  • Security & Business Resilience
  • Profiles in Excellence
  • PHYSICAL
  • Access Management
  • Fire & Life Safety
  • Identity Management
  • Physical Security
  • Video Surveillance
  • Case Studies (Physical)
  • CYBER
  • Cybersecurity News
  • More
  • COLUMNS
  • Cyber Tactics
  • Leadership & Management
  • Security Talk
  • Career Intelligence
  • Leader to Leader
  • Cybersecurity Education & Training
  • EXCLUSIVES
  • Annual Guarding Report
  • Most Influential People in Security
  • The Security Benchmark Report
  • Top Guard and Security Officer Companies
  • Top Cybersecurity Leaders
  • Women in Security
  • SECTORS
  • Arenas / Stadiums / Leagues / Entertainment
  • Banking/Finance/Insurance
  • Construction, Real Estate, Property Management
  • Education: K-12
  • Education: University
  • Government: Federal, State and Local
  • Hospitality & Casinos
  • Hospitals & Medical Centers
  • Infrastructure:Electric,Gas & Water
  • Ports: Sea, Land, & Air
  • Retail/Restaurants/Convenience
  • Transportation/Logistics/Supply Chain/Distribution/ Warehousing
  • EVENTS
  • Industry Events
  • Webinars
  • Solutions by Sector
  • Security 500 Conference
  • MEDIA
  • Videos
  • Podcasts
  • Polls
  • Photo Galleries
  • Videos
  • Cybersecurity & Geopolitical Discussion
  • Ask Me Anything (AMA) Series
  • MORE
  • Call for Entries
  • Classifieds & Job Listings
  • Continuing Education
  • Newsletter
  • Sponsor Insights
  • Store
  • White Papers
  • EMAG
  • eMagazine
  • This Month's Content
  • Advertise
Security Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Security Magazine logo
  • NEWS
    • Security Newswire
    • Technologies & Solutions
  • MANAGEMENT
    • Leadership Management
    • Enterprise Services
    • Security Education & Training
    • Logical Security
    • Security & Business Resilience
    • Profiles in Excellence
  • PHYSICAL
    • Access Management
    • Fire & Life Safety
    • Identity Management
    • Physical Security
    • Video Surveillance
    • Case Studies (Physical)
  • CYBER
    • Cybersecurity News
    • More
  • BLOG
  • COLUMNS
    • Cyber Tactics
    • Leadership & Management
    • Security Talk
    • Career Intelligence
    • Leader to Leader
    • Cybersecurity Education & Training
  • EXCLUSIVES
    • Annual Guarding Report
    • Most Influential People in Security
    • The Security Benchmark Report
    • Top Guard and Security Officer Companies
    • Top Cybersecurity Leaders
    • Women in Security
  • SECTORS
    • Arenas / Stadiums / Leagues / Entertainment
    • Banking/Finance/Insurance
    • Construction, Real Estate, Property Management
    • Education: K-12
    • Education: University
    • Government: Federal, State and Local
    • Hospitality & Casinos
    • Hospitals & Medical Centers
    • Infrastructure:Electric,Gas & Water
    • Ports: Sea, Land, & Air
    • Retail/Restaurants/Convenience
    • Transportation/Logistics/Supply Chain/Distribution/ Warehousing
  • EVENTS
    • Industry Events
    • Webinars
    • Solutions by Sector
    • Security 500 Conference
  • MEDIA
    • Videos
      • Cybersecurity & Geopolitical Discussion
      • Ask Me Anything (AMA) Series
    • Podcasts
    • Polls
    • Photo Galleries
  • MORE
    • Call for Entries
    • Classifieds & Job Listings
    • Continuing Education
    • Newsletter
    • Sponsor Insights
    • Store
    • White Papers
  • EMAG
    • eMagazine
    • This Month's Content
    • Advertise
  • SIGN UP!
CybersecuritySecurity NewswireCybersecurity News

Phishing Scams Can Deceive Large Language Models

By Jordyn Alger, Managing Editor
Coding on screen

Walkator via Unsplash

July 8, 2025

Netcraft researchers have discovered that large language models (LLMs), when asked to identify how to log into various platforms, would provide concerning results one-third of the time. 

Two-thirds of the time, the LLM would provide the correct login URL. The concerning one-third can be broken down as follows: 

  • 30% sent users to domains that were unregistered, parked or inactive (which could leave them at risk of takeover). 
  • 5% directed users to unrelated organizations.

Essentially, more than one in three individuals were sent to a site unassociated with the brand in question. 

The research asserts that the tests run were not edge-case prompts; rather, researchers used simple, casual phrases to simulate how an average user might realistically submit a prompt to an LLM. 

In one observed instance, the live AI-powered search engine Perplexity directed researchers to a phishing link. According to the researchers, the phishing link was not associated with a subtle scam, and Perplexity ignored signals such as domain authority or reputation.

Below, security leaders discuss these findings and their implications. 

Security Leaders Weigh In

Gal Moyal, CTO Office at Noma Security:

If AI suggests unregistered or inactive domains, threat actors can register those domains and set up phishing sites. As long as users trust AI-provided links, attackers gain a powerful vector to harvest credentials or distribute malware at scale.

Without guardrails enforcing URL correctness, AI responses can mislead users. Guardrails should validate domain ownership before recommending login, Any request/response containing a URL can be vetted using common practices, or use common practices such as domain reputation, known malicious URL websites, etc. 

AI can easily become a phishing delivery mechanism, highlighting the urgency of runtime protection to be put in place.

Nicole Carignan, Senior Vice President, Security & AI Strategy, and Field CISO at Darktrace:

LLMs provide semantic probabilistic answers with intentional variability to avoid repetitive outputs. Unfortunately, this mitigation strategy can also introduce hallucinations or inaccuracies. 

The research shows that approximately one-third of domains provided by the LLM were unregistered, parked, or unavailable — highlighting an emerging risk that can be easily weaponized by threat actors. When AI suggests one of these domains, it opens the door to malicious redirection, phishing, and credential harvesting. This, however, is not a new tactic. Threat actors have been leveraging typo-squatting — registering intentionally misspelled or lookalike domains to deceive users — for more than two decades.

The research also revealed a more dangerous threat; the intentional data poisoning or bias interjected into promoted GitHub repositories. The compromise of data corpuses used in the AI training pipeline underscores a growing AI supply chain risk. Data integrity, data sourcing, cleansing, and verification are critical to ensuring the safety and accuracy of LLM-generated outputs.

LLMs can and should have guardrails in place to mitigate this risk. One basic mitigation is to have LLMs ground or source any URL that is cited, essentially removing “generated” hostnames and replacing them with grounded, accurate hostnames.  

More broadly, this research points to a deeper issue: users are relying on generated, synthetic content from the outputs of LLMs as if it is fact-based data retrieval. LLMs don’t “retrieve” information — they generate it based on learned semantic probabilities from training data that users typically have no visibility into. Without proper sourcing, these systems become ripe for both inaccuracy and exploitation.

J Stephen Kowski, Field CTO at SlashNext Email Security+:

AI sending users to unregistered, parked or unavailable URLs creates a perfect storm for cybercriminals. When AI models hallucinate URLs pointing to unregistered domains, attackers can simply register those exact domains and wait for victims to arrive. It’s like having a roadmap of where confused users will end up — attackers just need to set up shop at those addresses and collect whatever sensitive information people try to enter.

This LMM behavior definitely needs immediate attention through input and output filtering systems. Traditional security measures struggle with AI-generated content because it looks legitimate and bypasses normal detection patterns. Real-time URL validation and domain verification before presenting results to users would catch these hallucinated links before they cause damage.

The one instance when AI actually provided a link to a phishing site is the most concerning finding because it shows AI can directly serve up active threats, not just create opportunities for future ones. This demonstrates how AI systems can become unwitting accomplices in phishing campaigns, essentially doing the attacker’s work by delivering malicious links with the authority and trust that comes with an AI recommendation. When users trust AI responses as authoritative, a single malicious link recommendation can compromise thousands of people who would normally be more cautious about clicking suspicious URLs. 

KEYWORDS: artificial intelligence (AI) phishing scams phishing threats

Share This Story

Looking for a reprint of this article?
From high-res PDFs to custom plaques, order your copy today!

Jordynalger

Jordyn Alger is the managing editor for Security magazine. Alger writes for topics such as physical security and cyber security and publishes online news stories about leaders in the security industry. She is also responsible for multimedia content and social media posts. Alger graduated in 2021 with a BA in English – Specialization in Writing from the University of Michigan. Image courtesy of Alger

Recommended Content

JOIN TODAY
To unlock your recommendations.

Already have an account? Sign In

  • Security's Top Cybersecurity Leaders 2024

    Security's Top Cybersecurity Leaders 2024

    Security magazine's Top Cybersecurity Leaders 2024 award...
    Security Enterprise Services
    By: Security Staff
  • cyber brain

    The intersection of cybersecurity and artificial intelligence

    Artificial intelligence (AI) is a valuable cybersecurity...
    Security Enterprise Services
    By: Pam Nigro
  • artificial intelligence AI graphic

    Assessing the pros and cons of AI for cybersecurity

    Artificial intelligence (AI) has significant implications...
    Cybersecurity Education & Training
    By: Charles Denyer
close

1 COMPLIMENTARY ARTICLE(S) LEFT

Loader

Already Registered? Sign in now.

Manage My Account
  • Security eNewsletter & Other eNews Alerts
  • eMagazine Subscriptions
  • Manage My Preferences
  • Online Registration
  • Mobile App
  • Subscription Customer Service

The Money Laundering Machine: Inside the global crime epidemic - Episode 24

The Money Laundering Machine: Inside the global crime epidemic - Episode 24

Security’s Top 5 – 2024 Year in Review

Security’s Top 5 – 2024 Year in Review

Middle East Escalation, Humanitarian Law and Disinformation – Episode 25

Middle East Escalation, Humanitarian Law and Disinformation – Episode 25

More Videos

Sponsored Content

Sponsored Content is a special paid section where industry companies provide high quality, objective, non-commercial content around topics of interest to the Security audience. All Sponsored Content is supplied by the advertising company and any opinions expressed in this article are those of the author and not necessarily reflect the views of Security or its parent company, BNP Media. Interested in participating in our Sponsored Content section? Contact your local rep!

close
  • Sureview screen
    Sponsored bySureView Systems

    The Evolution of Automation in the Command Center

  • Crisis Response Team
    Sponsored byEverbridge

    Automate or Fall Behind – Crisis Response at the Speed of Risk

  • Perimeter security
    Sponsored byAMAROK

    Why Property Security is the New Competitive Advantage

Popular Stories

Security’s 2025 Women in Security

Security’s 2025 Women in Security

Verizon on phone screen

61M Records Listed for Sale Online, Allegedly Belong to Verizon

blurry multicolored text on black screen

PowerSchool Education Technology Company Announces Data Breach

Half closed laptop

Sudo Vulnerability Discovered, May Exposes Linux Systems

Person holding cellphone

Millions of Android, iPhone Users Could Be Sending Data to China

Events

August 7, 2025

Threats to the Energy Sector: Implications for Corporate and National Security

The energy sector has found itself in the crosshairs of virtually every bad actor on the global stage.

August 27, 2025

Risk Mitigation as a Competitive Edge

In today’s volatile environment, a robust risk management strategy isn’t just a requirement—it’s a foundation for organizational resilience. From cyber threats to climate disruptions, the ability to anticipate, withstand, and adapt to disruption is becoming a hallmark of industry leaders.

View All Submit An Event

Products

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

Security Culture: A How-to Guide for Improving Security Culture and Dealing with People Risk in Your Organisation

See More Products

Related Articles

  • Tax documents and coffee mug

    Phishing tax scams increase as tax deadlines approach

    See More
  • Group brainstorming security methodologies

    Red teaming large language models: Enterprise security in the AI era

    See More
  • Tax documents and coffee mug

    Cybersecurity experts share how AI could enhance tax-related scams

    See More

Related Products

See More Products
  • s and the law.jpg

    Surveillance and the Law: Language, Power and Privacy

See More Products
×

Sign-up to receive top management & result-driven techniques in the industry.

Join over 20,000+ industry leaders who receive our premium content.

SIGN UP TODAY!
  • RESOURCES
    • Advertise
    • Contact Us
    • Store
    • Want More
  • SIGN UP TODAY
    • Create Account
    • eMagazine
    • eNewsletter
    • Customer Service
    • Manage Preferences
  • SERVICES
    • Marketing Services
    • Reprints
    • Market Research
    • List Rental
    • Survey/Respondent Access
  • STAY CONNECTED
    • LinkedIn
    • Facebook
    • YouTube
    • X (Twitter)
  • PRIVACY
    • PRIVACY POLICY
    • TERMS & CONDITIONS
    • DO NOT SELL MY PERSONAL INFORMATION
    • PRIVACY REQUEST
    • ACCESSIBILITY

Copyright ©2025. All Rights Reserved BNP Media.

Design, CMS, Hosting & Web Development :: ePublishing

Security Magazine logo
search
cart
facebook twitter linkedin youtube
  • Sign In
  • Create Account
  • Sign Out
  • My Account
Security Magazine logo
  • NEWS
    • Security Newswire
    • Technologies & Solutions
  • MANAGEMENT
    • Leadership Management
    • Enterprise Services
    • Security Education & Training
    • Logical Security
    • Security & Business Resilience
    • Profiles in Excellence
  • PHYSICAL
    • Access Management
    • Fire & Life Safety
    • Identity Management
    • Physical Security
    • Video Surveillance
    • Case Studies (Physical)
  • CYBER
    • Cybersecurity News
    • More
  • BLOG
  • COLUMNS
    • Cyber Tactics
    • Leadership & Management
    • Security Talk
    • Career Intelligence
    • Leader to Leader
    • Cybersecurity Education & Training
  • EXCLUSIVES
    • Annual Guarding Report
    • Most Influential People in Security
    • The Security Benchmark Report
    • Top Guard and Security Officer Companies
    • Top Cybersecurity Leaders
    • Women in Security
  • SECTORS
    • Arenas / Stadiums / Leagues / Entertainment
    • Banking/Finance/Insurance
    • Construction, Real Estate, Property Management
    • Education: K-12
    • Education: University
    • Government: Federal, State and Local
    • Hospitality & Casinos
    • Hospitals & Medical Centers
    • Infrastructure:Electric,Gas & Water
    • Ports: Sea, Land, & Air
    • Retail/Restaurants/Convenience
    • Transportation/Logistics/Supply Chain/Distribution/ Warehousing
  • EVENTS
    • Industry Events
    • Webinars
    • Solutions by Sector
    • Security 500 Conference
  • MEDIA
    • Videos
      • Cybersecurity & Geopolitical Discussion
      • Ask Me Anything (AMA) Series
    • Podcasts
    • Polls
    • Photo Galleries
  • MORE
    • Call for Entries
    • Classifieds & Job Listings
    • Continuing Education
    • Newsletter
    • Sponsor Insights
    • Store
    • White Papers
  • EMAG
    • eMagazine
    • This Month's Content
    • Advertise
  • SIGN UP!