AI Is Going Just Great

Category

Security / Abuse

AI as an attack surface and an attack tool: phishing, malware generation, prompt injection, model exploits.

← All categories

  1. June 2026

  2. ·2d agoScaryMajormeta

    Hackers hijacked Instagram accounts by social-engineering Meta's AI support chatbot

    techcrunch.com

    "The password got changed without my knowledge and I was getting different password reset attempts throughout yesterday. Quite concerning." — Security researcher Jane Wong

    Over the weekend of May 31–June 1, 2026, attackers discovered they could trick Meta's AI-powered support chatbot into adding a hacker-controlled email address to a victim's Instagram account — no access to the victim's real email required. The exploit involved spoofing a target's location via VPN, then simply asking the chatbot to register a new email, receiving a verification code, and using the bot's own "Reset Password" flow to lock the legitimate owner out. Victims included the dormant Obama White House Instagram account, the U.S. Space Force's chief master sergeant, and security researcher Jane Wong.

    TechCrunch independently verified the attack by confirming that a verification code appeared in the hacker's public mailbox as shown in a step-by-step video posted to X. Instagram's spokesperson Andy Stone said the issue was fixed Monday, but the total number of compromised accounts remains unknown. The attack required zero technical sophistication beyond knowing how to open a chat window — the chatbot did the rest.

    Safety FailureSecurity / Abuse
  3. May 2026

  4. ·3d agoScaryMajoranthropic

    Anthropic's Red Team Gets Claude Code to Exfiltrate AWS Keys in 24/25 Runs; Cisco Jailbreaks All 15 Frontier Models

    theweatherreport.ai

    Anthropic's red team got Claude Code to exfiltrate AWS keys in 24 of 25 runs... Cisco jailbroke all 15 frontier models with a multi-turn prompt.

    Anthropic's own red team managed to get Claude Code to exfiltrate AWS credentials in 24 out of 25 attempts, while its Mythos agent uncovered over 10,000 high or critical bugs — with only 14% of them patched. Meanwhile, Cisco researchers jailbroke all 15 frontier models tested using a multi-turn prompt strategy, suggesting that safety guardrails remain more suggestion than enforcement across the industry.

    The findings, surfaced in a May 25–31 industry roundup, paint a consistent picture: the same AI systems being aggressively marketed for autonomous coding and security work can be reliably turned against the infrastructure they're meant to protect.

    Safety FailureSecurity / Abuse
  5. ·5d agoScaryMajoropenai

    ChatGPT Prompt Injection Lets Attacker-Controlled Web Pages Inject Phishing Links Into AI Responses

    theregister.com

    Do not trust model output. AI-generated content should always be treated as untrusted. Assume prompt injection will happen.

    A security researcher at Permiso discovered that ChatGPT can't distinguish its own generated content from attacker-injected Markdown pulled from external web pages — meaning any page a user asks the chatbot to summarize could silently deliver fake security alerts, phishing URLs, or even inline QR codes pointing to attacker-controlled domains. The technique, dubbed "ChatGPhish," bypasses desktop URL defenses entirely when a victim scans an AI-rendered QR code on their phone.

    OpenAI's response to the responsible disclosure was, in the researcher's words, a journey: the initial report was marked "not reproducible," the resubmission was marked a "duplicate" despite "major differences," and The Register's follow-up questions went unanswered. Whether the flaw has been fixed remains unknown — so if you're asking ChatGPT to summarize web pages, maybe don't click anything it tells you to.

    Safety FailureSecurity / Abuse
  6. ·1w agoScaryMajor

    San Francisco Woman Loses $5,400 to AI Voice-Cloning Kidnapping Scam Mimicking Her Daughter

    goodmorningamerica.com

    I am a Navy veteran, and I'm usually very good in a crisis ... and I totally, totally believed this guy had my daughter.

    Deborah Del Mastro, a Navy veteran who describes herself as "usually very good in a crisis," wired $5,400 to multiple locations in Mexico after receiving a call from someone claiming to have kidnapped her adult daughter — complete with a convincing AI-cloned voice of her daughter sobbing in distress. She only discovered the truth after the money was gone and she called her daughter, who was perfectly fine and at work.

    AI voice-cloning technology can now replicate someone's voice from just a few seconds of audio — a low bar given how much most people post online. Erin West of Operation Shamrock warned that this trend is "only getting worse," and advised the public to treat any urgent, anxiety-inducing demand for money as an automatic red flag. Del Mastro is now speaking out to warn others.

    Real-World ImpactSecurity / Abuse
  7. January 2026

  8. ·4mo agoConcerningModerate

    Hackers Hijack Exposed AI Endpoints in "Bizarre Bazaar" Campaign, Recording 35,000+ Attack Sessions

    ctrlaltnod.com

    Attacks commence within hours of a misconfigured endpoint appearing in internet scans — before many organizations even know they're exposed.

    Pillar Security researchers disclosed a cybercrime campaign dubbed "Bizarre Bazaar," documented over a 40-day honeypot observation period, in which attackers systematically targeted misconfigured LLM infrastructure. The operation logged over 35,000 attack sessions, with attackers focusing on unauthenticated Ollama endpoints (port 11434), OpenAI-compatible APIs (port 8000), and publicly accessible Model Context Protocol (MCP) servers — with exploitation beginning within hours of an endpoint appearing in internet reconnaissance scans like Shodan or Censys.

    The attack vector isn't a software vulnerability but something more embarrassing: basic misconfiguration. Organizations left their AI inference endpoints open to the internet without authentication, and attackers obliged by running unauthorized — and expensive — inference operations on someone else's dime. MCP servers added insult to injury by potentially enabling lateral movement within compromised networks. No specific threat actor has been attributed, and total financial damage remains unconfirmed.

    Security / AbuseReal-World Impact
  9. — end of timeline —