AI-aided Malvertising: Exploiting X's Grok Chatbot to Spread Phishing Scams
Cybercriminals are exploiting X’s AI chatbot, Grok, to disseminate phishing scams through a technique dubbed 'Grokking.' Threat actors bypass X’s restrictions on links in promoted posts by embedding malicious URLs within the metadata of clickbait video cards. When users or automated systems query Grok about the video's source, the AI reads the hidden link and amplifies it in its response, effectively turning the trusted bot into an unwitting accomplice. This method leverages the high domain reputation of Grok to enhance SEO and credibility, allowing scams to reach millions of impressions. The redirected links lead to credential-stealing forms and malware downloads, posing significant risks of identity theft and account takeover. Security researchers from ESET highlight that this indirect prompt injection attack underscores the broader dangers of trusting generative AI outputs. With prompt injection incidents rising globally, this case illustrates how attackers ingeniously circumvent security mechanisms on platforms integrating large language models. Experts warn that similar vulnerabilities could exist in other GenAI tools, urging users to treat AI-generated content with skepticism and caution.
Wire timeline
AI-aided Malvertising: Exploiting X's Grok Chatbot to Spread Phishing Scams
Cybercriminals are exploiting X’s AI chatbot, Grok, to disseminate phishing scams through a technique dubbed 'Grokking.' Threat actors bypass X’s restrictions on links in promoted posts by embedding malicious URLs within the metadata of clickbait video cards. When users or automated systems query Grok about the video's source, the AI reads the hidden link and amplifies it in its response, effectively turning the trusted bot into an unwitting accomplice. This method leverages the high domain reputation of Grok to enhance SEO and credibility, allowing scams to reach millions of impressions. The redirected links lead to credential-stealing forms and malware downloads, posing significant risks of identity theft and account takeover. Security researchers from ESET highlight that this indirect prompt injection attack underscores the broader dangers of trusting generative AI outputs. With prompt injection incidents rising globally, this case illustrates how attackers ingeniously circumvent security mechanisms on platforms integrating large language models. Experts warn that similar vulnerabilities could exist in other GenAI tools, urging users to treat AI-generated content with skepticism and caution.
WeLiveSecurity