OpenAI reported three hacker groups that used ChatGPT to create malware. In particular, a Russian-speaking hacker group was identified that used the chatbot to create and refine a remote access trojan (RAT) and credential-stealing software optimized to evade detection, The Hacker News writes.
They also used multiple ChatGPT accounts to test and debug components that enable data theft and subsequent exploitation.
OpenAI said: “These accounts appear to be linked to Russian-speaking criminal groups, as we observed them posting proof of their activity in a Telegram channel.”
The company noted that although large language models refused to generate malicious content in direct requests, the hackers found a workaround. They bypassed the restriction by forcing the AI to produce separate fragments of code that were later assembled into full working workflows.
The attackers used only a few ChatGPT accounts but continuously refined the same code across different sessions. This indicates a prolonged development process rather than one-off test attempts.
A second cluster of activity originated from North Korea. OpenAI determined that those hackers used ChatGPT to create malware and command-and-control (C2) servers. They focused on tasks such as developing Finder extensions for macOS, configuring VPN connections on Windows Server, or converting Chrome extensions into versions compatible with Safari.
Additionally, the attackers used the chatbot to craft phishing emails, experiment with cloud services and GitHub features, and explore techniques for DLL sideloading, in-memory execution, Windows API hooking, and credential theft.
The third set of blocked accounts is linked to the UNK_DropPitch (UTA0388) cluster. This is a Chinese hacker group known for phishing attacks on large investment firms, especially in Taiwan’s semiconductor sector, using the HealthKick (GOVERSHELL) backdoor.
The attackers generated phishing campaign content in English, Chinese, and Japanese; sought help developing tools to speed up routine tasks (such as remote execution and protecting traffic via HTTPS); and researched how to install open-source tools like nuclei and fscan.
