Researchers Warn AI Coding Tools Need Stronger Security and Privacy by Default
Developers reported concerns ranging from unexpected file access and code execution to opaque data collection as AI coding assistants gain greater autonomy.
By The Register
Researchers are calling for stronger security and privacy protections to be built into AI coding tools by default after analysing hundreds of developer reports involving some of the industry's most widely used platforms.
The study examined so-called LLM-native development environments, or LIDEs, which integrate large language models directly into software development workflows. These tools can generate and modify code, inspect project files, run commands and increasingly carry out tasks with limited human intervention.
Researchers collected around 1.1 million posts from 29 Reddit communities focused on AI-assisted programming and identified 446 posts discussing genuine security or privacy problems. More than 6,000 associated comments were also analysed.
The findings covered tools including Cursor, GitHub Copilot and OpenAI Codex and identified a broad range of concerns. These included unauthorised file operations, unexpected or unsafe code execution, destructive actions, unclear data flows, telemetry collection and the potential exposure of sensitive information.
The researchers said many of the reported problems were linked not directly to the underlying AI models, but to the way development tools are designed around them. Giving an AI assistant access to large parts of a project, local files and system commands can increase the potential impact when the tool behaves unexpectedly.
Privacy concerns featured prominently in the research. The study identified 194 posts specifically related to privacy, with lack of transparency emerging as the most common complaint.
Developers also raised concerns about unauthorised access to data, unexpected collection or transmission of information and sensitive material being included in the wider context available to an AI assistant.
One reported example involved a Claude Desktop user apparently receiving messages associated with another user's session, which researchers classified as a failure of context integrity.
The study also found that developers frequently relied on protections outside the AI tools themselves, including sandboxing, manual code reviews and restricting how assistants were allowed to operate.
Researchers said this reliance on external safeguards suggested users did not always trust the security protections built into the tools they were using.
They are recommending that developers of AI coding platforms introduce stronger architectural guardrails, tighter protection around sensitive files and clearer controls over what information assistants can access or transmit.
The authors also called for generated code to be checked against security and privacy requirements, formal assessments of third-party tools and strict security settings to be enabled by default rather than requiring individual developers to configure them.
As AI assistants take on a larger role in software development, the findings underline a growing challenge for technology companies: improving productivity without giving increasingly autonomous tools more access than they need.