Zhipu AI’s ZCode tool secretly uploads user code repositories, sparking trust crisis

Common ground

• Zhipu AI's ZCode tool had a serious design flaw that silently uploaded user code, including sensitive data like passwords and API keys.
• The company's response—open-sourcing the code, inviting a CAICT audit, and promising zero-data retention—was faster and more transparent than typical industry practice.
• The encryption design, where data was encrypted client-side with a key only the server could decrypt, made it hard for users to detect what was being collected.
• The incident highlights a broader industry problem where AI companies treat user data as a resource to be extracted until public pressure forces change.
• Small developers and freelancers are most vulnerable in these situations, as they lack the resources to challenge large tech firms.

Points of contention

• Whether the data upload was a simple bug or a deliberate design choice reflecting a business model based on data extraction.
• Whether the controversy is a manufactured crisis aimed at undermining China's AI industry or a legitimate breach of trust that Zhipu brought on itself.
• Whether the encryption asymmetry was standard cybersecurity practice or an unusual design that prioritized data collection over user transparency.
• Whether the power imbalance between Zhipu and its users is a uniquely Chinese problem or a global feature of platform capitalism.
• Whether Zhipu's quick fix and audit prove accountability or merely show they acted only after public shaming.

Blind spots

• The debate largely ignored the specific legal and regulatory enforcement challenges that small developers face in practice, even under strong laws like China's Personal Information Protection Law.
• There was little discussion of what concrete, verifiable steps Zhipu could take to prove no data was retained, beyond promises and audits.
• The perspectives of Zhipu's actual enterprise clients—why they stayed silent and what contractual protections they have—were not explored in depth.
• No one addressed how the AI industry could shift from reactive crisis management to genuine privacy-by-design in coding tools.

WorldAttention’s read

The Zhipu AI controversy reveals a deep tension between rapid AI industry growth and user trust. While the company's response was faster and more transparent than many Western counterparts, the core issue remains: ZCode was designed in a way that made it hard for users to detect data collection, and the fix only came after public outcry. The debate shows that regardless of geopolitical framing or technical arguments, the real victims are small developers and freelancers who lack the power to push back. Moving forward, the industry needs to treat source code as sacred property, not free training data, and build privacy protections from the start—not as a damage-control afterthought.