Unprotected AI service streams private Slack messages for 30 bucks a month

Struct Chat, a $29.95 per month AI-powered Slack tool, organizes and summarizes threads, answers questions, generates newsletters and exposes its users’ private data and communications.

Struct Chat is a paid AI-powered chat application integrated into Slack and other messaging tools like Discord. It vows to take a privacy-first approach.

However, more than three months ago, the Cybernews research team discovered a company-owned unprotected web service streaming user data without authorization or validation.

The exposed instance of Apache Kafka Broker, a real-time distributed message streaming platform, handles large amounts of data. It acts as a central hub for moving data between different applications and is, therefore, a treasure trove for hackers.

Despite multiple responsible disclosure attempts, the leaking instance is still open as of January 27th, 2025, leaving customers in danger.

“While observing the data stream for a brief period, we encountered examples of GitLab commits, Slack Huddle conversations, and data from other services. This enables threat actors to track and read messages and other events in real-time and extract sensitive company and personal information without any restraints,” the researchers said.

Struct uses OpenAI’s ChatGPT service to create discussion summaries and organize information. Only the Slack integration was found to be leaking information.

The exposed service constantly streams the Slack users’ data, including the following:

• Tokens, IDs, first and last names
• Email addresses
• Conversations with other users and the bot AI, timestamps
• Internal team names and other general information
• Event data and type (what the user is doing, for example, updating Slack profile)
• Links to pipelines, internal URLs, CD/CI (Continuous Integration and Continuous Deployment) statuses.

“In one hour, the unprotected instance transmitted data from over 1,000 unique users from 200 unique companies. This leak can easily be exploited to gather users' personally identifiable information, such as full names, email addresses, chats, and other internal communications, various internal links and resources,” Cybernews researchers said.

Struct Chat claims to be an ‘innovative communication platform designed primarily for open-source communities.’ Cybernews reached out to the company for comment but to no avail.

The San Francisco-based firm self-proclaims to be a “pre-seed business.” The company’s website is two years old. However, the latest blog post was posted almost a year ago, while the last update in the community section of the website was made over ten months ago.

“This is not a Slack security issue but rather a case of an unlisted third-party app mishandling user data in violation of Slack’s API Terms of Service. Once we became aware of the issue, we took immediate action to revoke the app’s API access to protect customers and users,” Slack said in a statement.

Slack provides customers with tools to manage app installations and the scope of that data access, including the ability to restrict access to only vetted apps from the Slack Marketplace.

“Struct Chat was not listed in our Marketplace directory and did not go through our security and compliance review. We encourage all customers to review their installed apps, the permitted scopes of those apps, and use Slack’s security controls to manage third-party integrations,” the Slack team said.

The leak highlights AI integration risks

Struct Chat is just one of countless third-party AI tools that promise trustworthy and useful services but may introduce new disproportionately high risks due to poor implementation.

Cybernews previously reported on a chatbot service that collected and exposed user passports, detailed medical records, resumes, and other sensitive personal records.

“Out-of-the-box AI solutions can do more harm than good. The main danger of this leak is that some of the leaked information could potentially be used to breach the company's defenses, for extortion, blackmail, or to steal sensitive information,” Cybernews researchers warn.

“Be careful not to blindly install/use applications which are not fully tested or approved.”

The clients of Struct should be aware that their data is constantly leaking and should take appropriate action. Any resources shared in the exposed messages can be intercepted by threat actors.

It is critical to protect internal chat messaging services and ensure that personal or sensitive information is not accessible to outsiders. Businesses should also recognize that the responsibility for data security extends beyond the internal systems.

“Make sure to have proper access controls and reset compromised credentials. Stolen data can be used for phishing, scamming, or advertising purposes. Some of the worst cybercrime attempts could involve identity theft or business email compromise,” the researchers said.

• Leak discovered: October 14th, 2024
• Initial disclosure: October 16th, 2024
• CERT contacted: December 4th, 2024

Updated on January 31st [07:40 a.m. GMT] with a statement from Slack.