Securing Millions of Messages in Real Time
At a glance
CLIENT
A global communications security and compliance platform provider
SERVICE
- Digital Risk Management, Cloud & Data Security, Cybersecurity Advisory, Data & Analytics
INDUSTRY
- Software & Hi-Tech / Business Information & Publishing
The client runs a multi-tenant SaaS platform that ingests and analyses over 4 billion email, chat and collaboration messages per day for more than 30,000 enterprise customers. The original product was a single monolithic application backed by a classic relational database. As message volumes and data residency requirements grew, the monolith hit scaling limits: ingest latency spiked during peak hours, new detection rules took months to roll out and compliance exports regularly overran their batch windows.
LeanCoded was asked to modernise the core platform into a cloud-native data and analytics layer for communications security. Over 14 months, we re-platformed the monolith into a streaming data architecture, introduced ML-based classification, and decoupled detection pipelines from UI and reporting. The result: ingest throughput increased by 8×, median detection latency dropped from 5 minutes to under 40 seconds, storage costs per message fell by ~35%, and the risk team can now introduce new policies in days instead of weeks. The programme combined custom software development services, cloud security assessments, data analytics consulting services and a cyber security service provider–style operating model for continuous improvement.
From Monolith Limits to Streaming Data Platform
When the monolith became the bottleneck
By the time LeanCoded engaged, the platform processed around 45 TB of new communications data per day across email, enterprise chat and collaboration tools. The monolithic codebase had grown to several million lines, and all detection logic, storage and reporting lived in a single deployment unit. During traffic spikes, message queues lengthened, causing delays in phishing and data-leak detection; some customers saw hours of lag between a message being sent and a policy action being applied. Adding new channels (for example, another collaboration suite) meant touching multiple layers of the monolith and coordinating long release cycles.
The client needed a new foundation that could scale horizontally, support channel-agnostic detection and meet regional data residency rules without spawning separate monoliths. They chose LeanCoded as a partner because we could combine custom software development skills with deep experience in cloud software development company patterns and data security service design, not just lift-and-shift the existing system.
Designing a communications security data platform
We started with a 10-week discovery and cloud security assessments phase. Together with the client’s architects and threat research team, we mapped the existing flows: ingestion connectors, normalisation, policy evaluation, machine learning components, archiving and legal hold processes. On top of that, we defined a target architecture: a streaming ingest layer, a centralised event bus, channel-agnostic enrichment services, pluggable detection engines and separated storage tiers for hot, warm and cold data.
The modernised platform uses an event-driven design: each message or event is normalised into a common schema and published to a central stream. Stateless services subscribe to that stream to run policy checks, ML-powered classification and compliance enrichment. This separation lets the client roll out new rules without redeploying the entire stack, and to scale specific components independently. The design also incorporates custom software application development for tenant-aware routing and cloud security network controls that enforce data residency and access policies per region.
Turning Communications Data into Actionable Risk Signals
Modernising the core architecture was only part of the job; the value of the platform comes from clear, high-precision signals about risk. LeanCoded worked with the client’s threat research and data science teams to redesign detection pipelines around three pillars: insider risk, phishing/malware and compliance violations (PII leakage, regulatory keywords, retention obligations).
We introduced a two-stage detection approach. First, lightweight stateless filters flag potentially risky messages based on structural and behavioural indicators: unusual sender/recipient patterns, suspicious domains, anomalous attachment types or deviations from normal communication graphs. Second, heavier ML models evaluate only those flagged items, drastically reducing compute. This design enabled wider deployment of ML without exploding infrastructure costs and aligned with best practices in AI software development services and data analytics consulting services for high-volume security workloads.
Cloud-native streaming ingest layer
Unified communications risk schema
Pluggable detection engines for risk and compliance
Tenant-aware storage and data residency controls
APIs and data feeds for downstream tools
Business and Security Impact
Before the programme, the platform’s growth was constrained by monolith limits: scaling required increasingly expensive hardware, rules were tightly coupled to channels, and analytics teams struggled to extract consistent data for customers and regulators. After the re-platforming, the client runs a modern communications security platform that can keep up with new channels and regulations without redesigning the core every time.
- 8× throughput and sub-minute detection
Ingest capacity grew by a factor of eight while median end-to-end detection latency dropped from about 5 minutes to under 40 seconds, even during traffic peaks.
- ~35% lower storage cost per message
Tiered storage and compression reduced effective storage cost per retained message by roughly a third, freeing budget for new detection capabilities.
- 40% faster rollout of new policies
Because detection engines are decoupled, the time to introduce new insider-risk or compliance policies fell from 4–6 weeks to 1–2 weeks, including testing and staged rollout. - Higher signal quality for customers
Better normalisation and ML-based scoring cut noisy alerts; several early-adopter customers reported a 30–50% reduction in false positives in their downstream SIEM rules after switching to the new feeds.
How LeanCoded Delivers Communications Security Platforms
LeanCoded approached this engagement not just as a software development company, but as a partner combining custom software development services, cloud security assessments and digital risk management expertise. We treated the platform as a long-lived product: designed an incremental migration path from monolith to microservices, established clear SLOs for ingest and detection latency, and set up joint teams with the client’s developers, operations and threat researchers.
Our engineers implemented the new streaming and storage layers while our security architects defined data-classification and access-control models aligned with the client’s regulatory obligations. A dedicated data team applied data analytics consulting services techniques to tune ML models and detection thresholds, and we helped the client’s operations group set up monitoring and runbooks similar to those used by a mature cyber security service provider. The result is a platform that can evolve with new channels, regulations and customer demands without a full rewrite every few years.