Signal out
Normalized events, resolved entity references, collection metadata and early correlation findings, delivered to the TDIR engine.
Platform · Collection layer
The collection layer decides what is possible everywhere else
iCollector is the collection layer of the ClearSkies iISOC platform. It is deployed on customer premises as a virtual or physical appliance, it gathers log and event data from native and third-party systems, and it forwards the result to the platform over an encrypted channel. That much it shares with any forwarder.
What distinguishes it is the work it performs before the forwarding step.
EdgeProcessing where the data is produced
De-identifiedMasked inside the boundary
Complete recordBuffered locally through outages
Why it matters
The consequence is architectural rather than incremental
Correlation across sources works because the schema was shared before the event arrived, not because a connector reconciled two formats afterward. De-identification protects the organization because the fields never travel, not because they were removed at the far end after the disclosure had already occurred. The archive is usable as evidence because the signature belongs to the customer. And the record stays complete through an outage because the appliance holds what it cannot send and delivers it when the link returns.
Every outcome feeds back into the next cycle. Detection tuning, routing thresholds and playbook selection are adjusted on measured results rather than on assumption, which is what makes this a lifecycle rather than a pipeline
The problem
Masking performed centrally is masking performed after the data has left
Four things go wrong in conventional ingestion. They are not independent, and a platform that addresses only some of them inherits the rest. The one below matters most to a regulated organization, and it is the one most often described as solved when it is not.
Primary issue01
Whatever the platform does with a cardholder number or a national identifier at that point, the number has already crossed the boundary, been transmitted across a network the organization does not fully control, and landed in storage governed by somebody else's contract.
GDPR frames this as data minimization, which is an obligation about what is transferred rather than about what is eventually deleted. NIS2 and DORA both turn on the ability to evidence what happened, which is difficult to reconcile with a design that discloses first and remediates second.
Three related failure points
Select each item to understand the consequence.
02Collection treated as transport
Every appliance in the infrastructure becomes a pipe, and all of the processing lands on a single ingestion point. Cost concentrates there, latency accumulates there, and the appliances themselves contribute nothing to detection despite sitting closest to the evidence.
03Schema translation after the fact
The quality of every correlation downstream depends on how well independent mappings happen to agree. Sources that change their format are a silent problem, because the mapping keeps running and simply stops being right.
04Events lost without anyone noticing
Raw volume is transported and stored in full, parsers are re-tuned whenever a source changes, and events are lost whenever a link drops with nobody discovering the gap until an investigation reaches the window where the evidence should have been.
Ingestion cost is now the most common objection raised against platforms of this class, and it is driven almost entirely by decisions taken, or not taken, at the point of collection.
What happens before anything is forwarded
Five functions, each a mechanism rather than a feature
- 01
Normalize
Maps a copy of the collected data into the one context-aware data model that the TDIR engine and every Core Function read. This is the function that makes cross-source correlation possible.
Why it matters: correlation quality does not degrade as sources are added, because each new source joins the same model rather than requiring a new reconciliation.
- 02
Mask
Obscures nominated sensitive fields, such as cardholder numbers or national identifiers, before the data is forwarded. Masking is configured per deployment against the fields the organization identifies.
Why it matters: the disclosure does not occur, rather than occurring and being remediated at the far end.
- 03
Archive
Compresses collected data, digitally signs it using a signature created during installation and held by the customer, and optionally encrypts it. Because the key belongs to the customer rather than to the platform, the archive preserves original fidelity in a form suitable for forensic investigation or legal evidence.
Why it matters: the archive is verifiable independently of the platform, because the key is yours.
- 04
Collect
Gathers structured, semi-structured and unstructured data from security devices, network infrastructure, servers, applications, the native add-ons and Marketplace integrations. New and evolving sources are handled with AI-assisted parsing, which reduces the engineering effort a previously unseen format would otherwise require.
Why it matters: a previously unseen format is not a fresh engineering project each time.
- 05
Correlate
Applies statistical and heuristic models to the collected stream, surfacing indicators of targeted activity closer to the source and earlier in the timeline. This is early indication rather than incident resolution, which remains the work of the TDIR engine.
Why it matters: indicators surface earlier in the timeline than a central-only design allows.
The collection and processing path. Every function shown to the left of the boundary occurs inside the customer premises.
From events to entities
The value is not the event but the entity
The collection layer sees something no other layer sees: every event from every in-scope source in its original form, before any consumer has interpreted it.
It also holds metadata that exists nowhere downstream: which source produced the event, when the collector received it as against when the source claims it occurred, which path and which collector role it arrived through, whether it carries a valid signature, and whether it came in live or through buffered catch-up after an outage.
That metadata is not administrative detail. A clock that runs two minutes fast on one system will misorder an attack chain unless something recorded both timestamps, and the only component in a position to record both is the one that received the event.
When a field is mapped into the shared model at collection, it stops being a string in one vendor's format and becomes a reference to a user, a host, an identity, a domain, an asset or a session. The Centric-AI Fabric then resolves those references across every source that contributed one. Entity resolution downstream can only be as good as the field mapping upstream, which is what makes this layer decisive rather than clerical.
The exchange runs in both directions
Between the platform and the collection layer, context and instructions flow down as well as up. What does not flow is intelligence between capabilities. No component reads what another reported.
Context in
Parser and mapping updates from the Detection Factory, masking and retention policy per tenant, source scoping informed by what the platform actually detects on, and collection priorities that follow the current risk picture. A standalone forwarder holds none of this.
Action back
During response the platform reaches through the collection layer as a control point: adjusting scope, raising retention on a source under investigation, and preserving a signed archive segment for evidence.
The governing rule is the platform's own. No component acts on the intelligence of another component except as the platform governs it.
Four sources, four formats, four identifiers for two entities
The chain below is illustrative and is drawn from four sources that describe two entities between them. The point is not the sophistication of the attack. It is that nothing in the left-hand column can be joined to anything else in it.
What each source produces alone
A firewall logs an outbound session, identifying the internal host by IP address only. Severity: informational.
What normalization at collection makes possible
The host resolves to an asset entity at collection, using context the appliance holds locally, so the session is attributable before the engine ever sees it.
What each source produces alone
A domain controller logs an authentication, identifying the user by SAM account name.
What normalization at collection makes possible
The account maps into the same identity entity that the cloud identity provider populates under a different attribute. Two sources now describe one identity rather than two.
What each source produces alone
An endpoint agent logs a process creation under a local username and a machine name.
What normalization at collection makes possible
The endpoint event joins the same asset and identity, placing process behavior on one timeline with the network and authentication events.
What each source produces alone
A SaaS application logs an administrative change under an email address, with a clock running two minutes ahead.
What normalization at collection makes possible
Collection timestamps recorded alongside event timestamps let the platform sequence the change correctly rather than placing it before the authentication that enabled it.
Cross-layer correlation is not something the engine achieves in spite of the data it receives. It is something the collection layer makes possible before the engine sees an event at all.
A conventional forwarder handles the same four sources very differently. It sends the full raw volume across, sensitive fields travel intact instead of being masked, and records are matched up only after they arrive, often against identifiers that do not line up. There is no integrity guarantee on the record, and the timeline carries a gap for every minute the connection was down. Doing this work at the point of collection, before the data ever moves, is the only stage at which it can still be done cheaply.
Capacity grows by addition rather than by enlargement
An infrastructure spread across sites is served by deploying multiple collectors and aggregating their output. A role describes an architectural arrangement rather than a commercial one.
| Role | What a collector in this role does |
|---|---|
| Enterprise | Communicates with the platform. Supports all collector functionality. Receives normalized data from collectors in the Standard and Forwarder roles. |
| Standard | Supports collection, archiving and normalization. Reduces the processing load carried by the Enterprise-role collector. Receives data from Forwarders. |
| Forwarder | Forwards log and event data to a collector in a higher role. |
Because processing is distributed across the arrangement rather than concentrated at one ingestion point, capacity grows by adding collectors close to the sources that need them. That is a different scaling property from enlarging a single appliance, and it is the reason a large distributed infrastructure does not develop a bottleneck at the boundary.
The distributed collection model. Collectors in the Forwarder and Standard roles sit close to the sources, and the Enterprise-role collector aggregates every site.
High availability
For environments where a collection outage is itself a compliance event, collectors run as a paired active and standby configuration. The two units share a single virtual IP, so log sources are unaffected by a failover and require no reconfiguration. They stay synchronized across a dedicated interface, and if the active unit fails the standby takes over automatically. Every size can be deployed as a high-availability pair.
Buffering through outages
Pairing and buffering address different failures and both are required. Pairing covers the loss of an appliance: collectors run as a paired active and standby configuration sharing a single virtual IP, so log sources are unaffected by a failover and require no reconfiguration. Buffering covers the loss of the link to the platform, which pairing does nothing about. When the channel is interrupted the collector holds what it cannot send and delivers it automatically once connectivity returns.
Buffering is bounded by local capacity. The bound is a fair question for any evaluator, and it is answered per deployment against the volume profile established at sizing.
Sizing and form factor
Six sizes, licensed per tenant rather than per appliance
| iCollector size | License, GB per day |
|---|---|
| Micro | Under 5 GB |
| Small | 5 – 20 GB |
| Medium | 20 – 75 GB |
| Large | 75 – 150 GB |
| XL | 150 – 300 GB |
| XXL | Over 300 GB |
A tenant licensed at a given size may deploy as many collectors as the layout of its sites requires, and the size governs the total daily volume across all of them. Each size is available as a virtual or as a physical appliance, and every size can be deployed as a high-availability pair.
Sizing is therefore a two-part exercise, and the two parts are independent. Total daily volume for the tenant determines the size, which is the commercial decision. The geography of the sources determines how many collectors are placed and in which role, which is the architectural one. A single site with concentrated volume and six sites sharing that same volume carry identical licensing and entirely different layouts.
What changes as the estate grows
Two of these effects appear on day one. The other two appear later, when sources multiply and the book of business does.
For the organization
- Normalization at collection means correlation quality does not degrade as sources are added, because each new source joins the same model rather than requiring a new reconciliation.
- Compression and de-identification reduce transported and stored volume, which is where ingestion cost is actually determined.
- Local buffering keeps the record complete through connectivity loss, so investigations and audit evidence do not run into a gap that nobody noticed at the time.
- Sensitive fields are masked before transmission, so the organization does not have to rely on a downstream deletion to satisfy an obligation about disclosure.
- The archive is signed with a key the organization created and holds, which makes it verifiable independently of the platform and usable in forensic or legal contexts.
For the service provider
- Collectors are placed close to each customer's sources and scale by addition rather than by enlargement, so a growing book does not concentrate load at one ingestion point.
- Processing performed on customer premises reduces the central capacity a provider has to fund per tenant.
- Six sizes and two form factors make collection a productized, per-tenant line item rather than a bespoke integration for every onboarding.
- Per-tenant masking and retention policy is applied at the collector, which supports the isolation position described in the MSSP brief.
- High-availability pairing protects the collection service level that regulated customers contract for, and failover requires no action from the customer.
The last point at which any of this is inexpensive
A platform can only correlate across sources that were mapped to a common model, can only protect fields that were removed before transmission, can only rely on records whose integrity somebody guaranteed, and can only investigate windows for which evidence exists.
See where it sits
How it works