Event Streaming Client Architecture

Event Streaming Client can be deployed with or without high availability.

In all deployment modes, the solution includes:

• No service to be published publicly on the customer side, all requests from Event Streaming Client are outgoing
• Events are pushed from Event Streaming Server located in the tenant Management Plane to Event Streaming Client over GRPC tunnel. Event Streaming Client is not pulling events.
• The events are packaged in batches of 50MB maximum.
• No event loss with end to end acknowledgement (with syslog TCP): each event batch is acknowledged when fully streamed to the syslog target. In case of interruption during the processing or transfer, the batch will be restarted.
• An event batch is following a single path: all events of a batch are processed by a single Streaming Client and sent to only one syslog server.
• An event batch is only sent once, it cannot be distributed to multiple syslog servers.
• There are no communication required between the Event Streaming Clients.
• Data retention is only done in the Management Plane, not on the Event Streaming Client.
• Data processing is done in memory only, events are not stored on disk inside the Event Streaming Client.

Network flows

To work properly, the Event Streaming Client require outgoing Internet connectivity direct or via HTTP Proxy.

All flows are initiated by the Event Streaming Client, there are no service to publish publicly.

Direct

Event Streaming Client – Direct flows

HTTP Proxy

Event Streaming Client – HTTP Proxy flows

Event Streaming Client without High Availability

In this mode all batches are processed by a single Event Streaming Client and sent to a single syslog Server.

In this example, a single target is configured:

• Client 1 → Syslog Server A

Event path without High Availability

Event Streaming Client with High Availability on the Client

When two or more clients are available, the Event Streaming Servers are load balancing the event batches to each Event Streaming Clients.

In this example, 2 targets are configured:

• Client 1 → Syslog Server A
• Client 2 → Syslog Server A

Event path with High Availability

Event Streaming Client with High Availability on the Client and the Syslog Server

In addition to Event Streaming Client load balancing, when a Client has two or more Syslog Servers defined, the batches will be load balanced by the Client between available Syslog Servers.

• In this example, 4 targets are configured:
  • Client 1 → Syslog Server A
  • Client 1 → Syslog Server B
  • Client 2 → Syslog Server A
  • Client 2 → Syslog Server B

Event path with High Availability and Syslog load balancing

Event Streaming Client with High Availability on Multiple Datacenters

To provide High Availability on multiple datacenters, it’s possible to have one Event Streaming Client on each location.

• In this example, 4 targets are configured:
  • Client 1 → Syslog Server A
  • Client 2 → Syslog Server A
  • Client 3 → Syslog Server B
  • Client 4 → Syslog Server B

Event path with load balancing on multiple locations