Azure Event Hubs
- Azure Event Hubs:
- Azure Event Hubs is a highly scalable data streaming platform and event ingestion(digesting) service capable of receiving and processing millions of events per second. Event Hubs can process and store events, data, or telemetry produced by distributed software and devices. Data sent to an event hub can be transformed and stored using any real-time analytics provider or batching/storage adapters. With the ability to provide publish-subscribe capabilities with low latency and at massive scale. Example use in IOT
- The service can be used to receive and process millions of events per seconds. It is a bigdata streaming platform and event ingestion service.
- Events hubs helps to manage data coming from multiple devices.
- If we want to send messages to event hubs i.e from .net core app we use the nuget package Azure.messaging.EventHubs
- EventHubProducerClient is used to send data to event hub.
- EventHubConsumerClient is used to consume the data present on event Hub.
- minimum 2 partition is required in event hubs. it is recommened to have 1 consumer for each partition. Maximum number of partition could be 32.
- We can not go and delete the messages from Eventhub even when we read the messages. The only way to delete the messages is to set the retention period of event hub.
- Offset: We must write our consumer in such a way it must remember where it last read the messages from event hub or it must remember the last batch our consuer read from event hub as in event hub we can not directly delete the messages. So old messages will be still avaliabel to consumer if those old messages are yet not delete. This is called offset, we must remember the offset value to start with. EventPostion.FromSequenceNumber() is the method used to get the offset value.
- We can track of offset by using Azure.Messaging.EventHubs.Processor instead of tracking by ourself in our consumer code. this stores the value of offset in azure sotrage account.
No comments:
Post a Comment