Posted by Harjinder Singh

Nowadays cloud-based, microservice-based, or Internet-of-Things (IoT) applications every so often depend on communicating with other systems across an unreliable network, which leads to unavailability or unreachability for these systems due to transient faults such as network problems and timeouts, being offline, under load or maybe non-responsive.

Polly, a .NET resilience and transient-fault-handling library, offers multiple resilience policies which enable software architects to design suitable reactive strategies for handling transient faults, and also proactive strategies for promoting resilience and stability. Through this post, I will walk you through all the policies/strategies/approaches which Polly library offers to handle transient faults.

Reactive transient fault handling approaches

1. Retry

These short-term faults typically correct themselves after a short span of time, and a robust cloud application should be prepared to deal with them by using a strategy like the “Retry pattern”.

Technically Retry allows callers to retry operations in the anticipation that many faults are short-lived and may self-correct; retrying the operation may succeed maybe after a short delay.

Waiting between retries, allows faults to self-correct. For example, Practices such as ‘Exponential backoff’ and ‘jitter’ enhance this by scheduling retries to avoid them becoming sources of further load or spikes.

2. Circuit-Breaker

There can also be circumstances where faults are because of unexpected events that might take much longer to fix themselves. In these situations, it might be useless for an application to continually retry an operation that is unlikely to succeed. As an alternative, the application should be coded to accept that the operation has failed and handle the failure accordingly.

Using Http retries in these situations could lead to creating a Denial of Service (DoS) attack within your own software. Therefore, you need a defence barrier so that excessive requests stop when it is not worth to keep trying. That defence barrier is precisely the “Circuit Breaker”.

How does circuit-breaker work?

A circuit breaker design pattern perceives the level of faults in calls placed through it and prevents calls when a configured fault threshold is reached.

The circuit breaker can have the following states:

• Closed to Open
• Open to Half-Open
• Half-Open to Open
• Half-Open to Closed

When faults exceed the threshold, the circuit breaks itself (Opens). An open circuit will result in call failure placed through it instead of being actioned and throws an exception immediately. Which means that call is not attempted at all. Thus, it will significantly protect both, a faulting system from an extra load, and allows the working system to avoid placing calls which are unlikely to succeed. Failing instantly in this scenario usually also promotes a better user experience.

After a certain configured time has been elapsed, the circuit moves to a half-open state, where the next call will be considered as a trial to determine the faulty system’s well-being. Based on this trial call, the breaker decides whether to close the circuit (resume normal operation) or break it again.

Note: Circuit-breaker implementation in software systems is like what is in electrical wiring; substantial faults will ‘trip’ the circuit, protecting systems regulated by the circuit.

3. Fallback

A Fallback policy defines how the operation should react in case, even with retries – or because of a broken circuit – the underlying operation fails repeatedly. How much well resilience-engineered your system, failures likely to occur. Fallback means defining a substitute as what you will do when that happens. Its a plan for failure, rather than leave it to have unpredictable effects on your system.

Proactive transient fault handling approaches

Retry and Circuit-Breaker are primary approaches for resilience to transient faults, however, both are reactive, as they react after the failure response to a call has been received.

What will happen if…???

1. Response never comes 
2. The response so delayed that we do not wish to continue waiting
3. Waiting-delay / Waiting-to-React logic could itself have problems

Therefore, to handle the above questions, is it possible to be more proactive in our approach to resource management and resilience? 

Consider a scenario, wherein a high-throughput system, many calls can be put through to a recently failed system before the first timeout is received. For illustration (Example one), with 100 calls/second to a faulted system that is configured with a 10-second timeout, 1000 calls could have been placed before the first timeout is received. A circuit-breaker will react to this situation as soon as defined threshold failures are reached, but till then a resource bulge has certainly occurred and cannot be backtracked.

While Retry and Circuit-Breaker are reactive; Timeout, Bulkhead, and Caching policies configurations allow pre-emptive and proactive strategies. High-throughput systems can achieve increased resilience by explicitly managing load for stability. We will walk through each of these in the below sections one by one.

1. Timeout

Timeout allows callers to walk away from an awaiting call. It will improve resilience by setting callers free when a response seems unlikely.

Moreover, as the above given ‘Example one’ data demonstrates, opting for timeouts can influence resource consumption in a faulting, high-throughput system. 

These kinds of scenarios often lead to blocking up of threads or connections, the memory those awaiting calls consume, and which causes further failures. Consider how long you want to let your awaiting calls consume these costly system resources.

2. Bulkhead

The excessive load is one of the main reasons for system instability and failure. Therefore, building resilience into systems involves explicitly dealing with that load and/or pro-actively scaling to support it.

Excessive load can either be due to:

1. Genuine external demand, for example, spikes in user traffic 
2. Or maybe due to faulting scenarios, where large numbers of calls back up.

Bulkhead policies promote stability by directly managing load and thus resource consumption. The Polly Bulkhead limits parallelism of calls placed through it, with the option to queue and/or reject excessive calls.

• Bulkhead as isolation : Enforcing a bulkhead policy, a parallelism limit, around one stream of calls, limits the resource that stream of calls can consume. If that stream faults, it cannot consume all resources in the host and thus save bringing down the whole host.
• Bulkhead as load segregation : More than one Bulkhead policy can also be enforced in the same process to accomplish load segregation and relative resource allocation.
  A good comparison will be the check-out lanes of supermarkets, where there are often distinct lanes for “baskets only” as opposed to full shopping carts. This segregation permits for ‘baskets-only’ always to check-out quickly. Otherwise, they could be blocked waiting behind an excess of full shopping carts.
  Configuring multiple bulkhead policies to separate software operations delivers similar advantages, both in terms of relative resource allocation for different call streams; and in ensuring one kind of call cannot exclusively prevent another.
• Bulkhead as load-shedding : Bulkhead policies can also be configured to proactively reject calls beyond a certain limit.
  Why actively reject calls when the host might yet have more capability to service them?
  The answer is it depends on whether you prefer managed or unmanaged failure. Recommending explicit limits enables your systems to fail in predictable and testable ways. Intentionally ignoring the system’s capacity to the server does not mean there is not a limit; it just shows you do not know where the limit ends. Thus, your system is liable to unpredictable, unexpected failures.

3. Cache

Whatever that lessens network traffic, and overall call duration; in turn, rises resilience and improves user experience.

Caching can be configured with multiple caches – in-memory/local caches/distributed caches – in combination. Polly CachePolicy provisions multiple caches in the same call, using PolicyWrap.

Conclusion: This post walks us through multiple configurable resilience policies offered by Polly Library. Neova has the expertise to implement in business applications to make them fault resilient.

Note: For code level details you can reach GitHub community demonstrating all these configurations in detail.

Introduction

Rapidly increasing count of electronic gadgets connected to the internet has given birth to a new IT jargon, Internet-of-things (IoT). It is a system of interconnected computing devices, digital and mechanical machines identified by a unique number/name, and its facility to transfer data over a network without requiring human-to-computer or human-to-human interaction. Therefore, this requirement of improving system integration triggered the development of Message-Brokers, which facilitate the inter-application communication technology to support building a common integration mechanism to support cloud native, microservices-based, serverless and hybrid cloud architectures. 

Why Did Message Brokers emerge?

Can you imagine the existing volume of data for gadgets connected to the internet across the globe? Nowadays, approximately 12 billion Smart-Machines are connected to it. Bearing in mind that currently, around 7 billion people live on this planet, it is almost one-and-a-half devices per person. By the end of this year, this number could presumably surge to 200 billion, or even further. With technological development, like the building of Smart-Houses and other automatic systems, our everyday life becomes more and more digitized.

What is a message broker?

Wikipedia’s formal definition says: “A message broker translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver.” 

Functional features for a message broker

• A message broker is computer software that enables apps, systems, and services to communicate with each other by exchanging information, irrespective of programming languages used to build them up or software/hardware platforms wherein they are deployed.
• Message brokers are software programs deployed in messaging middleware or message-oriented middleware (MOM) solutions.
• Message brokers can validate, store, route, and deliver messages to the intended destinations. This enables decoupling of services and processes within systems.
• Built-in “Message-Queues” hold intact messages until not delivered to intended destinations and ensure message reliability and guaranteed delivery. 
• Message brokers provide queue managers to handle the interactions between multiple message queues, as well as services providing data routing, message translation, persistence, and client state management functionalities.
• Supports Asynchronous messaging, which prevents the loss of valuable data and enables systems to continue functioning even in the face of the intermittent connectivity or latency issues common on public networks. 

Message broker implementation patterns

Message brokers can be configured in two basic message distribution patterns:

• Point-to-point messaging: Each message in the queue is sent to only one recipient and is consumed only once, so it represents a one-to-one relationship between the message’s sender and receiver. Suitable use cases for this messaging pattern include payroll and financial transaction processing. In these systems, both senders and receivers need an assurance that each payment will be sent once and once only.

• Publish/subscribe messaging: Often referred to as “Publisher/Subscriber”, wherein the producer of each message publishes it to a queue, and multiple message consumers subscribe to a queue from which they want to receive messages. All messages published to a queue are distributed to all the applications subscribed to it. It is a broadcast-style distribution method, depicting a one-to-many relationship between the message’s publisher and its consumers. For example, an airline was to circulate updates about the landing times or delay status of its flights, multiple parties could make use of the information: ground crews performing aircraft maintenance and refuelling, baggage handlers, flight attendants and pilots preparing for the plane’s next trip, and the operators of visual displays notifying the public. A pub/sub messaging style would be suitable for this scenario.

Note: Third possible pattern could be – Hybrid of both

Best Message Broker Software Tools

These tools are typically leveraged by IT professionals like system administrators, and software developers. Business organizations use this software to synchronise distributed applications, simplify coding dissimilar applications, improve performance, and automate communication-related tasks. 

A software tool must possess below given to qualify as a Message Broker:

• Facilitate asynchronous messaging
• Store, deliver, and delete messages
• Document communication information
• Allow administrative control over messaging permissions

Major available tools

• RabbitMQ
• Apache Kafka
• IBM MQ
• Apache ActiveMQ

There are numerous message broker software tools available in the market which can be the best candidate for a given factors like Flexible Message Routing and Priority configurations, Message Replay, UI to monitor messages etc. E.g. RabbitMQ is a great fit for Flexible Message Routing and Priority configurations and Kafka is a great choice for message replay. Next, we will explore RabbitMQ as it covers the maximum share in the current market.

RabbitMQ a most widely deployed open source message broker

RabbitMQ is the most popular open-source message broker, having a huge number of production deployments world-wide. It is lightweight, and easy to deploy on-premises and in the cloud and execute on all major operating systems. It supports multiple messaging protocols, maximum developer platforms and can be deployed in distributed and federated configurations to satisfy high-scale, high-availability requirements.

RabbitMQ Features

Asynchronous Messaging

It supports multiple messaging protocols, message queuing, multiple exchange types, delivery acknowledgement, flexible routing to queues.

Developer Experience

It allows developers to deploy with Docker, BOSH, Chef and Puppet. Cross-language messaging with favourite programming languages such as: .NET, Java, PHP, Python, JavaScript, Ruby, and many others.

Distributed Deployment

Offers Clustered deployment for high availability and throughput and federate across multiple availability zones and regions.

Enterprise & Cloud Ready

It supports pluggable authentication, authorisation as well as supports LDAP and TLS. Lightweight and easy to deploy across public and private clouds.

Tools & Plugins

Variety of plugins and tools supporting continuous integration, operational metrics, and integration with other enterprise systems. Flexible plug-in approach for extending its functionality.

Management & Monitoring

It supports HTTP-API, command line tool, and UI for managing and monitoring itself.

.NET Libraries for RabbitMQ

Below are given two client libraries which help to integrate .NET applications with RabbitMQ very easily.

• RabbitMQ .NET Client (supports .NET Core and .NET 4.5.1+)
• RawRabbit, a higher-level client that targets ASP.NET next and supports .NET Core.

Note: Here in, I will be covering only RawRabbit by highlighting its core features and providing essential code snippets.

RawRabbit

It is a modern .NET framework for communication over RabbitMQ. The modular design and middleware-oriented architecture makes the client highly customizable while providing workable default for topology, routing and more.

Easily Configurable and extendable

RawRabbit can be easily configured with RawRabbitOptions, an option specifying object which allows registering client configuration, plugins, and permits overriding internal services.

var client = RawRabbitFactory.CreateSingleton(new RawRabbitOptions
{
ClientConfiguration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("rawrabbit.json")
.Build()
.Get<RawRabbitConfiguration>(),
Plugins = plugin => plugin
.UseProtobuf()
.UsePolly(config => config
.UsePolicy(queueBindPolicy, PolicyKeys.QueueBind)
.UsePolicy(queueDeclarePolicy, PolicyKeys.QueueDeclare)
.UsePolicy(exchangeDeclarePolicy, PolicyKeys.ExchangeDeclare)
),
DependencyInjection = ioc => ioc
.AddSingleton<IChannelFactory, CustomChannelFactory>()
});

Configuring Publish/Subscribe

Just a few lines of code and you can set up strongly typed publish/subscribe.

var rawClient = RawRabbitFactory.CreateSingleton();
await rawClient.SubscribeAsync<BasicMessage>(async message =>
{
Console.WriteLine($"Received: { message.Prop}.");
});

await rawClient.PublishAsync(new BasicMessage { Prop = "Hello, world!"});

Configuring Request/Response

RawRabbit’s (RPC) request/response configuration uses the direct reply-to feature for better performance and lower resource allocation.

var rawClient = RawRabbitFactory.CreateSingleton();
rawClient.RespondAsync<BasicRequest, BasicResponse>(async request =>
{
return new BasicResponse();
});
var rawResponse = await client.RequestAsync<BasicRequest, BasicResponse>();

Configuring Ack, Nack, Reject and Retry

Unlike many other clients, basic.ack, basic.nack and basic.reject are first class citizens in the message handling.

var client = RawRabbitFactory.CreateSingleton();
await client.SubscribeAsync<BasicMessage>(async message =>
{
if(UnableToProcessMessage(message))
{
return new Nack(requeue: true);
}
ProcessMessage(message)
return new Ack();
});

In addition to the essential acknowledgements, RawRabbit also supports delayed retries

var client = RawRabbitFactory.CreateSingleton();
await client.SubscribeAsync<BasicMessage>(async message =>
{
try
{
ProcessMessage(message)
return new Ack();
}
catch (Exception e)
{
return Retry.In(TimeSpan.FromSeconds(30));
}
});

Granular control over each call

Allow users to add or change properties in the IPipeContext to adapt calls for specific types of messages. This makes it possible to modify the topology features for calls, publish confirm timeout, consumer concurrency.

await subscriber.SubscribeAsync<BasicMessage>(received =>
{
receivedTcs.TrySetResult(received);
return Task.FromResult(true);
}, context => context
.UseSubscribeConfiguration(config => config
.Consume(consume => consume
.WithRoutingKey("CustomKey")
.WithConsumerTag("CustomTag")
.WithPrefetchCount(2)
.WithNoLocal(false))
.FromDeclaredQueue(queue => queue
.WithName("CustomQueue")
.WithAutoDelete()
.WithArgument(QueueArgument.DeadLetterExchange, "dlx"))
.OnDeclaredExchange(exchange=> exchange
.WithName("CustomExchange")
.WithType(ExchangeType.Topic))
));

Conclusion

Neova Solutions Pvt Ltd has the expertise to integrate Message Broker Software, like Kafka, RabbitMQ, into business applications to meet high-scale, high-availability requirements.