Batching

At dealer.com our applications have jobs that are made up of many different tasks run asychronously in many different way -- thread pools, messages queued onto message brokers, remote calls -- and we needed a centralized way to report out all the tasks for a given job had completed.  With this, we could kick off other work.  

We developed a system of batching that could work for all the different ways we kick off jobs.  At the beginning of a job, the initiator registers itself as a batch and sets the actions to execute when the batch completes, then each individual task registers itself.  On completion of each task, a message is sent to a queue where a listener looks up the task in the registry and determines if there are any other tasks left to be completed.  If not, the batch is considered complete, and a list of actions are executed.

 

 

Before

After