Round ’em messages up and bring them home!

Message Cowboy is a standalone integration program that allows for scheduling of tasks that transfer files or messages from one directory, JMS queue etc to another directory, JMS queue etc. Messages may be transformed or processed in some other way before being delivered to the destination.
Some features are:

• Dynamic updates of task schedule.
  Tasks can be added, removed enabled or disabled without having to restart Message Cowboy.

• Dynamic updates of configuration files.
  Message Cowboy uses Mule 3.4.0 CE under the hood and will read Mule configuration files from certain directories. Configuration files may be modified, added or removed and changes will come into effect without having to restart the program.

• Embedded or external database server.
  Message Cowboy can be configured to use either an embedded HSQLDB database server or an external database server to store task configuration in.

• Embedded ActiveMQ JMS message broker.
  Message Cowboy can be configured to start an embedded Apache ActiveMQ JMS message broker.

Message Cowboy is open source, licensed under the GPL v3 license and is available here: https://github.com/krizsan/message-cowboy

Background

I started out wanting to see if I could create a program that allowed me to schedule transfers of files from one directory to another without having to restart the program after each configuration change.
I also stumbled across this article by Simon Brown about an architecturally evident coding style and wanted to, as far as my understanding allowed me, incorporate these thoughts in the design.

Technology-wise I have used the following frameworks and technologies:

• Java 7
  Due to the version of the Spring framework used, Message Cowboy will not run properly on Java 8. This is also the most recent Java version recommended for use with Mule.

• Mule 3.4.0 Community Edition
  Used to receive and dispatch messages and for any additional processing before a message is being delivered. As before, Message Cowboy allows for loading of any Mule configuration file so there is a potential (and a danger) in that you can create as complex Mule configurations as you want.

• Spring framework.
  Due to Mule having a dependency on Spring 3.2.1.RELEASE, this is the version I settled for. Since I have mostly worked with Spring XML configuration, I used Spring Java configuration. I feel that this helped me to produce code that is easier to understand and that has a stronger cohesion, with implementation and configuration of a component located next to each other.

• Spring Data JPA.
  Why implement repositories/DAOs if you do not have to?
  Due to the choice of version of the Spring framework above, I had to chose version 1.4.5.RELEASE of Spring Data JPA.

• Quartz scheduler 2.2.1.
  For scheduling. Are there any free, open-source, alternatives?

• EclipseLink 2.5.1

• HSQLDB
  For the embedded database. Message Cowboy can easily be configured to use an external database.

• Apache ActiveMQ 5.9.0
  Using an embedded JMS broker is optional.

• JUnit and Mockito.
  Old companions when it comes to testing.

As you may already suspect, the program grew to be a little more than just a proof-of-concept for scheduled file transfers. However I would still insist that it is just a little more than a proof-of-concept and far from being finished.

Design

The program consists of four service components, a (very small) domain model and some additional classes and configuration to tie things together.

I have tried to make the different services technology-agnostic, in order to be able to, for instance, swap Quartz for another scheduler or to create a version of Message Cowboy that uses Apache Camel. Time will tell whether I have succeeded in this area.
As hinted by the diagram above, I have also tried to encapsulate the inner workings of each service exposing only a single interface for each service. The encapsulation has been enforced using Java package visibility for classes and interfaces that are not to be available outside of the service. This aspiration has succeeded in all but one case; in the scheduling service, the class MethodInvokingJob has to be public in order for Spring to be able to create instances of this class using reflection.

The Domain Model

Before we look at the different services in detail, I want to say a few words about what I have called the domain model of the application.

There are three different types of entities:

• MoverMessage
  Represents a message received from once endpoint that is to be dispatched to another endpoint.

• SchedulableTaskConfig
  Holds the configuration of a schedulable task that is scheduled to run according to a cron expression between a start date and an end date.

• TaskJob
  Implements a job that is to be executed according to a schedule. I am not entirely sure that this is a proper entity.

The Scheduling Service

The scheduling service supports scheduling of message transfer tasks as well as tasks internal to the Message Cowboy, such as scheduled refresh of configuration etc.
Implemented using the Quartz scheduler.

The Task Configuration Service

The task configuration service is responsible for retrieving task configuration from the database – basically just a service that wraps a repository.
The repository, SchedulableTaskConfigurationRepository, is local to the service and cannot be accessed outside of the package it resides in.
The repository implementation is created by Spring Data JPA.

The Transport Service

The transport service receives and dispatches files and messages from and to different types of endpoints. As before, such endpoints may be directories in the file system, a JMS queue etc but can be any type of endpoint supported by Mule.

Also mentioned earlier is the feature that the transport service supports runtime refresh of its configuration. This is accomplished by disposing the old Mule context and creating a new one with the modified configuration files. Since this is an operation that may be time-consuming, the transport service retains a list of configuration files and their respective checksum and will only refresh the configuration if either the number of configuration files is different or if the checksum of one of the configuration files has changed.
Note that since the transport service’s configurations are refreshed periodically, it is not recommended to start some service that either holds state or that is assumed to be available at all times from such a configuration file since it may, at any point in time, be stopped and restarted.

The Starter Service

The starter service is responsible for starting and stopping the Message Cowboy, which also include scheduling of system tasks such as message transfer task refresh and transport service configuration refresh.

The starter service also refreshes the message transfer task schedule periodically. This is accomplished by unscheduling these tasks, reading the transfer task configuration from the database and scheduling all enabled tasks found in the database.

Building a Standalone Program

To obtain a stand-alone version of the Message Cowboy first build it using “mvn -DskipTests package” and then copy the following files and folders to the location of your choice:

• message-cowboy-1.0.0-SNAPSHOT.jar Application JAR-file.
  The version number may be different.
• message-cowboy-configuration.properties
  Message Cowboy configuration file.
• production-configurations
  Directory containing Mule configuration files.
• libraries
  Directory containing third-party libraries.

The relationships between the different files and folders must be preserved, in order for Message Cowboy to be able to run correctly.
In the default configuration, if using the embedded database, Message Cowboy will create a directory named “MessageCowboyDatabase” containing the HSQLDB database files in the same directory as the application JAR-file.

In the next article I will take a closer look at how to use the Message Cowboy.