In this article I will show how I created a Docker image for Elastalert and create an automated build for the image on Docker Hub.
Among other things, I will show how to wait on other services that a service running in a Docker container depends on.

Introduction

For those who are not familiar with Elastalert I can briefly mention that it is a piece of software that can monitor data in an Elasticsearch instance and generate alerts upon detecting a variety of conditions. I were originally looking at Watcher from the makers of Elasticsearch, but it is not open source and after having failed to obtain a price quote over e-mail I gave up and started looking elsewhere.
In a subsequent article I will show how to use the Elastalert Docker image with the ELK Docker image I created in an earlier article.

Dockerfile

The Dockerfile looks like this:

# Elastalert Docker image running on Ubuntu 15.04.
# Build image with: docker build -t ivankrizsan/elastalert:v1 .

FROM ubuntu:15.04

MAINTAINER Ivan Krizsan, https://github.com/krizsan

# URL from which to download Elastalert.
ENV ELASTALERT_URL https://github.com/Yelp/elastalert/archive/v0.0.63.zip
# Directory holding configuration for Elastalert and Supervisor.
ENV CONFIG_DIR /opt/config
# Elastalert rules directory.
ENV RULES_DIRECTORY /opt/rules
# Elastalert configuration file path in configuration directory.
ENV ELASTALERT_CONFIG ${CONFIG_DIR}/elastalert_config.yaml
# Directory to which Elastalert and Supervisor logs are written.
ENV LOG_DIR /opt/logs
# Elastalert home directory name.
ENV ELASTALERT_DIRECTORY_NAME elastalert
# Elastalert home directory full path.
ENV ELASTALERT_HOME /opt/${ELASTALERT_DIRECTORY_NAME}
# Supervisor configuration file for Elastalert.
ENV ELASTALERT_SUPERVISOR_CONF ${CONFIG_DIR}/elastalert_supervisord.conf
# Alias, DNS or IP of Elasticsearch host to be queried by Elastalert. Set in default Elasticsearch configuration file.
ENV ELASTICSEARCH_HOST elasticsearch_host
# Port on above Elasticsearch host. Set in default Elasticsearch configuration file.
ENV ELASTICSEARCH_PORT 9200

WORKDIR /opt

# Copy the script used to launch the Elastalert when a container is started.
COPY ./start-elastalert.sh /opt/

# Install software required for Elastalert.
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y wget python python-dev unzip gcc && \
    rm -rf /var/lib/apt/lists/* && \
    apt-get clean && \
# Install pip - required for installation of Elastalert.
    wget https://bootstrap.pypa.io/get-pip.py && \
    python get-pip.py && \
    rm get-pip.py && \
# Download and unpack Elastalert.
    wget ${ELASTALERT_URL} && \
    unzip *.zip && \
    rm *.zip && \
    mv e* ${ELASTALERT_DIRECTORY_NAME}

WORKDIR ${ELASTALERT_HOME}

# Install Elastalert.
RUN python setup.py install && \
    pip install -e . && \

# Install Supervisor.
    easy_install supervisor && \

# Make the start-script executable.
    chmod +x /opt/start-elastalert.sh && \

# Create directories.
    mkdir ${CONFIG_DIR} && \
    mkdir ${RULES_DIRECTORY} && \
    mkdir ${LOG_DIR} && \

# Copy deafult configuration files to configuration directory.
    cp ${ELASTALERT_HOME}/config.yaml.example ${ELASTALERT_CONFIG} && \
    cp ${ELASTALERT_HOME}/supervisord.conf.example ${ELASTALERT_SUPERVISOR_CONF} && \

# Elastalert configuration:
    # Set the rule directory in the Elastalert config file to external rules directory.
    sed -i -e"s|rules_folder: [[:print:]]*|rules_folder: ${RULES_DIRECTORY}|g" ${ELASTALERT_CONFIG} && \
    # Set the Elasticsearch host that Elastalert is to query.
    sed -i -e"s|es_host: [[:print:]]*|es_host: ${ELASTICSEARCH_HOST}|g" ${ELASTALERT_CONFIG} && \
    # Set the port used by Elasticsearch at the above address.
    sed -i -e"s|es_port: [0-9]*|es_port: ${ELASTICSEARCH_PORT}|g" ${ELASTALERT_CONFIG} && \

# Elastalert Supervisor configuration:
    # Redirect Supervisor log output to a file in the designated logs directory.
    sed -i -e"s|logfile=.*log|logfile=${LOG_DIR}/elastalert_supervisord.log|g" ${ELASTALERT_SUPERVISOR_CONF} && \
    # Redirect Supervisor stderr output to a file in the designated logs directory.
    sed -i -e"s|stderr_logfile=.*log|stderr_logfile=${LOG_DIR}/elastalert_stderr.log|g" ${ELASTALERT_SUPERVISOR_CONF} && \
    # Modify the start-command.
    sed -i -e"s|python elastalert.py|python -m elastalert.elastalert --config ${ELASTALERT_CONFIG}|g" ${ELASTALERT_SUPERVISOR_CONF} && \

# Copy the Elastalert configuration file to Elastalert home directory to be used when creating index first time an Elastalert container is launched.
    cp ${ELASTALERT_CONFIG} ${ELASTALERT_HOME}/config.yaml && \

# Add Elastalert to Supervisord.
    supervisord -c ${ELASTALERT_SUPERVISOR_CONF}

# Define mount points.
VOLUME [ "${CONFIG_DIR}", "${RULES_DIRECTORY}", "${LOG_DIR}"]

# Launch Elastalert when a container is started.
CMD ["/opt/start-elastalert.sh"]

Some noteworthy things are:

• Version 0.0.63 of Elastalert is used.
  At the time of writing, this was the most current tag.
• When using the Elastalert Docker image, you have to create a link to the Docker container in which the Elasticsearch instance that Elastalert is to query for data.
  The port is assumed to be 9200.
• After having installed Elastalert, there is a command “pip install -e .”.
  I had some troubles when installing Elastalert and the Elastalert package did not get installed. This command fixed this (big thanks to Qmando for helping me out with this one!). If everything already has bee installed correctly, this will be detected and nothing will happen when this command is executed.
• Supervisor is installed.
  Supervisor is used to start Elastalert in a Docker container and to redirect log output to files.
• The Elastalert and Supervisor configuration files are taken from the Elastalert download and modified slightly.

I originally wanted to create a “elastalert” user in the Docker image and run Elastalert using this user but ran into some problems that caused me to postpone this and settle for letting the root user run Elastalert.

Elastalert Start Script

The Elastalert start script were a little tricky, since Elastalert require one-time initialization and also require that the Elasticsearch instance it is to query to be online.

#!/bin/sh

set -e

# Wait until Elasticsearch is online since otherwise Elastalert will fail.
rm -f garbage_file
while ! wget -O garbage_file ${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT} 2>/dev/null
do
	echo "Waiting for Elasticsearch..."
	rm -f garbage_file
	sleep 1
done
rm -f garbage_file
sleep 5

# Check if the Elastalert index exists in Elasticsearch and create it if it does not.
if ! wget -O garbage_file ${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}/elastalert_status 2>/dev/null
then
	echo "Creating Elastalert index in Elasticsearch..."
    elastalert-create-index --index elastalert_status --old-index ""
else
    echo "Elastalert index already exists in Elasticsearch."
fi
rm -f garbage_file

echo "Starting Elastalert..."
exec supervisord -c ${ELASTALERT_SUPERVISOR_CONF} -n

Checking whether the Elasticsearch instance is online is done with the wget command. The while-loop will wait indefinitely until something answers at the Elasticsearch port and address. I did try the technique employed by docker-wait, but the script hang once the Elasticsearch instance came online.

The wget command is also used to determine if an index with the name “elastalert_status” exists in Elasticsearch. If it does not, the index is created.

Having created these two files and a readme-file, I created a new GitHub project.

Creating an Automated Build on DockerHub

First and foremost: There is no need whatsoever to push an image to DockerHub if you want to create an automated build, in which you display the Dockerfile of your image.
The following steps describes how to create an automated build on DockerHub that will build a new version of your Docker image whenever there is a change in the repository in which the source-files are stored.

• Log in to DockerHub.
• In the Create menu, select Create Automated Build.

• If you haven’t already, you need to link your GitHub account to DockerHub.
  Do this in the Link Accounts menu.
• Select the repository containing your Dockerfile.
  The following screen will appear:

• Now is the last chance to name your Docker image.
  If I were to leave the parameters in the above picture unchanged, my Docker image would be named ivankrizsan/elastalert-docker. I did not want that, so I changed “elastalert-docker” to plain “elastalert”.
• Enter a short description of your Docker image.
  The contents of any readme-file from your GitHub repository will be copied into the long description of the Docker image.
• Configure additional tags for your Docker image.
  These can be for, for instance, different version of the program that is to be run in Docker containers created from the image.
• Decide whether the image is to be public or private.
• Decide whether or not new pushes to the repository are to trigger automatic builds of the image.
• Click the Create button.

Since I have already committed all the files to my GitHub repository, I will need to trigger the first build manually:

• In DockerHub, go to the Docker image.
• Select the Build Settings tab.

• Click the Trigger a Build button.
• Wait.

The Docker image is now alive and kickin’ in DockerHub!