Creating an Elastalert Docker Image on Docker Hub

By | October 19, 2015

In this article I will show how I created a Docker image for Elastalert and create an automated build for the image on Docker Hub.
Among other things, I will show how to wait on other services that a service running in a Docker container depends on.

Introduction

For those who are not familiar with Elastalert I can briefly mention that it is a piece of software that can monitor data in an Elasticsearch instance and generate alerts upon detecting a variety of conditions. I were originally looking at Watcher from the makers of Elasticsearch, but it is not open source and after having failed to obtain a price quote over e-mail I gave up and started looking elsewhere.
In a subsequent article I will show how to use the Elastalert Docker image with the ELK Docker image I created in an earlier article.

Dockerfile

The Dockerfile looks like this:

Some noteworthy things are:

  • Version 0.0.63 of Elastalert is used.
    At the time of writing, this was the most current tag.
  • When using the Elastalert Docker image, you have to create a link to the Docker container in which the Elasticsearch instance that Elastalert is to query for data.
    The port is assumed to be 9200.
  • After having installed Elastalert, there is a command “pip install -e .”.
    I had some troubles when installing Elastalert and the Elastalert package did not get installed. This command fixed this (big thanks to Qmando for helping me out with this one!). If everything already has bee installed correctly, this will be detected and nothing will happen when this command is executed.
  • Supervisor is installed.
    Supervisor is used to start Elastalert in a Docker container and to redirect log output to files.
  • The Elastalert and Supervisor configuration files are taken from the Elastalert download and modified slightly.

I originally wanted to create a “elastalert” user in the Docker image and run Elastalert using this user but ran into some problems that caused me to postpone this and settle for letting the root user run Elastalert.

Elastalert Start Script

The Elastalert start script were a little tricky, since Elastalert require one-time initialization and also require that the Elasticsearch instance it is to query to be online.

Checking whether the Elasticsearch instance is online is done with the wget command. The while-loop will wait indefinitely until something answers at the Elasticsearch port and address. I did try the technique employed by docker-wait, but the script hang once the Elasticsearch instance came online.

The wget command is also used to determine if an index with the name “elastalert_status” exists in Elasticsearch. If it does not, the index is created.

Having created these two files and a readme-file, I created a new GitHub project.

Creating an Automated Build on DockerHub

First and foremost: There is no need whatsoever to push an image to DockerHub if you want to create an automated build, in which you display the Dockerfile of your image.
The following steps describes how to create an automated build on DockerHub that will build a new version of your Docker image whenever there is a change in the repository in which the source-files are stored.

  • Log in to DockerHub.
  • In the Create menu, select Create Automated Build.
Creating a new automated build in DockerHub.

Creating a new automated build in DockerHub.

  • If you haven’t already, you need to link your GitHub account to DockerHub.
    Do this in the Link Accounts menu.
  • Select the repository containing your Dockerfile.
    The following screen will appear:

New Automated Build in DockerHub

  • Now is the last chance to name your Docker image.
    If I were to leave the parameters in the above picture unchanged, my Docker image would be named ivankrizsan/elastalert-docker. I did not want that, so I changed “elastalert-docker” to plain “elastalert”.
  • Enter a short description of your Docker image.
    The contents of any readme-file from your GitHub repository will be copied into the long description of the Docker image.
  • Configure additional tags for your Docker image.
    These can be for, for instance, different version of the program that is to be run in Docker containers created from the image.
  • Decide whether the image is to be public or private.
  • Decide whether or not new pushes to the repository are to trigger automatic builds of the image.
  • Click the Create button.

Since I have already committed all the files to my GitHub repository, I will need to trigger the first build manually:

  • In DockerHub, go to the Docker image.
  • Select the Build Settings tab.
Trigger the first build of the new Docker image in DockerHub.

Trigger the first build of the new Docker image in DockerHub.

  • Click the Trigger a Build button.
  • Wait.

The Docker image is now alive and kickin’ in DockerHub!

 

6 thoughts on “Creating an Elastalert Docker Image on Docker Hub

  1. jennifer

    Hi,
    Thank you very much for sharing it.
    However, I could not get the elastalert docker container running. Here is the error I am getting:

    docker run -d –cap-add=sys_nice -e “SET_CONTAINER_TIMEZONE=true” -it –link elasticsearch_host:sebp/elk –name elastalert ivankrizsan/elastalert

    docker logs d1279492f73a

    /opt/start-elastalert.sh: line 11: setup-timezone: not found
    reset adjtime failed: Operation not permitted
    constraint certificate verification turned off
    adjtimex failed: Operation not permitted
    Elastalert index already exists in Elasticsearch.
    Starting Elastalert…
    2016-04-12 21:33:47,882 CRIT Supervisor running as root (no user in config file)
    Unlinking stale socket /var/run/elastalert_supervisor.sock
    2016-04-12 21:33:48,192 INFO RPC interface ‘supervisor’ initialized
    2016-04-12 21:33:48,192 CRIT Server ‘unix_http_server’ running without any HTTP authentica tion checking
    2016-04-12 21:33:48,192 INFO supervisord started with pid 1
    2016-04-12 21:33:49,194 INFO spawned: ‘elastalert’ with pid 25
    2016-04-12 21:33:49,522 DEBG ‘elastalert’ stderr output:
    ERROR:root:No rules loaded. Exiting
    None

    [edit: removed repeated log messages]

    2016-04-12 21:33:56,427 DEBG fd 9 closed, stopped monitoring <POutputDispatcher at 1397391 81785312 for (stder r)>
    2016-04-12 21:33:56,427 DEBG fd 7 closed, stopped monitoring <POutputDispatcher at 1397391 82263760 for (stdou t)>
    2016-04-12 21:33:56,427 INFO exited: elastalert (exit status 1; not expected)
    2016-04-12 21:33:56,427 DEBG received SIGCHLD indicating a child quit
    2016-04-12 21:33:57,428 INFO gave up: elastalert entered FATAL state, too many start retri es too quickly

    Reply
  2. Sunil Chaudhari

    Hi,
    When I run start script, it says timezone is not modified.
    It goes in the else block all the time, even if I set timzeone to true in the dockerfile.

    Please help on this.

    br.
    Sunil

    Reply
    1. Ivan Krizsan Post author

      Hi Sunil!
      You should set the environment variable when you run the Docker image, like in the following example:
      docker run -e SET_CONTAINER_TIMEZONE=true [additional parameters] ivankrizsan/elastalert
      Happy coding!

      Reply
  3. Serg

    Hi, Ivan
    after I running you image I see error:
    elastalert_1_1 | 2017-08-17 18:23:07,286 DEBG received SIGCLD indicating a child quit
    elastalert_1_1 | 2017-08-17 18:23:08,289 INFO spawned: ‘elastalert’ with pid 13
    elastalert_1_1 | 2017-08-17 18:23:08,311 DEBG ‘elastalert’ stderr output:
    elastalert_1_1 | /usr/bin/python: No module named elastalert

    I use default supervisor.conf (but little modify start up command):
    [unix_http_server]
    file=/var/run/elastalert_supervisor.sock

    [supervisord]
    logfile=/opt/logs/elastalert_supervisord.log
    logfile_maxbytes=1MB
    logfile_backups=2
    loglevel=debug
    nodaemon=false
    directory=%(here)s

    [rpcinterface:supervisor]
    supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

    [supervisorctl]
    serverurl=unix:///var/run/elastalert_supervisor.sock

    [program:elastalert]
    # running globally
    command=python -m elastalert.elastalert –config /opt/config/config.yaml –verbose
    #command =
    # python elastalert.py
    # –verbose
    # (alternative) using virtualenv
    # command=/path/to/venv/bin/elastalert –config /path/to/config.yaml –verbose
    process_name=elastalert
    autorestart=true
    startsecs=15
    stopsignal=INT
    stopasgroup=true
    killasgroup=true
    stderr_logfile=/opt/logs/elastalert_stderr.log
    stderr_logfile_maxbytes=5MB

    Any ideas?

    Reply
    1. Ivan Krizsan Post author

      Hello!
      Sorry, I have not encountered those kind of problems myself. Someone mentioned that my Elastalert image sometimes does not build properly. I suggest that you download the sources and try to build it yourself.
      I am in the process of contributing the Elastalert Docker image to Yelp in order to have an official Elastalert Docker image, so I won’t make any modifications to it.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *