Spring Boot Prometheus Disk-Space Metrics

By | January 23, 2021

In this article I will show how I added disk-space metrics to the set of default metrics made available for scraping by Prometheus. There are a few ways to accomplish this but I have chosen to use data from the Spring Boot Actuator health data.

A complete Spring Boot application with disk-space metrics is available in this GitHub repository.

Prerequisites

The example in this article can be implemented in a Spring Boot application of a fairly recent version. I have used Spring Boot version 2.4.1 in the example of this article.

Added Dependencies

The following two dependencies are required in order to have disk-space metrics in a Spring Boot application:

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>

The Spring Boot Actuator starter is required in order to have Spring Boot Actuator in the application, regardless of whether there will be disk-space metrics or not.
The Micrometer Registry Prometheus dependency is required if you want to make Spring Boot Actuator metrics available for scraping by Prometheus.

Management Application Properties

Before being able to view any metrics, the following two properties need to be added to the application.properties file:

management.endpoints.enabled-by-default=true
management.endpoints.web.exposure.include=*

The first property configures Spring Boot Actuator as to enable all endpoints as default. The default behaviour is to have all endpoints disabled as default and then specify the endpoints that are to be enabled. This is of course the preferred way to do it in a production environment.
The second property specify which Actuator endpoints to expose as web endpoints. The above configuration will expose all endpoints as web endpoints.

Prometheus Endpoint

With the above dependencies and configuration added, the Spring Boot application can now be started and we should be able to view metrics exposed in a format that can be scraped by Prometheus.

# HELP system_cpu_count The number of processors available to the Java virtual machine
# TYPE system_cpu_count gauge
system_cpu_count 8.0
# HELP jvm_gc_live_data_size_bytes Size of long-lived heap memory pool after reclamation
# TYPE jvm_gc_live_data_size_bytes gauge
jvm_gc_live_data_size_bytes 0.0
# HELP jdbc_connections_active Current number of active connections that have been allocated from the data source.
# TYPE jdbc_connections_active gauge
jdbc_connections_active{name="dataSource",} 0.0
# HELP hikaricp_connections_min Min connections
# TYPE hikaricp_connections_min gauge
hikaricp_connections_min{pool="HikariPool-1",} 10.0
# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool
# TYPE jvm_buffer_count_buffers gauge
jvm_buffer_count_buffers{id="mapped",} 0.0
jvm_buffer_count_buffers{id="direct",} 4.0
# HELP hikaricp_connections_idle Idle connections
# TYPE hikaricp_connections_idle gauge
hikaricp_connections_idle{pool="HikariPool-1",} 10.0
…

There will be a lot more metrics but the above should give you an idea of what it should look like. The above shows that Spring Boot Actuator metrics have been successfully exposed on the Prometheus web endpoint.

Disk-Space Metrics Configuration

To add custom metrics to the set of metrics that are exposed on the Prometheus endpoint, a MeterRegistryCustomizer bean is created and in the bean-creation method, three disk-space metrics gauges are created and registered. The Spring configuration class that does that looks like this:

package se.ivankrizsan.restexample;

import io.micrometer.core.instrument.Gauge;
import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryCustomizer;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.io.File;

/**
 * Configuration of bean which expose disk-space metrics that can be scraped by Prometheus.
 * Enable or disable the disk-space metrics using the property "diskspace.metrics.enabled".
 * Configure the path for which disk-space metrics are to be calculated using the property
 * "diskspace.metrics.path".
 *
 * @author Ivan Krizsan
 */
@Configuration
@ConditionalOnProperty(name = "diskspace.metrics.enabled", havingValue = "true")
public class DiskSpaceMetricsConfiguration {
    /* Constant(s): */
    public final static String FREE_DISKSPACE_GAUGE_NAME = "free_disk_space";
    public final static String TOTAL_DISKSPACE_GAUGE_NAME = "total_disk_space";
    public final static String USED_DISKSPACE_PERCENTAGE_GAUGE_NAME = "used_disk_space";
    public final static String REMAINING_DISKSPACE_PERCENTAGE_GAUGE_NAME ="remaining_disk_space";

    /* Dependencies: */
    @Value("${diskspace.metrics.path}")
    protected String mDiskspaceMetricsPath;

    /**
     * Meter registry customizer bean that adds disk-space gauges to the set of metrics that
     * can be scraped by Prometheus.
     *
     * @return Meter registry customizer for disk space metrics.
     */
    @Bean
    MeterRegistryCustomizer<MeterRegistry> meterRegistryCustomizer() {
        final File theDiskSpaceMetricsFile = new File(mDiskspaceMetricsPath);

        return registry -> {
            Gauge
                .builder(FREE_DISKSPACE_GAUGE_NAME, theDiskSpaceMetricsFile::getFreeSpace)
                .description("Free disk-space in bytes")
                .baseUnit("bytes")
                .register(registry);

            Gauge
                .builder(TOTAL_DISKSPACE_GAUGE_NAME, theDiskSpaceMetricsFile::getTotalSpace)
                .description("Total disk-space in bytes")
                .baseUnit("bytes")
                .register(registry);

            Gauge
                .builder(USED_DISKSPACE_PERCENTAGE_GAUGE_NAME, () -> {
                    final double theFreeDiskSpace = theDiskSpaceMetricsFile.getFreeSpace();
                    final double theTotalDiskSpace = theDiskSpaceMetricsFile.getTotalSpace();
                    final double theUsedDiskSpace = theTotalDiskSpace - theFreeDiskSpace;
                    return (theUsedDiskSpace / theTotalDiskSpace) * 100.0;
                })
                .description("Used disk-space in percent")
                .baseUnit("percent")
                .register(registry);

            Gauge
                .builder(REMAINING_DISKSPACE_PERCENTAGE_GAUGE_NAME, () -> {
                    final double theFreeDiskSpace = theDiskSpaceMetricsFile.getFreeSpace();
                    final double theTotalDiskSpace = theDiskSpaceMetricsFile.getTotalSpace();
                    return (theFreeDiskSpace / theTotalDiskSpace) * 100.0;
                })
                .description("Remaining disk-space in percent")
                .baseUnit("percent")
                .register(registry);
        };
    }
}

Note that:

  • There is a bean named meterRegistryCustomizer.
    The bean is a MeterRegistryCustomizer that, when invoked, creates and registers three disk-space gauges.
  • When creating the disk-space gauges, free and total disk-space are retrieved inside the lambda sent as a parameter to the Gauge.builder method.
    This ensures that data is retrieved every time the gauge is read.
  • A base unit is set for each gauge.
    The unit will be appended to the name of the gauge, as we will shortly see when retrieving the custom metric data.

Disk-Space Metrics Application Properties

As the astute reader have noticed, there are two application property values injected into the above configuration. The first one is “diskspace.metrics.enabled” which, as the name suggests, allows for enabling or disabling disk-space metrics. The second property is “diskspace.metrics.path” which configures the path for which disk-space metrics will be calculated.
These two properties need to be added to the application property file, after which it will look like this in my example application:

spring.jpa.hibernate.use-new-id-generator-mappings=true
spring.main.allow-bean-definition-overriding=true
management.endpoints.enabled-by-default=true
management.endpoints.web.exposure.include=*

logging.level.root=INFO

# Enable logging of requests, including request headers, on the server-side.
logging.level.org.springframework.web.server.adapter.HttpWebHandlerAdapter=TRACE
logging.level.org.springframework.web=DEBUG
spring.mvc.log-request-details=true
spring.codec.log-request-details=true

# Configures disk path for which disk-space metrics are to be calculated.
diskspace.metrics.path=/
# Enable or disable the disk-space metrics.
diskspace.metrics.enabled=true

View Disk-Space Metrics

Having added the disk-space metrics configuration class and the application properties, the new metrics should appear when sending a request to the Prometheus endpoint.

  • Start, or if it is already started, restart the application.
    As before, the application can be run in an IDE or standalone.
  • View the http://localhost:8080/actuator/prometheus URL in a browser.
  • Search for “disk-space” in the response of the above request.
    The following information, not necessarily adjacent, have been added to the metrics.
    Note that the values depend on your system and will not be the same as in this example.
# HELP remaining_disk_space_percent Remaining disk-space in percent
# TYPE remaining_disk_space_percent gauge
remaining_disk_space_percent 40.60552111974506
# HELP free_disk_space_bytes Free disk-space in bytes
# TYPE free_disk_space_bytes gauge
free_disk_space_bytes 9.7127452672E10
# HELP used_disk_space_percent Used disk-space in percent
# TYPE used_disk_space_percent gauge
used_disk_space_percent 59.39447888025493
# HELP total_disk_space_bytes Total disk-space in bytes
# TYPE total_disk_space_bytes gauge
total_disk_space_bytes 2.39197650944E11

The custom disk-space metrics have been successfully added. In the next article, I will show how to create a low disk-space alarm using these disk-space metrics. Until then, happy coding!

Leave a Reply

Your email address will not be published. Required fields are marked *