Monitoring services and applications in just a few clicks – that’s the dream of every SRE and developer, but this is very difficult because most applications don’t expose metrics in a standardized format.
This article will introduce the current pros and cons of the Prometheus exporters ecosystem and how we leveraged the power of the open-source exporters in Sysdig to radically simplify the user experience to allow you to monitor your applications in just a few clicks.
Finding the right exporter
First of all, why use Prometheus exporters? Because we love open source! Embracing the Prometheus ecosystem removes the vendor lock-in that other monitoring platforms have with their custom scripts and solutions. After all, you can always use your own Prometheus to get metrics from the exporters.
Some services are instrumented natively, which means they expose Prometheus metrics directly. But for the vast majority of applications, we need to rely on third-party, open-source exporters to produce metrics. These exporters transform whatever metric format the service uses into Prometheus metrics. This situation has created a wide and diverse ecosystem of applications and Prometheus exporters in which finding and correctly configuring the right exporter for a given application is not straightforward. For example, it’s possible that for a given application more than one exporter (or forks of the same exporter) exists which may or may not be maintained.
Last year we created PromCat.io to offer solutions to these problems to the open-source community. We use PromCat.io to test and select the best exporter for every supported application, contributing to the open-source exporters when needed and committing back to the Prometheus ecosystem.
Now it’s time to simplify things even further.
Automatically discovering and managing your Prometheus monitoring integrations
At Sysdig, we think that the main entity in Kubernetes is the workload. They can be either Deployment, DaemonSet, StatefulSet, or even DeploymentConfig in OpenShift. With this in mind, we show you all the workloads, ordered by clusters and namespaces, and identify which kind of application they are.
This way, you can manage Prometheus monitoring integrations more easily. At a glance you can see which workloads need attention, which are reporting metrics correctly, and install exporters for some or all of the workloads easily.
Auto discovery in other monitoring solutions can find out if you have a service running, but it doesn’t allow you to configure metric collection on a per instance basis. Sysdig Monitor approach identifies every instance of every workload, even if they are in the same host, and without requiring host access. This creates a great experience not only for the first user that is setting up a monitoring integration, but for everyone that comes behind as well, making every user self-sufficient even if they only have access to their namespace, and not the entire cluster, the host or the monitoring agent.
Guided configuration for exporters
We know that one of the hardest steps of deploying an exporter is configuring it to connect to your application and generate metrics. This is one of the most exciting improvements that we have created for you.
You don’t need to dive into the documentation and code of a given exporter to guess the name of the variable to set up the user and password of your database, or how to configure the SSL certificate in the connection string. We do it for you!
In every application, you will find a wizard-like form that will ask you all the needed information so it can provide a one-line command that will set up the exporter in your cluster.
Curated and enterprise-grade images
Like most open-source projects, Prometheus exporters are normally maintained by the community and its images are hosted in free image repositories, where pull restrictions, volatility, or lack of control can affect your production environment.
At Sysdig, we own our image repository where we host the images and secure them. All the exporter images that we use pass Sysdig Secure image scanning to detect possible threats. If threats are found, we fix them.
If you need images with special requirements, like Universal Based Images, we also provide a version ready for hardened environments.
Universal Based Images (UBI) are OCI-compliant, freely redistributable, container base operating system images that include complimentary runtime languages and packages.
Also, the compatibility of the exporter with different versions of the application is not straightforward to find and verify. Don’t worry, though, we got you covered. Our engineering team has verified the compatibility of the chosen exporter with different versions of the application, and we offer to our customers a compatibility matrix of exporter vs. application versions that you can rely on.
Direct integration in your CI/CD workflow
Once the exporter is ready to deploy, we offer you different options. First, you can quickly install it with a one-line command. Superfast!.
If there are hard controls over your production clusters and you deploy through a pipeline and you cannot run commands directly in production, we also offer the option to generate the manifests to upload to your repository. Ready to commit and push into your CI/CD.
Do you need more options? The wizard that we offer is flexible enough for the vast majority of the users, but we know that environments are diverse and you can have special needs.
For customers optionsthat prefer a package management approach, we have created Helm charts for the Prometheus monitoring integrations that are easy to use and well documented, so you can customize all the possible options.
Is it working?
So, you deployed the exporter. Now what? How do you know that the exporter is working?
Sysdig Monitor automatically detects the metrics arriving at your account and associates them to the workload. This way, it is easy to visually detect the applications that are correctly reporting metrics and those that need some attention.
Image of integration successfully reporting metrics
Ready to use dashboards and alerts
You have metrics now in your account, let’s make the most of them! Normally, now you’ll have to create a set of alerts and dashboards to start monitoring your applications, but we’ve already done this for you.
We have created special out-of-the-box dashboards for each integration. These dashboards are made using the monitoring best practices (Golden Signals, USE, or RED frameworks) and they are focused on use cases and usability. To help you make the best of them, we also included some information and troubleshooting tips next to the visualization panels.
You can go directly to the dashboards from each of the integration workloads. They’ll automatically be scoped to the relevant cluster, namespace, and workload.
Here, we’ve added more magic under the hood. Even if the metrics are generated by a Prometheus exporter, that will be transparent for you and your team. The metrics will be associated directly with the namespace and workload of the application that you are monitoring. You don’t need to remember every time which exporter is generating the metrics of what application, for the metrics will seem to come directly from your Redis, PostgreSQL, ElasticSearch, etc. This is part of what we call ServiceVision on PromQL, our extended version that simplifies querying.
When Sysdig Monitor starts receiving metrics for a configured integration, you can find all the available dashboards for that integration in the Dashboard Templates section.
Along with the dashboards, we also automatically create a curated set of alert templates for your applications in our Alerts Library.
This set of alerts help you to easily start monitoring your applications following the best practices. Also, it lets you customize the alerts for different workloads to different levels and notification channels. You can find them in the Alerts section.
Start now
The Prometheus open-source ecosystem is wide and diverse. We believe our approach will save you a lot of time and effort while avoiding vendor lock-in with our open standards approach and professional support. Sysdig has developed integrations using the best Prometheus exporters for your applications to give you the ability to easily configure and use them. This reduces time and complexity by having ready-to-use metrics, dashboards, and alerts in your account so you can focus on what matters most.
Sounds too good to be true? Take a few minutes and try it yourself. Monitoring your services is now just a few clicks away, just sign-up for a free trial of Sysdig Monitor.