Using custom CA certificates with Netdata
When streaming over an encrypted connection, exporting metrics to a secure endpoint, collecting metrics from secure services, or connecting to an on-premises instance of Netdata Cloud, the Netdata Agent needs to be able to verify the TLS certificate of the remote system to ensure the security of the connection.
When the remote system is using a TLS certificate issued by a public certificate authority, this will work correctly out of box without a need to configure anything extra. However, if the TLS certificate was issued by a private CA, the certificate for that private CA must be installed on the system the Netdata Agent is running on for the connections to succeed.
The exact method of installing a certificate for a private CA depends on the installation type and the underlying platform:
- For native DEB/RPM packages, install the certificate in the system certificate store
- For static builds on Linux, see the instructions for using custom certificates with our static builds
- For our Docker images, see the instructions for using custom certificates with our Docker containers
- For local builds of Netdata on Linux, install the certificate in the system certificate store
- For Windows, see the instructions for installing custom certificates for Netdata
Installing certificates in the system certificate store on Linux
Exact instructions for installing certificates in the system certificate store on Linux vary based on the distribution. If instructions for your Linux distribution are not listed below, consult the documentation for your distribution for instructions.
Debian, Ubuntu, and derivatives
To install a custom CA certificate in the system certificate store on a Debian or Ubuntu system:
- Ensure the certificate file to be installed is in PEM or DER format.
- Copy the certificate file to
/usr/local/share/certificateswith a.crtfile extension (for example, if the certificate file is namedlocal.pem, copy it to this directory aslocal.crt). You may need to create this directory. The certificate file (and the directory) should have permissions set such that all users can read the file, but only the root user can write to it. - Run the command:
sudo update-ca-certificates.
Red Hat Enterprise Linux, Fedora, and derivatives
To install a custom CA certificate in the system certificate store on a Red Hat Enterprise Linux or Fedora system:
- Ensure the certificate file to be installed is in PEM or DER format.
- Copy the certificate file to
/etc/pki/ca-trust/source/anchorswith a.crtfile extension (for example, if the certificate file is namedlocal.pem, copy it to this directory aslocal.crt). The certificate file should have permissions set such that all users can read the file, but only the root user can write to it. - Run the command:
sudo update-ca-trust
Suse Linux Enterprise and openSUSE
To install a custom CA certificate in the system certificate store on a Suse Linux Enterprise or openSUSE system:
- Ensure the certificate file to be installed is in PEM or DER format.
- Copy the certificate file to
/etc/pki/trust/anchorswith a.crtfile extension (for example, if the certificate file is namedlocal.pem, copy it to this directory aslocal.crt). The certificate file should have permissions set such that all users can read the file, but only the root user can write to it. - Run the command:
sudo update-ca-certificates
Arch Linux and derivatives
To install a custom CA certificate in the system certificate store on an Arch Linux system:
- Ensure the certificate file to be installed is in PEM or DER format.
- Copy the certificate file to
/etc/ca-certificates/trust-store/anchorswith a.crtfile extension (for example, if the certificate file is namedlocal.pem, copy it to this directory aslocal.crt). The certificate file should have permissions set such that all users can read the file, but only the root user can write to it. - Run the command:
sudo update-ca-trust
Alpine Linux
To install a custom CA certificate in the system certificate store on an Alpine Linux system:
- Install the
ca-certificatespackage if it is not already installed. - Ensure the certificate file to be installed is in PEM or DER format.
- Copy the certificate file to
/usr/local/share/certificateswith a.crtfile extension (for example, if the certificate file is namedlocal.pem, copy it to this directory aslocal.crt). You may need to create this directory. The certificate file (and the directory) should have permissions set such that all users can read the file, but only the root user can write to it. - Run the command:
sudo update-ca-certificates.
Using custom certificates with our static builds
For most users of our static builds, simply installing the required certificate files in the system trust store as outlined above will be sufficient to get things working correctly, though the certificates should be installed in the system trust store before installing Netdata, otherwise they may not work until after the next time the agent is updated.
If you are using one of our static builds and installing the certificates in the system certificate store does not work, please open a bug report about it on GitHub, as this usually indicates that our static builds are not correctly handling certificates on your system.
Using custom certificates with our Docker images
The simplest way to use custom certificates with our Docker images is to create a custom Docker image that includes the required certificate.
A custom Docker image including the required certificate can be created using a Dockerfile similar to the following:
FROM netdata/netdata:stable
RUN mkdir -p /usr/local/share/certificates
COPY local.pem /usr/local/share/certificates
RUN update-ca-certificates
The COPY line should be updated to reflect the actual name of the certificate file to be included. Note that
the certificate must be in PEM or DER format with a .crt extension.
Using custom certificates on Windows
Currently, Netdata does not provide integration for most components with the system certificate store on Windows. Instead, certificates must be installed into the bundled MSYS2 environment shipped as part of Netdata using the following instructions:
- Ensure the certificate file to be installed is in PEM or DER format, with a
.crtextension. - From an elevated (administrator) shell, create the directory
C:\Program Files\Netdata\etc\pki\ca-trust\source\anchorsif it does not exist, and copy the certificate file into it. - Refresh the certificate store by running
update-ca-trustinside the bundled MSYS2 environment: startC:\Program Files\Netdata\msys2.exeas administrator and runupdate-ca-trust. From an elevated PowerShell session the same can be done with one line:& "C:\Program Files\Netdata\usr\bin\bash.exe" -lc /usr/bin/update-ca-trustThe script has no.exeextension - it is a shell script provided by the MSYS2ca-certificatespackage, and it cannot be run fromcmd.exeor PowerShell directly. - Restart the agent:
Restart-Service Netdata - Check that the store was refreshed: the files under
C:\Program Files\Netdata\etc\pki\ca-trust\extracted\andC:\Program Files\Netdata\usr\ssl\cert.pemshould have fresh timestamps.
Five things to keep in mind:
- After upgrading the agent, run
update-ca-trustagain. The installer resets the trust-store files it owns underC:\Program Files\Netdata\etc\pki; certificates you copied intoca-trust\source\anchorsare left in place, but the derived bundles are rebuilt. go.d.pluginis a native Go binary and uses the Windows certificate store, not the MSYS2 bundle, unless the collector job setstls_ca. If a job setstls_ca, the file it names must contain the corporate CA; otherwise removetls_caso the collector uses the Windows store. Behind a TLS-intercepting proxy, install the corporate CA in the MSYS2 anchors directory and, for jobs withouttls_ca, inLocal Computer>Trusted Root Certification Authorities.- The agent's C components (Netdata Cloud connection, claiming, streaming, exporting) verify
peers against the OpenSSL bundle maintained by
update-ca-trust, not the Windows store. ssl skip certificate verification = yesin the[stream]section ofstream.confdisables certificate verification on the child-to-parent sender, including unknown issuing CAs. To trust a private CA, useCAfileorCApathin the same section instead.INSECURE=1is a Windows installer property for the Netdata Cloud connection - claiming and ACLK - not astream.confoption. It relaxes certificate checks for Cloud only, not for streaming or the collectors, and is not a substitute for installing the certificate.
Do you have any feedback for this page? If so, you can open a new issue on our netdata/learn repository.