Configuring Network Encryption in GlusterFS

Encryption is the process of converting data into a cryptic format, or code when it is transmitted on a network. Encryption prevents unauthorized use of the data.

GlusterFS supports network encryption using TLS/SSL. GlusterFS uses TLS/SSL for authentication and authorization, in place of the home grown authentication framework used for normal connections. GlusterFS supports the following encryption types:

  • I/O encryption - encryption of the I/O connections between the GlusterFS clients and servers

  • Management encryption - encryption of the management (glusterd) connections within a trusted storage pool.

Important

Currently, management encryption using SSL on GlusterFS is supported only on FUSE and gNFS mounts. Enabling management encryption using SSL on GlusterFS will cause NFS Ganesha and Samba mount failure.

The following files will be used in configuring the network encryption:

  • /etc/ssl/glusterfs.pem - Certificate file containing the system’s uniquely signed TLS certificate. This file is unique for each system and must not be shared with others.

  • /etc/ssl/glusterfs.key - This file contains the system’s unique private key. This file must not be shared with others.

  • `/etc/ssl/glusterfs.ca `- This file contains the certificates of the Certificate Authorities (CA) who have signed the certificates. This file is not unique and should be the same on all servers in the trusted storage pool. All the clients also should have the same file, but not necessarily the same one as the servers. GlusterFS does not use the global CA certificates that come with the system. The CA file on the servers should contain the certificates of the signing CA for all the servers and all the clients.

    The CA file on the clients must contain the certificates of the signing CA for all the servers. In case self-signed certificates are being used, the CA file for the servers is a concatenation of the certificate files `/etc/ssl/glusterfs.pem `of every server and every client. The client CA file is a concatenation of the certificate files of every server.

  • /var/lib/glusterd/secure-access - This file enables encryption on the management (glusterd) connections between glusterd of all servers and the connection between clients. glusterd of all servers uses this file to fetch volfiles and notify the clients with the volfile changes. This file is empty and mandatory only if you configure management encryption. It must be present on all the servers and all the clients. This is required on the clients to indicate the mount command to use an encrypted connection to retrieve the volfiles.

Prerequisites

Before setting up the network encryption, you must first generate a private key and a signed certificate for each system and place it in the respective folders. You must generate a private key and a signed certificate for both clients and servers.

Perform the following to generate a private key and a signed certificate for both clients and servers:

  1. Generate a private key for each system.

    # openssl genrsa -out /etc/ssl/glusterfs.key 2048

  2. Use the generated private key to create a signed certificate by running the following command:

    # openssl req -new -x509 -key /etc/ssl/glusterfs.key -subj "/CN=COMMONNAME" -out /etc/ssl/glusterfs.pem

    If your organization has a common CA, the certificate can be signed by it. To do this a certificate signing request (CSR) must be generated by running the following command:

    # openssl req -new -sha256 -key /etc/ssl/glusterfs.key -subj '/CN=<COMMONNAME>' -out glusterfs.csr

    The common name in this command can be a hostname / FQDN / IP address, et cetera. The generated glusterfs.csr file should be given to the CA, and CA will provide a .pem `file containing the signed certificate. Place that signed `glusterfs.pem file in the`/etc/ssl/` directory.

  3. 1. For self signed CA certificates on servers, collect the .pem certificates of clients and servers, that is, /etc/ssl/glusterfs.pem files from every system. Concatenate the collected files into a single file. Place this file in /etc/ssl/glusterfs.ca on all the servers in the trusted storage pool. If you are using common CA, collect the certificate file from the CA and place it in /etc/ssl/glusterfs.ca on all servers.

  4. For self-signed CA certificates on clients, collect the .pem certificates of servers, that is,` /etc/ssl/glusterfs.pem` files from every server. Concatenate the collected files into a single file. Place this file in /etc/ssl/glusterfs.ca on all the clients. If you are using common CA, collect the certificate file from the CA and place it in` /etc/ssl/glusterfs.ca` on all servers.

Configuring Network Encryption for a New Trusted Storage Pool

You can configure network encryption for a new GlusterFS Trusted Storage Pool for both I/O encryption and management encryption. This section assumes that you have installed GlusterFS on the servers and the clients, but has never been run.

Enabling Management Encryption

Though GlusterFS can be configured only for I/O encryption without using management encryption, it is recommended to have management encryption. If you want to enable SSL only on the I/O path, skip this section and proceed with Enabling I/O encryption for a Volume.

On Servers.

Perform the following on all the servers

  1. Create the` /var/lib/glusterd/secure-access` file.

    # touch /var/lib/glusterd/secure-access

  2. Start glusterd on all servers.

    # service glusterd start

  3. Setup the trusted storage pool by running appropriate peer probe commands. For more information on setting up the trusted storage pool, see Trusted Storage Pools.

On Clients.

Perform the following on all the client machines

  1. Create the` /var/lib/glusterd/secure-access` file.

    # touch /var/lib/glusterd/secure-access

  2. Mount the volume on all the clients. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

Enabling I/O encryption for a Volume

Enable the I/O encryption between the servers and clients:

  1. Create the volume, but do not start it.

  2. Set the list of common names of all the servers to access the volume. Be sure to include the common names of clients which will be allowed to access the volume..

    # gluster volume set VOLNAME auth.ssl-allow 'server1,server2,server3,client1,client2,client3'

    Note

    If you set auth.ssl-allow option with as value, any TLS authenticated clients can mount and access the volume from the application side. Hence, you set the option’s value to or provide common names of clients as well as the nodes in the trusted storage pool.

  3. Enable the client.ssl and server.ssl options on the volume.

    # gluster volume set VOLNAME client.ssl on
# gluster volume set VOLNAME server.ssl on

  4. Start the volume.

    # gluster volume start VOLNAME

  5. Mount the volume on all the clients which has been authorized. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

Configuring Network Encryption for an existing Trusted Storage Pool

You can configure network encryption for an existing GlusterFS Trusted Storage Pool for both I/O encryption and management encryption.

Enabling I/O encryption for a Volume

Enable the I/O encryption between the servers and clients:

  1. Unmount the volume on all the clients.

    # umount mount-point

  2. Stop the volume.

    # gluster volume stop VOLNAME

  3. Set the list of common names for clients allowedto access the volume. Be sure to include the common names of all the servers.

    # gluster volume set VOLNAME auth.ssl-allow 'server1,server2,server3,client1,client2,client3'

    Note

    If you set auth.ssl-allow option with as value, any TLS authenticated clients can mount and access the volume from the application side. Hence, you set the option’s value to or provide common names of clients as well as the nodes in the trusted storage pool.

  4. Enable client.ssl and server.ssl on the volume.

    # gluster volume set VOLNAME client.ssl on
# gluster volume set VOLNAME server.ssl on

  5. Start the volume.

    # gluster volume start VOLNAME

  6. Mount the volume from the new clients. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

Enabling Management Encryption

Though, GlusterFS can be configured only for I/O encryption without using management encryption, management encryption is recommended. On an existing installation, with running servers and clients, schedule a downtime of volumes, applications, clients, and other end-users to enable management encryption.

You cannot currently change between unencrypted and encrypted connections dynamically. Bricks and other local services on the servers and clients do not receive notifications from glusterd if they are running when the switch to management encryption is made.

  1. Unmount the volume on all the clients.

    # umount mount-point

  2. Stop all the volumes.

    # gluster volume stop VOLNAME

  3. Stop glusterd on all servers.

    # service glusterd stop

  4. Stop all gluster-related processes on all servers.

    # pkill glusterfs

  5. Create the /var/lib/glusterd/secure-access file on all servers and clients.

    # touch /var/lib/glusterd/secure-access

  6. Start glusterd on all the servers.

    # service glusterd start

  7. Start all the volumes

    # gluster volume start VOLNAME

  8. Mount the volume on all the clients. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

Expanding Volumes

In a network encrypted GlusterFS trusted storage pool, you must ensure that you meet the prerequisites listed at Prerequisites.

Certificate Signed with a Common Certificate Authority

Adding a server to a storage pool is simple if the servers all use a common Certificate Authority.

  1. Copy /etc/ssl/glusterfs.ca file from one of the existing servers and save it on the`/etc/ssl/` directory on the new server.

  2. If you are using management encryption, create /var/lib/glusterd/secure-access file.

    # touch /var/lib/glusterd/secure-access

  3. Start glusterd on the new peer

    # service glusterd start

  4. Add the common name of the new server to the auth.ssl-allow list for all volumes which have encryption enabled.

    # gluster volume set VOLNAME auth.ssl-allow servernew

    Note

    The gluster volume set command does not append to existing values of the options. To append the new name to the list, get the existing list using gluster volume info command, append the new name to the list and set the option again using gluster volume set command.

  5. Run gluster peer probe [server] to add additional servers to the trusted storage pool. For more information on adding servers to the trusted storage pool, see Trusted Storage Pools.

Self-signed Certificates

Using self-signed certificates would require a downtime of servers to add a new server into the trusted storage pool, as the CA list cannot be dynamically reloaded. To add a new server:

  1. Generate the private key and self-signed certificate on the new server using the steps listed at Prerequisites.

  2. Copy the following files:

  3. On an existing server, copy the /etc/ssl/glusterfs.ca file, append the content of new server’s certificate to it, and distribute it to all servers, including the new server.

  4. On an existing client, copy the /etc/ssl/glusterfs.ca file, append the content of the new server’s certificate to it, and distribute it to all clients.

  5. Stop all gluster-related processes on all servers.

    # pkill glusterfs

  6. Create the /var/lib/glusterd/secure-access file on the server if management encryption is enable in the trusted storage pool.

  7. Start glusterd on the new peer

    # service glusterd start

  8. Add the common name of the new server to the auth.ssl-allow list for all volumes which have encryption enabled.

    Note

    If you set auth.ssl-allow option with as value, any TLS authenticated clients can mount and access the volume from the application side. Hence, you set the option’s value to or provide common names of clients as well as the nodes in the trusted storage pool.

  9. Restart all the glusterfs processes on existing servers and clients by performing the following .

  10. Unmount the volume on all the clients.

    # umount mount-point

  11. Stop all volumes.

    # gluster volume stop VOLNAME

  12. Restart glusterd on all the servers.

    # service glusterd start

  13. Start the volumes

    # gluster volume start VOLNAME

  14. Mount the volume on all the clients. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

  15. Peer probe the new server to add it to the trusted storage pool. For more information on peer probe, see <<../architecture/chap-Trusted_Storage_Pools.adoc#chap-Trusted_Storage_Pools, Trusted Storage Pools>.

Authorizing a New Client

If your GlusterFS trusted storage pool is configured for network encryption, and you add a new client, you must ensure to authorize a new client to access the trusted storage pool.

Certificate Signed with a Common Certificate Authority

Authorizing access to a volume for a new client is simple if the client has a certificate signed by a Certificate Authority already present in the /etc/ssl/glusterfs.ca file.

  1. Generate the glusterfs.key private key and glusterfs.csr certificate signing request. Send the glusterfs.csr to get it verified by CA and get the glusterfs.pem from the CA. Generate the private key and signed certificate for the new server and place the files in the appropriate locations using the steps listed at Prerequisites.

  2. Copy ` /etc/ssl/glusterfs.ca` file from another client and place it in the `/etc/ssl/ `directory on the new client..

  3. Create /var/lib/glusterd/secure-access file if management encryption is enabled in the trusted storage pool.

    # touch /var/lib/glusterd/secure-access

  4. Set the list of common names of all the servers to access the volume. Be sure to include the common names of clients which will be allowed to access the volume.

    # gluster volume set VOLNAME auth.ssl-allow 'server1,server2,server3,client1,client2,client3'

    Note

    The gluster volume set command does not append to existing values of the options. To append the new name to the list, get the existing list using gluster volume info command, append the new name to the list and set the option again using gluster volume set command.

  5. Mount the volume from the new client. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

Self-signed Certificates

Note

This procedure involves downtime as the volume has to be rendered offline.

To authorize a new client to access the GlusterFS trusted storage pool using self-signed certificate, perform the following.

  1. Generate the glusterfs.key private key and glusterfs.pem certificate for the client, and place them at the appropriate locations on the client using the steps listed at Prerequisites.

  2. Copy /etc/ssl/glusterfs.ca file from one of the clients, and add it to the new client.

  3. Create the /var/lib/glusterd/secure-access file on all the client, if the management encryption is enabled.

    # touch /var/lib/glusterd/secure-access

  4. Copy ` /etc/ssl/glusterfs.ca` file from one of the existing servers, append the content of new client’s certificate to it, and distribute the new CA file on all servers.

  5. Set the list of common names for clients allowed to access the volume. Be sure to include the common names of all the servers.

    # gluster volume set VOLNAME auth.ssl-allow 'server1,server2,server3,client1,client2,client3'

    Note

    The gluster volume set command does not append to existing values of the options. To append the new name to the list, get the existing list using gluster volume info command, append the new name to the list and set the option again using gluster volume set command.

    If you set auth.ssl-allow option with as value, any TLS authenticated clients can mount and access the volume from the application side. Hence, you set the option’s value to or provide common names of clients as well as the nodes in the trusted storage pool.

  6. Restart the volume

    # gluster volume stop VOLNAME # gluster volume start VOLNAME

  7. If the management encryption is enabled, restart glusterd on all the servers.

  8. Mount the volume from the new client. For example, to manually mount a volume and access data using Native client, use the following command:

    # mount -t glusterfs server1:/test-volume /mnt/glusterfs

results matching ""

    No results matching ""