Certificates Management Guide¶
Audience: Hyperledger Fabric network admins
This guide provides overview information and details for a network administrator to manage certificates (certs) in Hyperledger Fabric.
Prerequisites and Resources¶
The following Fabric documentation resources on identities, Membership Service Providers (MSPs) and Certificate Authorities (CAs) provide context for understanding certificate management:
Key Concepts¶
Registration – A username and password pair, stored in the Certificate Authority (CA). This registration is created by a CA admin user, has no expiration, and contains any required roles and attributes.
Enrollment – A public/private key pair and an X.509 certificate issued by the organization’s Certificate Authority (CA). The certificate encodes roles, attributes, and metadata, which represent an identity in a Fabric network. An enrollment is associated with a CA registration by username and password.
Identity - A public certificate and its private key used for encryption. The public certificate is the X.509 certificate issued by the CA, while the private key is stored out of band, on a secure storage.
TLS - A public Transport Layer Security (TLS) Certificate that authorizes client and node communications. On Fabric, registration and enrollment are the same for X.509 Certificates and TLS Certificates.
Certificate Types¶
Hyperledger Fabric implements two types of certificates: 1) Enrollment Certificates for identities and 2) TLS Certificates for node and client communications.
Enrollment Certificates¶
Enrollment Certificates are classed into four types:
Admin
Peer
Orderer
Client
Each Enrollment Certificate type has a specific role:
Admin: X.509 Certificates used to authenticate admin identities, which are required to make changes to Fabric configurations.
Peer: X.509 Certificates used to enroll peer nodes, located physically on the node or mapped to the node. For a Fabric peer node to start, it must have a valid Enrollment Certificate with the required attributes.
Orderer: X.509 Certificates used to enroll orderer nodes, located physically on the node or mapped to the node. For a Fabric orderer node to start, it must have a valid Enrollment Certificate with the required attributes.
Client: X.509 Certificates that allow signed requests to be passed from clients to Fabric nodes. Client certs define the identities of client applications submitting transactions to a Fabric network.
TLS Certificates¶
TLS Certificates allow Fabric nodes and clients to sign and encrypt communications. A valid TLS Certificate is required for any channel communication.
Certificate Expiration¶
Enrollment and TLS Certificates are assigned an expiration date by the issuing Certificate Authority (CA). Expiration dates must be monitored, and certificates must be re-enrolled before expiration. The most important certificate parameter is the Not After element, which indicates its expiration date.
Certificates and Locations¶
Organization CAs supply X.509 Enrollment Certificates for identities and the TLS CAs supply TLS Certificates for securing node and client communications.
Organization CA Certificates¶
Organization CA Root Certificates and Organization CA Admin Certificates provide authorization to interact with the certificate authority for the organization, as described below.
Organization CA Root Certificate¶
Description: Public Certificate that permits verification of all certificates issued by the Organization CA. Organization CA Root Certificates are self-signed certificates if creating a new Certificate Authority (CA), or provided by an external CA.
Location: Stored on disk in the Organization CA directory (ca-cert.pem), and copied into the channel configuration to verify identifies for the organization.
Impact if expired: A new Organization CA Root Certificate must be issued. Organization CA Root Certificates are valid for 15 years.
Organization CA Admin Certificate¶
Description: Certificate used when making admin requests to the Organization CA.
Location: Dependent on implementation:
**Note**: Each identity has a local **msp** directory structure which contains its certificate in the **signcerts** directory and its private key in the **keystore** directory. For details on the **msp** directory, refer to [MSP Structure](https://hyperledger-fabric.readthedocs.io/en/latest/membership/membership.html#msp-structure).
msp ├── IssuerPublicKey ├── IssuerRevocationPublicKey ├── cacerts │ └── localhost-7053.pem ├── keystore │ └── key.pem ├── signcerts │ └── cert.pem └── user
Impact if expired: The Organization Administrator cannot register new identities with the CA, but transaction traffic does not stop.
TLS CA Certificates¶
TLS CA Root Certificates and TLS CA Admin Certificates provide authorization to interact with the certificate authority for the TLS, as described below.
TLS CA Root Certificate¶
Description: Public certificate that permits verification of all certificates issued by the TLS CA. TLS CA Root Certificates are self-signed certificates if creating a new Certificate Authority (CA), or provided by an external CA.
Location: Stored on disk in the TLS CA directory (ca-cert.pem), and copied into the channel configuration to verify TLS Certificates for the organization.
Impact if expired: A new TLS CA Root Certificate must be issued. TLS CA Root Certificates are valid for 15 years.
TLS CA Admin Certificate¶
Description: Certificate used for admin requests to the TLS CA.
Location: Dependent on implementation:
msp ├── IssuerPublicKey ├── IssuerRevocationPublicKey ├── cacerts │ └── localhost-7053.pem ├── keystore │ └── key.pem ├── signcerts │ └── cert.pem └── user
Impact if expired: The Fabric Administrator will no longer be able to register TLS certificates in the TLS CA for nodes in the network.
Peer Certificates¶
A Peer Enrollment Certificate and a Peer TLS Certificate are issued for each peer in an organization.
Peer Enrollment Certificate¶
Description: Authenticates the identity of the peer node when endorsing transactions.
Location: Dependent on implementation:
org1ca
└── peer1
├── msp
│ ├── admincerts
│ │ └── cert.pem
│ ├── cacerts
│ │ └── localhost-7053.pem
│ ├── keystore
│ │ └── key.pem
│ ├── signcerts
│ │ └── cert.pem
│ └── user
|── tls
Impact if expired: Production outage. Peers do not start without a valid Enrollment Certificate.
Peer TLS Certificate¶
Description: Authenticates node component communication on the channel.
Location: Dependent on implementation:
org1ca/
└── peer1
├── msp
└── tls
├── cacerts
├── keystore
│ └── key.pem
├── signcerts
│ └── cert.pem
├── tlscacerts
│ └── tls-localhost-7053.pem
└── user
Impact if expired: Production outage. No communication to the peer is possible.
Orderer Certificates¶
Orderer Enrollment Certificates and Orderer TLS Certificates are issued for each ordering service node in an organization.
Orderer Enrollment Certificate¶
Description: The public key that the orderer uses to sign blocks.
Location: Dependent on implementation:
└── orderer1
├── msp
│ ├── admincerts
│ │ └── cert.pem
│ ├── cacerts
│ │ └── localhost-7053.pem
│ ├── keystore
│ │ └── key.pem
│ ├── signcerts
│ │ └── cert.pem
│ |── user
└── tls
Impact if expired: Production outage. Orderers do not start without a valid Enrollment Certificate.
Orderer TLS Certificate¶
Description: TLS Certificate for the ordering node communication.
Location: Dependent on implementation:
ordererca/
└── orderer1
├── msp
└── tls
├── cacerts
├── keystore
| └── key.pem
├── signcerts
│ └── cert.pem
├── tlscacerts
│ └── tls-localhost-7053.pem
└── user
Impact if expired: Production outage. Ordering nodes are no longer allowed to participate in cluster.
Admin Certificates¶
Ordering Service Organization Channel Admin Certificates and Peer Service Organization Channel Admin Certificates are issued for each organization.
Ordering Service Organization Channel Admin Certificate¶
Description: Certificate for an organization administrator to manage ordering service and channel updates.
Location: Dependent on implementation:
ordererca/
└── ordereradmin
└── msp
├── admincerts
│ └── cert.pem
├── cacerts
│ └── localhost-7053.pem
├── keystore
│ └── key.pem
├── signcerts
│ └── cert.pem
└── user
Impact if expired: Transactions can continue to work successfully. Cannot modify channels from a client application or manage the orderer from the console.
Peer Service Organization Channel Admin Certificate¶
Description - Certificate for an organization administrator to manage a peer, including channel and chaincode services.
Location - Dependent on implementation:
org1ca/ └── org1admin └── msp ├── admincerts │ └── cert.pem ├── cacerts │ └── localhost-7053.pem ├── keystore │ └── key.pem ├── signcerts │ └── cert.pem └── user
Impact if expired: Transactions can continue to work successfully. Cannot install new smart contracts from a client application or manage the peer from the console.
Client Certificates¶
Description: Two types of Client Certificates are issued for each organization:
Organization Enrollment Certificate - Authenticates the client identity for interactions with peers and orderers.
TLS Certificate - Authenticates client communications, and only required if mutual TLS is configured.
Client Certificates expire after one year, using the Hyperledger Fabric CA default settings. Client Certificates can be re-enrolled using the command line Hyperledger Fabric CA utility.
Impact if expired: Client Certificates must be re-enrolled before expiration or the client application will not be able to interact with the Fabric nodes.
Certificate Decoding¶
X.509 Certificates are created by an enrollment of the certificate, based on its registration. The X.509 Certificate contains metadata describing its purpose and identifying the parent CA. The cert expiration is specified in the Not After field.
The certificate details can be decoded using the OpenSSL utility:
# openssl x509 -in cert.pem -text -noout
The following example shows a decoded certificate:
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
47:4d:5d:f6:db:92:6b:54:98:8d:9c:44:0c:ad:b6:77:c5:de:d2:ed
Signature Algorithm: ecdsa-with-SHA256
Issuer: C = US, ST = North Carolina, O = Hyperledger, OU = Fabric, CN = orderer1ca
Validity
Not Before: Feb 4 14:55:00 2022 GMT
Not After : Feb 4 15:51:00 2023 GMT
Subject: C = US, ST = North Carolina, O = Hyperledger, OU = orderer, CN = orderer1
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:29:ec:d5:53:3e:03:9d:64:a4:a4:28:a5:fe:12:
e2:f0:dd:e4:ee:b9:3f:3e:01:b2:3a:d4:68:b1:b2:
4f:82:1a:3a:33:db:92:6d:10:c9:c2:3b:3d:fc:7a:
f0:fa:cc:8b:44:e8:03:cb:a1:6e:eb:b3:6c:05:a2:
f8:fc:3c:af:24
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
63:97:F5:CA:BB:B7:4B:26:84:D9:65:40:E3:43:14:A4:7B:EE:79:FF
X509v3 Authority Key Identifier:
keyid:BA:2A:F8:EA:A5:7D:DF:1D:0F:CF:47:37:41:82:03:7E:04:61:D0:D8
X509v3 Subject Alternative Name:
DNS:server1.testorg.com
1.2.3.4.5.6.7.8.1:
{"attrs":{"hf.Affiliation":"","hf.EnrollmentID":"orderer1","hf.Type":"orderer"}}
Signature Algorithm: ecdsa-with-SHA256
30:45:02:21:00:e1:93:f6:3c:08:f2:b9:fb:06:c9:02:d0:cf:
e1:a6:23:a3:05:78:10:d9:41:2c:1e:2c:91:80:fd:52:ad:62:
9c:02:20:51:33:42:5e:a0:8a:2a:ec:f5:83:46:f0:99:6a:7e:
eb:a8:97:1f:30:99:9d:ae:8d:ef:36:07:da:bb:67:ed:80
Certificate Renewal¶
All nodes and clients should have their enrollment and TLS certificates renewed before expiring to avoid disruption of service. Use the Fabric CA enroll or reenroll function to get updated certificates from the issuing Fabric Certificate Authority (CA). The reenroll function allows the reuse of an existing private key, which is especially important for orderer TLS certificates.
Renewal of Expired Certificates¶
Certificates can still be renewed by using the reenroll function after they expire, but Fabric CA must be at version v1.5.5 or above, and must be configured to allow it.
Enable the reenrollignorecertexpiry option on the Fabric CA server prior to reenrollment by adding the reenrollignorecertexpiry parameter to the fabric-ca-server-config.yaml, as follows:
ca:
certfile: /crypto/tlsca/cert.pem
chainfile: /crypto/tlsca/chain.pem
keyfile: /crypto/tlsca/key.pem
name: tlsca
reenrollignorecertexpiry: true
Alternatively, reenrollignorecertexpiry can be also be set with an environment variable or Fabric CA server startup flag:
Set the FABRIC_CA_SERVER_CA_REENROLLIGNORECERTEXPIRY environment variable to true at CA startup
Start the CA with the flag –ca.reenrollignorecertexpiry=true
Restart the Fabric CA servers.
Renew Peer Certificates¶
For peer enrollment certificates and TLS certificates you can use either the enroll or reenroll function from the Fabric CA client. Use reenroll if you would like to reuse a private key by passing the csr.keyrequest.reusekey option and indicating the mspdir location of the existing private key (the private key is found in the
fabric-ca-client enroll --<OTHER-OPTIONS>
fabric-ca-client reenroll --csr.keyrequest.reusekey --mspdir <LOCATION-OF-IDENTITY'S-MSP-DIRECTORY-THAT-CONTAINS-THE-EXISTING-PRIVATE-KEY> --<OTHER-OPTIONS>
Replace the peer’s enrollment certificate or TLS certificate.
The peer’s enrollment certificate is configured to be in the peer’s <peer.mspConfigPath>/signcerts directory (if private key was not reused the new private key should also be updated in the <peer.mspConfigPath>/keystore directory).
The peer’s TLS certificate is configured to be in the peer’s <peer.tls.cert.file> directory/file (if private key was not reused the new private key should also be updated in the <peer.tls.key.file>).
Restart the peer.
Renew Orderer Certificates¶
For orderer enrollment certificates you can also use the enroll or reenroll function, using the steps described above for peers.
Orderer TLS certificates must be reenrolled with the option to reuse the private key. This is due to the orderer TLS certificates being configured in the application channel (orderer to orderer communication is verified using the public key in the configured TLS certificate). As of v1.4.9 and v2.2.1, orderer nodes verify a matching key rather than the entire configured TLS certificate, enabling orderer TLS certificate renewal without requiring a channel configuration update to update the certificate.
Use reenroll and reuse the TLS certificate’s private key by passing the csr.keyrequest.reusekey option and indicating the mspdir location of the existing private key (the private key is found in the <mspdir>/keystore directory):
fabric-ca-client reenroll --csr.keyrequest.reusekey --mspdir <LOCATION-OF-IDENTITY'S-MSP-DIRECTORY-THAT-CONTAINS-THE-EXISTING-PRIVATE-KEY> --<OTHER-OPTIONS>`
Replace the orderer’s enrollment certificate or TLS certificate.
The orderer’s enrollment certificate is configured to be in the orderer’s <General.LocalMSPDir>/signcerts directory (if private key was not reused the new private key should also be updated in the <General.LocalMSPDir>/keystore directory).
The orderer’s TLS certificate is configured to be in the orderer’s <General.TLS.Certificate> directory/file (don’t update the private key since it was reused).
Restart the orderer node.
Renew Orderer TLS Certificates Without Reusing the Private Key¶
While it is recommended to reuse the private key for orderer TLS certificate renewal, this may not be possible in all situations. Additional steps and planning are required since each ordering service channel must be updated for the new orderer TLS certificate.
Suppose you do not reuse the orderer TLS private key during re-enrollment, and the original orderer TLS certificates have not yet expired. In that case, you must update the orderer TLS certificates one at a time on each node and in each channel configuration and then verify the orderer function before moving on to other orderer node TLS certificate updates.
If you do not reuse the orderer TLS private key and the original orderer TLS certificates have expired, the ordering service will not be able to form consensus and will therefore not be able to process transactions including channel configuration updates. The ordering service recovery process is complicated as you must temporarily utilize the orderer TLSHandshakeTimeShift property on all nodes and restart them in order to form consensus with the expired certificates so that channel configuration updates can be processed to update the TLS certificates. Update a majority of the orderer TLS certificates (e.g. 3 out of 5) one at a time on each node and in the channel configuration. Once a majority have been updated the new certificate expiration dates will no longer fall within the TLSHandshakeTimeShift and therefore the updated ordering nodes will fall out of the consenter set causing loss of consensus again. Next, remove the TLSHandshakeTimeShift setting from all nodes. Upon restart the majority of updated orderer nodes will now form consensus and you can then update the remaining orderer TLS certificates (e.g. 2 out of 5) one at a time on each node and in the channel configuration.
Renew CA Certificate¶
If an organization’s CA certificate is going to expire, or if the organization simply wants to utilize a different CA certificate, the organization can make the update in a phased approach.
First, the organization can submit a channel configuration update that includes both the old and the new CA certificate for their organization,
by including them both in their msp’s cacerts directory (and corresponding config.yaml updates) when creating the channel configuration transaction
(or in the tlscacerts directory for TLS CA certs).
The channel configuration transaction makes the new CA certificate known to all nodes on the channel.
Next, the organization can issue new orderer and peer certificates or TLS certificates (Fabric CA register and enroll commands) based on the new CA, and distribute the new credentials to each of the orderers and peers. Note that new orderer TLS certificates will also require channel configuration updates one at a time for each orderer node as described above. The organization can retire any remaining nodes that have certificates issued from the old CA.
Finally, the organization can submit another channel configuration update that removes the old CA certificate from their msp’s cacerts directory (or tlscacerts directory).