Attention

This version of Hyperledger Fabric is no longer supported or maintained. You can change the documentation version using the selector at the bottom of the navigation panel.

Metrics Reference

Orderer Metrics

Prometheus

The following orderer metrics are exported for consumption by Prometheus.

Name

Type

Description

Labels

blockcutter_block_fill_duration

histogram

The time from first transaction enqueing to the block being cut in seconds.

channel

broadcast_enqueue_duration

histogram

The time to enqueue a transaction in seconds.

channel

type

status

broadcast_processed_count

counter

The number of transactions processed.

channel

type

status

broadcast_validate_duration

histogram

The time to validate a transaction in seconds.

channel

type

status

cluster_comm_egress_queue_capacity

gauge

Capacity of the egress queue.

host

msg_type

channel

cluster_comm_egress_queue_length

gauge

Length of the egress queue.

host

msg_type

channel

cluster_comm_egress_queue_workers

gauge

Count of egress queue workers.

channel

cluster_comm_egress_stream_count

gauge

Count of streams to other nodes.

channel

cluster_comm_egress_tls_connection_count

gauge

Count of TLS connections to other nodes.

cluster_comm_ingress_stream_count

gauge

Count of streams from other nodes.

cluster_comm_msg_dropped_count

counter

Count of messages dropped.

host

channel

cluster_comm_msg_send_time

histogram

The time it takes to send a message in seconds.

host

channel

consensus_etcdraft_active_nodes

gauge

Number of active nodes in this channel.

channel

consensus_etcdraft_cluster_size

gauge

Number of nodes in this channel.

channel

consensus_etcdraft_committed_block_number

gauge

The block number of the latest block committed.

channel

consensus_etcdraft_config_proposals_received

counter

The total number of proposals received for config type transactions.

channel

consensus_etcdraft_data_persist_duration

histogram

The time taken for etcd/raft data to be persisted in storage (in seconds).

channel

consensus_etcdraft_is_leader

gauge

The leadership status of the current node: 1 if it is the leader else 0.

channel

consensus_etcdraft_leader_changes

counter

The number of leader changes since process start.

channel

consensus_etcdraft_normal_proposals_received

counter

The total number of proposals received for normal type transactions.

channel

consensus_etcdraft_proposal_failures

counter

The number of proposal failures.

channel

consensus_etcdraft_snapshot_block_number

gauge

The block number of the latest snapshot.

channel

consensus_kafka_batch_size

gauge

The mean batch size in bytes sent to topics.

topic

consensus_kafka_compression_ratio

gauge

The mean compression ratio (as percentage) for topics.

topic

consensus_kafka_incoming_byte_rate

gauge

Bytes/second read off brokers.

broker_id

consensus_kafka_last_offset_persisted

gauge

The offset specified in the block metadata of the most recently committed block.

channel

consensus_kafka_outgoing_byte_rate

gauge

Bytes/second written to brokers.

broker_id

consensus_kafka_record_send_rate

gauge

The number of records per second sent to topics.

topic

consensus_kafka_records_per_request

gauge

The mean number of records sent per request to topics.

topic

consensus_kafka_request_latency

gauge

The mean request latency in ms to brokers.

broker_id

consensus_kafka_request_rate

gauge

Requests/second sent to brokers.

broker_id

consensus_kafka_request_size

gauge

The mean request size in bytes to brokers.

broker_id

consensus_kafka_response_rate

gauge

Requests/second sent to brokers.

broker_id

consensus_kafka_response_size

gauge

The mean response size in bytes from brokers.

broker_id

deliver_blocks_sent

counter

The number of blocks sent by the deliver service.

channel

filtered

data_type

deliver_requests_completed

counter

The number of deliver requests that have been completed.

channel

filtered

data_type

success

deliver_requests_received

counter

The number of deliver requests that have been received.

channel

filtered

data_type

deliver_streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver_streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

fabric_version

gauge

The active version of Fabric.

version

grpc_comm_conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc_comm_conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc_server_stream_messages_received

counter

The number of stream messages received.

service

method

grpc_server_stream_messages_sent

counter

The number of stream messages sent.

service

method

grpc_server_stream_request_duration

histogram

The time to complete a stream request.

service

method

code

grpc_server_stream_requests_completed

counter

The number of stream requests completed.

service

method

code

grpc_server_stream_requests_received

counter

The number of stream requests received.

service

method

grpc_server_unary_request_duration

histogram

The time to complete a unary request.

service

method

code

grpc_server_unary_requests_completed

counter

The number of unary requests completed.

service

method

code

grpc_server_unary_requests_received

counter

The number of unary requests received.

service

method

ledger_blockchain_height

gauge

Height of the chain in blocks.

channel

ledger_blockstorage_commit_time

histogram

Time taken in seconds for committing the block to storage.

channel

logging_entries_checked

counter

Number of log entries checked against the active logging level

level

logging_entries_written

counter

Number of log entries that are written

level

StatsD

The following orderer metrics are emitted for consumption by StatsD. The %{variable_name} nomenclature represents segments that vary based on context.

For example, %{channel} will be replaced with the name of the channel associated with the metric.

Bucket

Type

Description

blockcutter.block_fill_duration.%{channel}

histogram

The time from first transaction enqueing to the block being cut in seconds.

broadcast.enqueue_duration.%{channel}.%{type}.%{status}

histogram

The time to enqueue a transaction in seconds.

broadcast.processed_count.%{channel}.%{type}.%{status}

counter

The number of transactions processed.

broadcast.validate_duration.%{channel}.%{type}.%{status}

histogram

The time to validate a transaction in seconds.

cluster.comm.egress_queue_capacity.%{host}.%{msg_type}.%{channel}

gauge

Capacity of the egress queue.

cluster.comm.egress_queue_length.%{host}.%{msg_type}.%{channel}

gauge

Length of the egress queue.

cluster.comm.egress_queue_workers.%{channel}

gauge

Count of egress queue workers.

cluster.comm.egress_stream_count.%{channel}

gauge

Count of streams to other nodes.

cluster.comm.egress_tls_connection_count

gauge

Count of TLS connections to other nodes.

cluster.comm.ingress_stream_count

gauge

Count of streams from other nodes.

cluster.comm.msg_dropped_count.%{host}.%{channel}

counter

Count of messages dropped.

cluster.comm.msg_send_time.%{host}.%{channel}

histogram

The time it takes to send a message in seconds.

consensus.etcdraft.active_nodes.%{channel}

gauge

Number of active nodes in this channel.

consensus.etcdraft.cluster_size.%{channel}

gauge

Number of nodes in this channel.

consensus.etcdraft.committed_block_number.%{channel}

gauge

The block number of the latest block committed.

consensus.etcdraft.config_proposals_received.%{channel}

counter

The total number of proposals received for config type transactions.

consensus.etcdraft.data_persist_duration.%{channel}

histogram

The time taken for etcd/raft data to be persisted in storage (in seconds).

consensus.etcdraft.is_leader.%{channel}

gauge

The leadership status of the current node: 1 if it is the leader else 0.

consensus.etcdraft.leader_changes.%{channel}

counter

The number of leader changes since process start.

consensus.etcdraft.normal_proposals_received.%{channel}

counter

The total number of proposals received for normal type transactions.

consensus.etcdraft.proposal_failures.%{channel}

counter

The number of proposal failures.

consensus.etcdraft.snapshot_block_number.%{channel}

gauge

The block number of the latest snapshot.

consensus.kafka.batch_size.%{topic}

gauge

The mean batch size in bytes sent to topics.

consensus.kafka.compression_ratio.%{topic}

gauge

The mean compression ratio (as percentage) for topics.

consensus.kafka.incoming_byte_rate.%{broker_id}

gauge

Bytes/second read off brokers.

consensus.kafka.last_offset_persisted.%{channel}

gauge

The offset specified in the block metadata of the most recently committed block.

consensus.kafka.outgoing_byte_rate.%{broker_id}

gauge

Bytes/second written to brokers.

consensus.kafka.record_send_rate.%{topic}

gauge

The number of records per second sent to topics.

consensus.kafka.records_per_request.%{topic}

gauge

The mean number of records sent per request to topics.

consensus.kafka.request_latency.%{broker_id}

gauge

The mean request latency in ms to brokers.

consensus.kafka.request_rate.%{broker_id}

gauge

Requests/second sent to brokers.

consensus.kafka.request_size.%{broker_id}

gauge

The mean request size in bytes to brokers.

consensus.kafka.response_rate.%{broker_id}

gauge

Requests/second sent to brokers.

consensus.kafka.response_size.%{broker_id}

gauge

The mean response size in bytes from brokers.

deliver.blocks_sent.%{channel}.%{filtered}.%{data_type}

counter

The number of blocks sent by the deliver service.

deliver.requests_completed.%{channel}.%{filtered}.%{data_type}.%{success}

counter

The number of deliver requests that have been completed.

deliver.requests_received.%{channel}.%{filtered}.%{data_type}

counter

The number of deliver requests that have been received.

deliver.streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver.streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

fabric_version.%{version}

gauge

The active version of Fabric.

grpc.comm.conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc.comm.conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc.server.stream_messages_received.%{service}.%{method}

counter

The number of stream messages received.

grpc.server.stream_messages_sent.%{service}.%{method}

counter

The number of stream messages sent.

grpc.server.stream_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a stream request.

grpc.server.stream_requests_completed.%{service}.%{method}.%{code}

counter

The number of stream requests completed.

grpc.server.stream_requests_received.%{service}.%{method}

counter

The number of stream requests received.

grpc.server.unary_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a unary request.

grpc.server.unary_requests_completed.%{service}.%{method}.%{code}

counter

The number of unary requests completed.

grpc.server.unary_requests_received.%{service}.%{method}

counter

The number of unary requests received.

ledger.blockchain_height.%{channel}

gauge

Height of the chain in blocks.

ledger.blockstorage_commit_time.%{channel}

histogram

Time taken in seconds for committing the block to storage.

logging.entries_checked.%{level}

counter

Number of log entries checked against the active logging level

logging.entries_written.%{level}

counter

Number of log entries that are written

Peer Metrics

Prometheus

The following peer metrics are exported for consumption by Prometheus.

Name

Type

Description

Labels

chaincode_execute_timeouts

counter

The number of chaincode executions (Init or Invoke) that have timed out.

chaincode

chaincode_launch_duration

histogram

The time to launch a chaincode.

chaincode

success

chaincode_launch_failures

counter

The number of chaincode launches that have failed.

chaincode

chaincode_launch_timeouts

counter

The number of chaincode launches that have timed out.

chaincode

chaincode_shim_request_duration

histogram

The time to complete chaincode shim requests.

type

channel

chaincode

success

chaincode_shim_requests_completed

counter

The number of chaincode shim requests completed.

type

channel

chaincode

success

chaincode_shim_requests_received

counter

The number of chaincode shim requests received.

type

channel

chaincode

couchdb_processing_time

histogram

Time taken in seconds for the function to complete request to CouchDB

database

function_name

result

deliver_blocks_sent

counter

The number of blocks sent by the deliver service.

channel

filtered

data_type

deliver_requests_completed

counter

The number of deliver requests that have been completed.

channel

filtered

data_type

success

deliver_requests_received

counter

The number of deliver requests that have been received.

channel

filtered

data_type

deliver_streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver_streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

dockercontroller_chaincode_container_build_duration

histogram

The time to build a chaincode image in seconds.

chaincode

success

endorser_chaincode_instantiation_failures

counter

The number of chaincode instantiations or upgrade that have failed.

channel

chaincode

endorser_duplicate_transaction_failures

counter

The number of failed proposals due to duplicate transaction ID.

channel

chaincode

endorser_endorsement_failures

counter

The number of failed endorsements.

channel

chaincode

chaincodeerror

endorser_proposal_acl_failures

counter

The number of proposals that failed ACL checks.

channel

chaincode

endorser_proposal_duration

histogram

The time to complete a proposal.

channel

chaincode

success

endorser_proposal_simulation_failures

counter

The number of failed proposal simulations

channel

chaincode

endorser_proposal_validation_failures

counter

The number of proposals that have failed initial validation.

endorser_proposals_received

counter

The number of proposals received.

endorser_successful_proposals

counter

The number of successful proposals.

fabric_version

gauge

The active version of Fabric.

version

gossip_comm_messages_received

counter

Number of messages received

gossip_comm_messages_sent

counter

Number of messages sent

gossip_comm_overflow_count

counter

Number of outgoing queue buffer overflows

gossip_leader_election_leader

gauge

Peer is leader (1) or follower (0)

channel

gossip_membership_total_peers_known

gauge

Total known peers

channel

gossip_payload_buffer_size

gauge

Size of the payload buffer

channel

gossip_privdata_commit_block_duration

histogram

Time it takes to commit private data and the corresponding block (in seconds)

channel

gossip_privdata_fetch_duration

histogram

Time it takes to fetch missing private data from peers (in seconds)

channel

gossip_privdata_list_missing_duration

histogram

Time it takes to list the missing private data (in seconds)

channel

gossip_privdata_pull_duration

histogram

Time it takes to pull a missing private data element (in seconds)

channel

gossip_privdata_purge_duration

histogram

Time it takes to purge private data (in seconds)

channel

gossip_privdata_reconciliation_duration

histogram

Time it takes for reconciliation to complete (in seconds)

channel

gossip_privdata_retrieve_duration

histogram

Time it takes to retrieve missing private data elements from the ledger (in seconds)

channel

gossip_privdata_send_duration

histogram

Time it takes to send a missing private data element (in seconds)

channel

gossip_privdata_validation_duration

histogram

Time it takes to validate a block (in seconds)

channel

gossip_state_commit_duration

histogram

Time it takes to commit a block in seconds

channel

gossip_state_height

gauge

Current ledger height

channel

grpc_comm_conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc_comm_conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc_server_stream_messages_received

counter

The number of stream messages received.

service

method

grpc_server_stream_messages_sent

counter

The number of stream messages sent.

service

method

grpc_server_stream_request_duration

histogram

The time to complete a stream request.

service

method

code

grpc_server_stream_requests_completed

counter

The number of stream requests completed.

service

method

code

grpc_server_stream_requests_received

counter

The number of stream requests received.

service

method

grpc_server_unary_request_duration

histogram

The time to complete a unary request.

service

method

code

grpc_server_unary_requests_completed

counter

The number of unary requests completed.

service

method

code

grpc_server_unary_requests_received

counter

The number of unary requests received.

service

method

ledger_block_processing_time

histogram

Time taken in seconds for ledger block processing.

channel

ledger_blockchain_height

gauge

Height of the chain in blocks.

channel

ledger_blockstorage_and_pvtdata_commit_time

histogram

Time taken in seconds for committing the block and private data to storage.

channel

ledger_blockstorage_commit_time

histogram

Time taken in seconds for committing the block to storage.

channel

ledger_statedb_commit_time

histogram

Time taken in seconds for committing block changes to state db.

channel

ledger_transaction_count

counter

Number of transactions processed.

channel

transaction_type

chaincode

validation_code

logging_entries_checked

counter

Number of log entries checked against the active logging level

level

logging_entries_written

counter

Number of log entries that are written

level

StatsD

The following peer metrics are emitted for consumption by StatsD. The %{variable_name} nomenclature represents segments that vary based on context.

For example, %{channel} will be replaced with the name of the channel associated with the metric.

Bucket

Type

Description

chaincode.execute_timeouts.%{chaincode}

counter

The number of chaincode executions (Init or Invoke) that have timed out.

chaincode.launch_duration.%{chaincode}.%{success}

histogram

The time to launch a chaincode.

chaincode.launch_failures.%{chaincode}

counter

The number of chaincode launches that have failed.

chaincode.launch_timeouts.%{chaincode}

counter

The number of chaincode launches that have timed out.

chaincode.shim_request_duration.%{type}.%{channel}.%{chaincode}.%{success}

histogram

The time to complete chaincode shim requests.

chaincode.shim_requests_completed.%{type}.%{channel}.%{chaincode}.%{success}

counter

The number of chaincode shim requests completed.

chaincode.shim_requests_received.%{type}.%{channel}.%{chaincode}

counter

The number of chaincode shim requests received.

couchdb.processing_time.%{database}.%{function_name}.%{result}

histogram

Time taken in seconds for the function to complete request to CouchDB

deliver.blocks_sent.%{channel}.%{filtered}.%{data_type}

counter

The number of blocks sent by the deliver service.

deliver.requests_completed.%{channel}.%{filtered}.%{data_type}.%{success}

counter

The number of deliver requests that have been completed.

deliver.requests_received.%{channel}.%{filtered}.%{data_type}

counter

The number of deliver requests that have been received.

deliver.streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver.streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

dockercontroller.chaincode_container_build_duration.%{chaincode}.%{success}

histogram

The time to build a chaincode image in seconds.

endorser.chaincode_instantiation_failures.%{channel}.%{chaincode}

counter

The number of chaincode instantiations or upgrade that have failed.

endorser.duplicate_transaction_failures.%{channel}.%{chaincode}

counter

The number of failed proposals due to duplicate transaction ID.

endorser.endorsement_failures.%{channel}.%{chaincode}.%{chaincodeerror}

counter

The number of failed endorsements.

endorser.proposal_acl_failures.%{channel}.%{chaincode}

counter

The number of proposals that failed ACL checks.

endorser.proposal_duration.%{channel}.%{chaincode}.%{success}

histogram

The time to complete a proposal.

endorser.proposal_simulation_failures.%{channel}.%{chaincode}

counter

The number of failed proposal simulations

endorser.proposal_validation_failures

counter

The number of proposals that have failed initial validation.

endorser.proposals_received

counter

The number of proposals received.

endorser.successful_proposals

counter

The number of successful proposals.

fabric_version.%{version}

gauge

The active version of Fabric.

gossip.comm.messages_received

counter

Number of messages received

gossip.comm.messages_sent

counter

Number of messages sent

gossip.comm.overflow_count

counter

Number of outgoing queue buffer overflows

gossip.leader_election.leader.%{channel}

gauge

Peer is leader (1) or follower (0)

gossip.membership.total_peers_known.%{channel}

gauge

Total known peers

gossip.payload_buffer.size.%{channel}

gauge

Size of the payload buffer

gossip.privdata.commit_block_duration.%{channel}

histogram

Time it takes to commit private data and the corresponding block (in seconds)

gossip.privdata.fetch_duration.%{channel}

histogram

Time it takes to fetch missing private data from peers (in seconds)

gossip.privdata.list_missing_duration.%{channel}

histogram

Time it takes to list the missing private data (in seconds)

gossip.privdata.pull_duration.%{channel}

histogram

Time it takes to pull a missing private data element (in seconds)

gossip.privdata.purge_duration.%{channel}

histogram

Time it takes to purge private data (in seconds)

gossip.privdata.reconciliation_duration.%{channel}

histogram

Time it takes for reconciliation to complete (in seconds)

gossip.privdata.retrieve_duration.%{channel}

histogram

Time it takes to retrieve missing private data elements from the ledger (in seconds)

gossip.privdata.send_duration.%{channel}

histogram

Time it takes to send a missing private data element (in seconds)

gossip.privdata.validation_duration.%{channel}

histogram

Time it takes to validate a block (in seconds)

gossip.state.commit_duration.%{channel}

histogram

Time it takes to commit a block in seconds

gossip.state.height.%{channel}

gauge

Current ledger height

grpc.comm.conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc.comm.conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc.server.stream_messages_received.%{service}.%{method}

counter

The number of stream messages received.

grpc.server.stream_messages_sent.%{service}.%{method}

counter

The number of stream messages sent.

grpc.server.stream_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a stream request.

grpc.server.stream_requests_completed.%{service}.%{method}.%{code}

counter

The number of stream requests completed.

grpc.server.stream_requests_received.%{service}.%{method}

counter

The number of stream requests received.

grpc.server.unary_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a unary request.

grpc.server.unary_requests_completed.%{service}.%{method}.%{code}

counter

The number of unary requests completed.

grpc.server.unary_requests_received.%{service}.%{method}

counter

The number of unary requests received.

ledger.block_processing_time.%{channel}

histogram

Time taken in seconds for ledger block processing.

ledger.blockchain_height.%{channel}

gauge

Height of the chain in blocks.

ledger.blockstorage_and_pvtdata_commit_time.%{channel}

histogram

Time taken in seconds for committing the block and private data to storage.

ledger.blockstorage_commit_time.%{channel}

histogram

Time taken in seconds for committing the block to storage.

ledger.statedb_commit_time.%{channel}

histogram

Time taken in seconds for committing block changes to state db.

ledger.transaction_count.%{channel}.%{transaction_type}.%{chaincode}.%{validation_code}

counter

Number of transactions processed.

logging.entries_checked.%{level}

counter

Number of log entries checked against the active logging level

logging.entries_written.%{level}

counter

Number of log entries that are written