Reconfiguring Your First Network

Note

These steps have been verified to work against the version “1.1.0-preview” tagged Docker images and corresponding utilities. Ensure that you have downloaded the appropriate images and binaries or that you have built them from a branch of Fabric at or beyond the “1.1.0-preview” tag.

This tutorial serves as an extension to the Building Your First Network sample, and will demonstrate the addition of a new organization - Org3 - to the autogenerated application channel - mychannel. It assumes a strong understanding of the BYFN sample, including the usage and functionality of the cryptogen and configtxgen utilities.

The document solely focuses on the integration of a new organization, however the same approach can be adopted when performing other channel configuration updates (e.g. updating modification policies, altering batch size, etc.). The demonstrated operations will be the responsibility of an organizational admin, and will very rarely fall into the purview of a chaincode or application developer.

Note

Ensure that you have installed the requisite Fabric images and utilities and that the automated byfn.sh script runs without error on your machine before continuing. The upcoming steps are reliant upon the generated network and artifacts. See the Prerequisites and Hyperledger Fabric Samples documentation if you have not yet configured your machine. The provided commands also assume that the Fabric utilities are present in the bin directory at the root of fabric-samples. If you have exported these binaries into your PATH variable, then you can modify the commands accordingly without passing the fully qualified path.

Setup the environment

We will be operating from the root of the first-network subdirectory within your local clone of fabric-samples. Change into that directory now. You will also want to open a few extra terminals for ease of use.

First, use the byfn.sh script to tidy up. This command will kill any active or stale docker containers and remove previously generated artifacts. Note that it is by no means necessary to bring down a Fabric network in order to perform reconfiguration tasks, however, for the sake of this tutorial, we want to operate from a known initial state. Therefore let’s run the following command to clean up any previous environments:

./byfn.sh -m down

Now generate the default BYFN artifacts:

./byfn.sh -m generate

And launch the network making use of the scripted execution within the CLI container:

./byfn.sh -m up

In another terminal, change into the org3-artifacts subdirectory.

cd org3-artifacts

There are two yaml files of interest within this workspace - org3-crypto.yaml & configtx.yaml. First, generate the crypto material for Org3:

../../bin/cryptogen generate --config=./org3-crypto.yaml

The above command reads in our new crypto yaml file - org3-crypto.yaml - and leverages the cryptogen utility to generate the keys and certs for an Org3 intermediate CA, as well as two peers bound to this new Org. As with the BYFN implementation, this crypto material is output into a newly generated crypto-config folder within the present working directory.

Now use the the configtxgen utility to print out the Org3-specific configuration material in JSON representation. We will preface the command by telling the tool to look in the current directory for the configtx.yaml file that it needs to ingest.

export FABRIC_CFG_PATH=$PWD && ../../bin/configtxgen -printOrg Org3MSP > ../channel-artifacts/org3.json

The above command creates a JSON file - org3.json - and outputs it into the channel-artifacts subdirectory at the root of first-network. This file contains the modification policy definitions for Org3, as well as three important certificates presented in base 64 format: admin user cert, CA root cert and TLS root cert. In an upcoming step we will append this JSON object to the channel configuration.

Our final piece of housekeeping is to port the Orderer Org’s MSP material into the Org3 crypto-config directory. In particular, we are concerned with the Orderer’s TLS root cert, which will allow for secure communication between Org3 entities and the network’s ordering node.

cd ../ && cp -r crypto-config/ordererOrganizations org3-artifacts/crypto-config/

Now we’re ready to reconfigure...

Start the configtxlator server

The update process makes use of the configuration translator tool - configtxlator. This tool provides a true stateless REST API, independent of the SDK, to simplify configuration tasks in Hyperledger Fabric blockchain networks. The tool converts easily between different equivalent data representations/formats. For example, in one mode of tool operation, the tool performs conversions between the binary protobuf format to a human-readable JSON textual format, and vice-versa. Additionally, the tool can compute configuration updates based on the differences between two distinct sets of configurations transactions.

First, exec into the CLI container. Recall that this container has been mounted with the BYFN crypto-config library, giving us access to the MSP material for the two original Peer Organizations and the Orderer Org. The bootstrapped identity is the Org1 admin user, meaning that any steps where we want to act on behalf of Org2 will require the export of MSP-specific environment variables.

docker exec -it cli bash

By default the CLI container exits after 10000 seconds. If the container has exited, make sure to restart it before continuing. First, check the status of your containers:

docker ps -a

If necessary, relaunch the CLI:

docker start cli

Now install the jq tool into the container. This tool allows us script interactions with JSON objects returned by the configtxlator tool:

# Press `y` when prompted by the command

apt update && apt install jq

Start the configtxlator REST server:

# Press enter twice

configtxlator start &

Set the URL:

CONFIGTXLATOR_URL=http://127.0.0.1:7059

Export the ORDERER_CA and CHANNEL_NAME variables:

export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem  && export CHANNEL_NAME=mychannel

Check to make sure the variables have been properly set:

echo $ORDERER_CA && echo $CHANNEL_NAME

Note

If for any reason you need to restart the CLI container, you will also need to restart the REST server and export the three environment variables - CONFIGTXLATOR_URL, ORDERER_CA and CHANNEL_NAME. The jq installation will persist, you need not install it a second time.

Form the update objects & reconfigure the channel

Now we have a running REST server within the CLI container and we have exported our two key environment variables - ORDERER_CA & CHANNEL_NAME. Let’s go fetch the most recent config block for the channel - mychannel.

peer channel fetch config config_block.pb -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA

The above command applies an arbitrary name - config_block.pb - to this binary protobuf channel configuration block. Note that you do have the ability to alter the naming conventions for the returned protobuf and JSON objects, however you should follow an approach that will allow for easy and intuitive identification.

When you issued the peer channel fetch command, there was a decent amount of output in the terminal. The last line in the logs is of interest:

2017-11-07 17:17:57.383 UTC [channelCmd] readBlock -> DEBU 011 Received block: 2

This is telling us that the most recent configuration block for mychannel is actually block 2, NOT the genesis block. By default, the peer channel fetch config command returns the most recent configuration block for the targeted channel, which is block 2 in our case. When the BYFN scenario executed, the embedded script made two additional configuration updates to the channel. Namely, anchor peers for our two organizations - Org1 & Org2 - were defined by means of two separate channel update transactions. As such, we have the following configuration sequence: block 0 - genesis; block 1 - Org1 anchor peer update; block 2 - Org2 anchor peer update.

Now we will make use of the configtxlator server and decode this channel configuration block into human-readable and editable JSON format.

curl -X POST --data-binary @config_block.pb "$CONFIGTXLATOR_URL/protolator/decode/common.Block" | jq . > config_block.json

We are naming the decoded output - config_block.json. (Again, you are free to apply your own naming conventions throughout these steps.) If you issue an ls within the CLI container, you should see our two objects: the binary protobuf channel configuration - config_block.pb - and the JSON representation of this object - config_block.json.

Now we need to scope the config_block.json object and strip away all of the encapsulating wrappers. We are not concerned with the headers, metadata, creator signature, etc., but, rather, only with the configuration definition inside the transaction. We accomplish this by means of the jq tool:

jq .data.data[0].payload.data.config config_block.json > config.json

This leaves us with a trimmed down JSON object - config.json - which will serve as the baseline for our config update. We’ll use the jq tool once more to append the Org3 configuration definition - org3.json - to the channel’s application groups field, and name the output - updated_config.json.

jq -s '.[0] * {"channel_group":{"groups":{"Application":{"groups": {"Org3MSP":.[1]}}}}}' config.json ./channel-artifacts/org3.json >& updated_config.json

Now, within the CLI container we have two JSON files of interest - config.json & updated_config.json. The initial file contains only Org1 and Org2 material, whereas the aptly named “updated config” file contains all three Orgs. At this point it’s simply a matter of re-encoding these two JSON files and calculating the delta.

First, encode config.json to config.pb:

curl -X POST --data-binary @config.json "$CONFIGTXLATOR_URL/protolator/encode/common.Config" > config.pb

Next, encode updated_config.json to updated_config.pb:

curl -X POST --data-binary @updated_config.json "$CONFIGTXLATOR_URL/protolator/encode/common.Config" > updated_config.pb

Now use the configtxlator server to calculate the delta between these two config protos. This command will output a new protobuf binary named - org3_update.pb:

curl -X POST -F channel=$CHANNEL_NAME -F "original=@config.pb" -F "updated=@updated_config.pb" "${CONFIGTXLATOR_URL}/configtxlator/compute/update-from-configs" > org3_update.pb

This new proto - org3_update.pb - contains the Org3 definitions and high level pointers to the Org1 and Org2 material. We are able to forgo the extensive MSP material and modification policy information for Orgs 1 and 2, because this data is already present within the channel’s genesis block. As such, we only need the delta between the two configurations.

Before submitting the channel update, we need to perform a few final steps. First, let’s decode this object into editable JSON format and call it org3_update.json:

curl -X POST --data-binary @org3_update.pb "$CONFIGTXLATOR_URL/protolator/decode/common.ConfigUpdate" | jq . > org3_update.json

Now, we have a decoded update file - org3_update.json - that we need to wrap in an envelope message. This step will give us back the header field that we stripped away earlier. We’ll name this file - org3_update_in_envelope.json:

echo '{"payload":{"header":{"channel_header":{"channel_id":"mychannel", "type":2}},"data":{"config_update":'$(cat org3_update.json)'}}}' | jq . > org3_update_in_envelope.json

Using our properly formed JSON - org3_update_in_envelope.json - we will leverage the configtxlator tool one last time and convert this object into the fully fledged proto format that Fabric requires. We’ll name our final update object - org3_update_in_envelope.pb:

curl -X POST --data-binary @org3_update_in_envelope.json "$CONFIGTXLATOR_URL/protolator/encode/common.Envelope" > org3_update_in_envelope.pb

Almost done! We now have a protobuf binary - org3_update_in_envelope.pb - within our CLI container, however we need signatures from the requisite Admin users before we can successfully submit the update. The modification policy (mod_policy) for our channel is set to the default of “MAJORITY”, which means that we need an Admin from both of the initial organizations - Org1 & Org2 - to sign off on this update transaction. If we fail to obtain these two signatures, then the ordering service will reject the transaction for failing to fulfill the policy. First, let’s sign this update proto as the Org1 Admin. Remember that the CLI container is bootstrapped with the Org1 MSP material, so we simply need to issue the peer channel signconfigtx command:

peer channel signconfigtx -f org3_update_in_envelope.pb

The final step is to switch the CLI container’s identity to reflect the Org2 Admin user. We do this by exporting four environment variables specific to the Org2 MSP.

Note

The following maneuver is not reflective of a real world operation. A single container would never be mounted with an entire network’s crypto material. Rather, the update object would need to be securely passed out-of-band to an Org2 Admin for inspection and approval.

Export the Org2 environment variables:

# you can issue all of these commands at once

export CORE_PEER_LOCALMSPID="Org2MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp
export CORE_PEER_ADDRESS=peer0.org2.example.com:7051

And lastly we will issue the peer channel update command. The Org2 Admin signature will be attached to this call, so there is no need to manually sign the proto a second time:

Note

The upcoming update call to the ordering service will undergo a series of systematic signature and policy checks. As such you may find it useful to stream and inspect the ordering node’s logs. From another shell, issue a docker logs -f orderer.example.com command to display them.

Send the update call:

peer channel update -f org3_update_in_envelope.pb -c $CHANNEL_NAME -o orderer.example.com:7050 --tls --cafile $ORDERER_CA

You should see a message digest indication similar to the following if your update is successful:

2017-11-07 21:50:17.435 UTC [msp/identity] Sign -> DEBU 00f Sign: digest: 3207B24E40DE2FAB87A2E42BC004FEAA1E6FDCA42977CB78C64F05A88E556ABA

The successful channel update call returns a new block - block 5 - to all of the peers on the channel. Blocks 0-2 are the initial channel configurations, and blocks 3-4 are the instantiation and invocation of the mycc chaincode. As such, block 5 serves as the most recent channel configuration with Org3 now defined on the channel.

Inspect the logs for peer0.org1.example.com:

docker logs -f peer0.org1.example.com

You will see verbose output reflecting the validation checks and update of the peer’s state database with the current configuration of the channel. You will also see the committal of our configuration transaction:

2017-11-15 15:41:05.000 UTC [kvledger] CommitWithPvtData -> DEBU 774 Channel [mychannel]: Committing block [5] to storage

Follow the demonstrated process to fetch and decode the new config block if you wish to inspect its contents. Let’s move on...

Join Org3 to the channel

At this point, the channel configuration has been updated to include our new organization - Org3 - meaning that peers attached to this member can now successfully join the channel.

First, let’s launch the containers for the Org3 peers and an Org3-specific CLI. From the root of first-network kick off the Org3 docker compose:

docker-compose -f docker-compose-org3.yaml up -d

This new compose file has been configured to bridge across our initial network, so the two peers and the CLI container will be able to resolve with the existing peers and ordering node. With the three new containers now running, exec into the Org3-specific CLI container:

docker exec -it Org3cli bash

Just as we did with the initial CLI container, export the two key environment variables - ORDERER_CA & CHANNEL_NAME:

export ORDERER_CA=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem && export CHANNEL_NAME=mychannel

Check to make sure the variables have been properly set:

echo $ORDERER_CA && echo $CHANNEL_NAME

Now let’s send a call to the ordering service asking for the genesis block of mychannel. The ordering service is able to verify the signature attached to this call as a result of our successful channel update. If Org3 had not been successfully appended to the channel config, then the ordering service would reject this request.

Note

Again, you may find it useful to stream the ordering node’s logs to reveal the sign/verify logic and policy checks.

Use the peer channel fetch command to retrieve this block:

peer channel fetch 0 mychannel.block -o orderer.example.com:7050 -c $CHANNEL_NAME --tls --cafile $ORDERER_CA

Notice, that we are passing a 0 to indicate that we want the first block on the channel’s ledger (i.e. the genesis block). If we simply passed the peer channel fetch config command, then we would have received block 5 - the updated config with Org3 defined. However, we can’t begin our ledger with a downstream block; rather we need to join with block 0.

Issue the peer channel join command and pass in the genesis block - mychannel.block:

peer channel join -b mychannel.block

If you want to join the second peer for Org3, export the TLS and ADDRESS variables and reissue the peer channel join command:

export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org3.example.com/peers/peer1.org3.example.com/tls/ca.crt && export CORE_PEER_ADDRESS=peer1.org3.example.com:7051
peer channel join -b mychannel.block

Upgrade & invoke

The final piece of the puzzle is to increment the chaincode version and update the endorsement policy to include Org3. Stay in the Org3 CLI container and install the chaincode. Since we know that an upgrade is coming, we can forgo the futile exercise of installing version 1 of the chaincode. We are solely concerned with the new version where Org3 will be part of the endorsement policy, therefore we’ll jump directly to version 2:

peer chaincode install -n mycc -v 2.0 -p github.com/chaincode/chaincode_example02/go/

Modify the environment variables accordingly and reissue the command if you want to install the chaincode on the second peer of Org3. Note that a second installation is not mandated, as you only need to install chaincode on peers that are going to serve as endorsers or otherwise interface with the ledger (i.e. query only). Peers will still run the validation logic and serve as committers without a running chaincode container.

Now jump back to the original CLI container and install the new version on the Org1 and Org2 peers. We submitted the channel update call with the Org2 admin identity, so the container is still acting on behalf of peer0.org2:

peer chaincode install -n mycc -v 2.0 -p github.com/chaincode/chaincode_example02/go/

Flip to the peer0.org1 identity:

export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=peer0.org1.example.com:7051

And install again:

peer chaincode install -n mycc -v 2.0 -p github.com/chaincode/chaincode_example02/go/

Now we’re ready to upgrade the chaincode. There have been no modifications to the underlying source code, we are simply adding Org3 to the endorsement policy for a chaincode - mycc - on a channel -mychannel.

Note

Any identity satisfying the chaincode’s instantiation policy can issue the upgrade call. By default, these identities are the channel Admins.

Send the call:

peer chaincode upgrade -o orderer.example.com:7050 --tls $CORE_PEER_TLS_ENABLED --cafile $ORDERER_CA -C $CHANNEL_NAME -n mycc -v 2.0 -c '{"Args":["init","a","90","b","210"]}' -P "OR ('Org1MSP.member','Org2MSP.member','Org3MSP.member')"

You can see in the above command that we are specifying our new version by means of the v flag. You also see that the endorsement policy has been modified to -P "OR ('Org1MSP.member','Org2MSP.member','Org3MSP.member')", accurately reflecting the addition of Org3 to the policy. The final area of interest is our constructor request specified with the c flag. As with an instantiate call, a chaincode upgrade requires usage of the init method. If your chaincode requires arguments be passed to the init method, then you will need to provide the appropriate key/vals and reinitialize the state. This is not the recommended practice, because the upgrade submitter could arbitrarily rewrite the world state. Instead, consider editing the source code to remove the argument dependency, or start with a chaincode that does not require args upon instantiation.

The upgrade call adds a new block - block 6 - to the channel’s ledger and allows for the Org3 peers to execute transactions during the endorsement phase. Hop back to the Org3 CLI container and issue a query for the value of a. This will take a bit of time because a chaincode image needs to be built for the targeted peer, and the container needs to start:

peer chaincode query -C $CHANNEL_NAME -n mycc -c '{"Args":["query","a"]}'

We should see a response of Query Result: 90.

Now issue an invocation to move 10 from a to b:

peer chaincode invoke -o orderer.example.com:7050  --tls $CORE_PEER_TLS_ENABLED --cafile $ORDERER_CA -C $CHANNEL_NAME -n mycc -c '{"Args":["invoke","a","b","10"]}'

Query one final time:

peer chaincode query -C $CHANNEL_NAME -n mycc -c '{"Args":["query","a"]}'

We should see a response of Query Result: 80, accurately reflecting the update of this chaincode’s world state.

Conclusion

The reconfiguration process is indeed quite involved, but there is a logical method to the various steps. The endgame is to form a delta transaction object represented in protobuf binary format and then accrue the requisite number of admin signatures such that the reconfiguration transaction fulfills the channel’s modification policy. The configtxlator and jq tools, along with the ever-growing peer channel commands, provide us with the functionality to accomplish this task.