1. Example demonstration
Here, take the XO trading family on the official website as an example. The trading family is a tic tac toe game. Before we start, we need to build a single node sawtooth environment. For details, please see the previous blog:
Sawtooth, using docker to start a single node
After confirming that the links are normal, we can link to the shell container to play the game:
docker exec -it sawtooth-shell-default bash
The container contains the environment related to sawtooth, including sawtooth commands and client xo commands related to the transaction family.
1.1. Create players
Tic tac toe chess requires two users, so two users need to be created. Creation here refers to creating the user's key and putting it under $HOME/.sawtooth/keys.
root@b872105f1c59:/usr/bin# sawtooth keygen jack writing file: /root/.sawtooth/keys/jack.priv writing file: /root/.sawtooth/keys/jack.pub root@b872105f1c59:/usr/bin# sawtooth keygen jill writing file: /root/.sawtooth/keys/jill.priv writing file: /root/.sawtooth/keys/jill.pub
1.2. Create a game
Use the newly created user jack to create a tic tac toe game room called my game.
xo create my-game --username jack
In order to view the creation of the game, you can view the current game list with the following command.
xo list
Here, I made the following errors. If I don't encounter them, I can ignore them:
Error: Failed to connect to http://127.0.0.1:8008/state?address=5b7349: HTTPConnectionPool(host='127.0.0.1', port=8008): Max retries exceeded with url: /state?address=5b7349 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f410b4537b8>: Failed to establish a new connection: [Errno 111] Connection refused',))
There should be a problem with the connection of docker, but if the container connectivity is OK just now, you can submit the parameters by manually specifying the url:
xo list --url rest-api:8008
If you encounter similar problems in subsequent commands, you should add this url parameter. If you don't, you don't have to worry about it. You won't add this parameter in subsequent instructions
Print out the following contents, indicating that the game is created successfully:
GAME PLAYER 1 PLAYER 2 BOARD STATE my-game --------- P1-NEXT
The essence of the xo list command is still a REST request, but it is wrapped, which is much simpler than submitting with curl.
1.3. Play chess as a player
The processing logic of the transaction family is to regard the first player playing chess as PLAYER 1 and the second as PLAYER 2, and you need to specify your identity with the -- username parameter when playing chess.
Here, the chessboard needs to be regarded as the arrangement of the numeric keypad:
1 | 2 | 3 ---|---|--- 4 | 5 | 6 ---|---|--- 7 | 8 | 9
Here, we use the xo take command to make jack and jill play the next chess piece at 5 and 1 respectively:
xo take my-game 5 --username jack xo take my-game 1 --username jill
After each successful submission of the take command, the following contents will be printed:
Response: { "link": "http://rest-api:8008/batch_statuses?id=fd61fb0946fc11807d9ae456a8689fad87b71c711a1eb408072b602beaaca36e420bb5ffd3ba2188f6a52354edab084687840f51e6a05de79f5c25c7b6ede627" }
Then we can the following current chessboard state and use the following command:
xo show my-game
The following outputs can be seen:
GAME: : my-game PLAYER 1 : 035446 PLAYER 2 : 03aa97 STATE : P1-NEXT O | | ---|---|--- | X | ---|---|--- | |
After each command is submitted and the ledger is successfully updated, the game status stored in the background will be updated and stored in the following format:
<game-name>,<board-state>,<game-state>,<player1-key>,<player2-key>
The current status is stored in the following format in the background:
my-game,O---X----,P1-NEXT,02403a...,03729b...
Every time a player tries to play chess, the background logic will check whether the player playing chess is the current player who should play chess. If the order is violated, the ledger will not be updated. For example, the player needs a jack to play chess, but jill tries to play chess again.
It can be seen that although the transaction was successfully submitted, the chess game was not successful:
root@754b02b93ab8:/# xo take my-game 7 --username jill --url rest-api:8008 Response: { "link": "http://rest-api:8008/batch_statuses?id=f7643aa33d47e1240584965ad8c65bb9116728c2f286c9242839122747265b492352d6e98b9d72b42c4c555112c994fe3c5edf9bbfdbfafaf085126d8ad30532" } root@754b02b93ab8:/# xo show my-game --url rest-api:8008 GAME: : my-game PLAYER 1 : 035446 PLAYER 2 : 03aa97 STATE : P1-NEXT O | | ---|---|--- | X | ---|---|--- | |
Therefore, on the other hand, we can also know that the legitimacy of the transaction data is verified by the verifier, while the legitimacy of the transaction logic is verified by the transaction processor, which can be compared to that the compiled program may not run normally.
Then play chess several times. Eventually, one side will win or draw. After winning, the status will change to the corresponding result. At this time, playing chess will still not take effect.
1.4. Delete game
Any player can delete the game by using the following command
xo delete my-game
View the current game list after submitting successfully:
root@754b02b93ab8:/# xo list --url rest-api:8008 GAME PLAYER 1 PLAYER 2 BOARD STATE
You can see that there are no more games.
In addition, the xo command can also specify -- auth user and -- auth password for simple authentication of the REST API.
The above is all the logic of the tic tac toe chess game. In fact, apart from the factors of blockchain, it is an ordinary small game, so as long as the application logic is written, it can basically be transplanted to sawtooth.
2. Introduction to development process
The following is the instructions for using the Python SDK of sawtooth on the official website Using the Python SDK , it mainly introduces the development process and terminology of transaction processor and client, but it is not a complete practice. The complete content of tic tac toe chess (XO) can be viewed https://github.com/hyperledger/sawtooth-sdk-python/tree/main/examples/xo_python
A transaction family contains the following three components:
- Transaction processor: defines the business logic of the application. Its responsibilities include registering the verifier, processing the transaction load and related metadata, and reading the state required for setting.
- Data model: used to record and store data.
- Client: handle the client logic of the application. His responsibilities include creating and signing transactions, compressing transactions into a batch, and then submitting them to the verifier. The client can send the batch to the verifier through the REST API, post method or ZeroMQ.
The client and processor must use the same data model, serialization / encoding method and addressing method.
After reading the complete document, I found that the transaction processor and the client are two real components, and the data model is a logical concept, which does not need to be implemented by using the API of the sawtoothSDK. Therefore, next, I will introduce the development process of the transaction processor and the client respectively.
2.1. Create transaction processor
The transaction processor has two top-level components:
- Processor class. The SDK provides a general processor class.
- The Handler class depends on the application and contains the business logic for a specific transaction family. Multiple handlers can be connected to a processor class instance.
My personal understanding here is that the Processor provides a framework using the template pattern, which takes the Handler as an interface for the user to implement, and then the Processor calls its methods.
2.1.1. Access point and Handler
Because the transaction processor is a long-running process, it needs an access point, which is equivalent to the main function.
In the access point, the TransactionProcessor class provides an address to connect the verifier and the Handler class.
Code sawtooth_xo/processor/main.py is as follows:
from sawtooth_sdk.processor.core import TransactionProcessor from sawtooth_xo.processor.handler import XoTransactionHandler def main(): # In docker, the url would be the validator's container name with # port 4004 processor = TransactionProcessor(url='tcp://127.0.0.1:4004') handler = XoTransactionHandler() processor.add_handler(handler) processor.start()
Here you can see that you have imported a Handler implemented by yourself, added it to the Handler manager of the open port processor, and then started the processor.
The specific XoTransactionHandler class code implemented here is sawtooth_xo/processor/handler.py is as follows:
class XoTransactionHandler(TransactionHandler): def __init__(self, namespace_prefix): self._namespace_prefix = namespace_prefix @property def family_name(self): return 'xo' @property def family_versions(self): return ['1.0'] @property def namespaces(self): return [self._namespace_prefix] def apply(self, transaction, context): # ...
The code contains the related method family of metadata settings_ Name, namespaces, etc. and an apply method. The metadata method is used to describe the handler for the processor, and the apply method writes specific business logic. The call of the apply method contains two parameters, and the transaction is defined and created by protobuf Transaction instance , hold the command to be executed, and the context is the context instance provided by the Python SDK, which stores relevant status and other information.
The trade fair contains the load byte stream transparent to the verification core and the description information of the transaction family. How to deal with the binary serialization protocol is decided by the implementer, that is, the verifier will ensure that the data finally processed by the logic is trusted.
2.1.2.apply method implementation
Here, an application implementation framework is as follows, and the code is still located in sawtooth_xo/processor/handler.py:
def apply(self, transaction, context): header = transaction.header signer = header.signer_public_key xo_payload = XoPayload.from_bytes(transaction.payload) xo_state = XoState(context) if xo_payload.action == 'delete': ... elif xo_payload.action == 'create': ... elif xo_payload.action == 'take': ... else: raise InvalidTransaction('Unhandled action: {}'.format( xo_payload.action))
Here, in order to split the state and operation, the XO instance has XoState class and XoPayload class. The former is used to process the actions given by the transaction, while the latter contains specific states, including whose turn is currently, what pieces are on the chessboard, etc., that is, the content stored by the latter is changed through the former. The details of these two classes will be described later.
Several game operations are defined here, including create, create a new game, take a child in an idle position on the chessboard, delete, and delete a game according to XO_ The action field of payload corresponds to different operations.
This paper describes the specific implementation of tic tac toe chess game, including the implementation of several specific operations defined in the apply method.
2.1.2.1.create
The code is as follows:
elif xo_payload.action == 'create': if xo_state.get_game(xo_payload.name) is not None: raise InvalidTransaction( 'Invalid action: Game already exists: {}'.format( xo_payload.name)) game = Game(name=xo_payload.name, board="-" * 9, state="P1-NEXT", player1="", player2="") xo_state.set_game(xo_payload.name, game) _display("Player {} created a game.".format(signer[:6]))
Here, first judge whether the Game corresponding to the given name already exists. If so, throw an error. Otherwise, use this name to create a new Game instance (this instance is described by the Game class, and its code can be seen later in this article, which can be simply understood as a structure). Then store it, and finally print a prompt. A user has created a Game.
2.1.2.2.Delete
The code is as follows:
if xo_payload.action == 'delete': game = xo_state.get_game(xo_payload.name) if game is None: raise InvalidTransaction( 'Invalid action: game does not exist') xo_state.delete_game(xo_payload.name)
First, judge whether the game exists. If it does not exist, throw an exception, otherwise call xo_state deletes the game by name.
2.1.2.3.Take
The code is as follows:
elif xo_payload.action == 'take': game = xo_state.get_game(xo_payload.name) if game is None: raise InvalidTransaction( 'Invalid action: Take requires an existing game') if game.state in ('P1-WIN', 'P2-WIN', 'TIE'): raise InvalidTransaction('Invalid Action: Game has ended') if (game.player1 and game.state == 'P1-NEXT' and game.player1 != signer) or \ (game.player2 and game.state == 'P2-NEXT' and game.player2 != signer): raise InvalidTransaction( "Not this player's turn: {}".format(signer[:6])) if game.board[xo_payload.space - 1] != '-': raise InvalidTransaction( 'Invalid Action: space {} already taken'.format( xo_payload)) if game.player1 == '': game.player1 = signer elif game.player2 == '': game.player2 = signer upd_board = _update_board(game.board, xo_payload.space, game.state) upd_game_state = _update_game_state(game.state, upd_board) game.board = upd_board game.state = upd_game_state xo_state.set_game(xo_payload.name, game) _display( "Player {} takes space: {}\n\n".format( signer[:6], xo_payload.space) + _game_data_to_str( game.board, game.state, game.player1, game.player2, xo_payload.name))
Here, we first need to judge whether the action is legal according to the new chess action. If it is legal, set the new chessboard position, etc., and then set it to the state corresponding to the current game name. In essence, it is an operation to update the current game state.
2.1.3. Load xopayload
The transaction consists of a header and a payload. The header contains the signer, which is used to represent the current player. The payload will contain specific transaction information, including the name of the game in the current application (it cannot contain | because | is regarded as a separator in the storage deconstruction. If your storage logic has a way to escape | you can make the game name contain |), behavior (create, delete and take) and location (if the behavior is not take, this field is set to null).
In fact, we can see that the data structure of the load is user-defined, which is equivalent to the setting of the request parameters of the interface in the web project.
Code sawtooth_xo/processor/xo_payload.py is as follows:
class XoPayload: def __init__(self, payload): try: # The payload is csv utf-8 encoded string name, action, space = payload.decode().split(",") except ValueError: raise InvalidTransaction("Invalid payload serialization") if not name: raise InvalidTransaction('Name is required') if '|' in name: raise InvalidTransaction('Name cannot contain "|"') if not action: raise InvalidTransaction('Action is required') if action not in ('create', 'take', 'delete'): raise InvalidTransaction('Invalid action: {}'.format(action)) if action == 'take': try: if int(space) not in range(1, 10): raise InvalidTransaction( "Space must be an integer from 1 to 9") except ValueError: raise InvalidTransaction( 'Space must be an integer from 1 to 9') if action == 'take': space = int(space) self._name = name self._action = action self._space = space @staticmethod def from_bytes(payload): return XoPayload(payload=payload) @property def name(self): return self._name @property def action(self): return self._action @property def space(self): return self._space
Here, the load will be a separated string, including three fields: name, action and space. Of course, developers can customize their own request structure, and even use json. Here, XoPayload is similar to an entity class. Its construction method receives a byte group, and then constructs its own three attributes according to the byte array, However, if the constructed property has illegal data, an exception will be thrown.
2.1.4. State xostate
XoState class can convert game information into bytes, then exist in the verifier's radius Merkle tree, and convert bytes stored in the radius Merkle tree into game information.
The status entry of Xostate class contains the following contents separated by spaces:
name,board,game-state,player-key-1,player-key-2
Of which:
- Name is the name of the game and cannot contain |.
- board is a string with a length of 9, containing O, X, or -, pieces or spaces representing a party.
- Game state is the game state, which can be P1-NEXT, P2-NEXT, P1-WIN, P2-WIN or TIE, that is, which side should play chess or which side has won and drawn.
- player-key-1 and player-key-2 are the public keys of players participating in the game
In the actual storage, the item entry hash is used as the key and the item is stored as the value. Therefore, there will be a hash collision problem. The processing method here is to sort the conflicting items in alphabetical order and use | segmentation for storage, that is < a-entry > | < b-entry > |.
The code is as follows: sawtooth_xo/processor/xo_state.py.
XO_NAMESPACE = hashlib.sha512('xo'.encode("utf-8")).hexdigest()[0:6] class Game: def __init__(self, name, board, state, player1, player2): self.name = name self.board = board self.state = state self.player1 = player1 self.player2 = player2 class XoState: TIMEOUT = 3 def __init__(self, context): """Constructor. Args: context (sawtooth_sdk.processor.context.Context): Access to validator state from within the transaction processor. """ self._context = context self._address_cache = {} def delete_game(self, game_name): """Delete the Game named game_name from state. Args: game_name (str): The name. Raises: KeyError: The Game with game_name does not exist. """ games = self._load_games(game_name=game_name) del games[game_name] if games: self._store_game(game_name, games=games) else: self._delete_game(game_name) def set_game(self, game_name, game): """Store the game in the validator state. Args: game_name (str): The name. game (Game): The information specifying the current game. """ games = self._load_games(game_name=game_name) games[game_name] = game self._store_game(game_name, games=games) def get_game(self, game_name): """Get the game associated with game_name. Args: game_name (str): The name. Returns: (Game): All the information specifying a game. """ return self._load_games(game_name=game_name).get(game_name) def _store_game(self, game_name, games): address = _make_xo_address(game_name) state_data = self._serialize(games) self._address_cache[address] = state_data self._context.set_state( {address: state_data}, timeout=self.TIMEOUT) def _delete_game(self, game_name): address = _make_xo_address(game_name) self._context.delete_state( [address], timeout=self.TIMEOUT) self._address_cache[address] = None def _load_games(self, game_name): address = _make_xo_address(game_name) if address in self._address_cache: if self._address_cache[address]: serialized_games = self._address_cache[address] games = self._deserialize(serialized_games) else: games = {} else: state_entries = self._context.get_state( [address], timeout=self.TIMEOUT) if state_entries: self._address_cache[address] = state_entries[0].data games = self._deserialize(data=state_entries[0].data) else: self._address_cache[address] = None games = {} return games def _deserialize(self, data): """Take bytes stored in state and deserialize them into Python Game objects. Args: data (bytes): The UTF-8 encoded string stored in state. Returns: (dict): game name (str) keys, Game values. """ games = {} try: for game in data.decode().split("|"): name, board, state, player1, player2 = game.split(",") games[name] = Game(name, board, state, player1, player2) except ValueError: raise InternalError("Failed to deserialize game data") return games def _serialize(self, games): """Takes a dict of game objects and serializes them into bytes. Args: games (dict): game name (str) keys, Game values. Returns: (bytes): The UTF-8 encoded string stored in state. """ game_strs = [] for name, g in games.items(): game_str = ",".join( [name, g.board, g.state, g.player1, g.player2]) game_strs.append(game_str) return "|".join(sorted(game_strs)).encode() def _make_xo_address(name): return XO_NAMESPACE + \ hashlib.sha512(name.encode('utf-8')).hexdigest()[:64]
First of all, there is an entity class Game of the Game project, which corresponds to the entry we mentioned earlier. XoState is the actual storage class here. The context parameter passed in init here is also the context object passed in from the SDK we analyzed earlier, that is, the object actually stored by sawtooth.
In addition, the XO needs to be explained_ Namespace, it is speculated here that multiple transaction families share the same repository, so a namespace is needed in front of the stored key to avoid conflicts between different transaction families due to the same key.
2.2. Create client
The information submitted by the client to the distributed ledger needs a series of encryption protection for identity confirmation and data validity verification. However, the SDK of sawtooth has implemented most functions and provided abstract interfaces to simplify the operation.
2.2.1. Create private key and signature
In order to confirm your identity and sign the message sent by yourself so that the verifier can verify effectively, you need a 256 bit key as your private key. Sawtooth uses the secp256k1 ECDSA standard to implement signature, that is, almost all 32 byte sets can be used as a valid key. Therefore, it is a relatively simple task to use the SDK signature module to generate a valid key.
from sawtooth_signing import create_context from sawtooth_signing import CryptoFactory context = create_context('secp256k1') private_key = context.new_random_private_key() signer = CryptoFactory(context).new_signer(private_key)
Here, secp256k1 is used to create a new private key, and the private key is used to create a new signer object.
The private key is very important. It is the only identification to prove the user's identity. If it is lost, it cannot be recovered. If the private key is obtained by others, the other party can pretend to be the real owner of the private key for all operations, so it needs to be kept properly.
2.2.2. Build transaction
The transaction changes the state of Sawtooh. The transaction consists of the following parts:
- Encoded binary load
- Encrypted binary encoded transaction header
- Auxiliary processing metadata
- Signature on transaction header
2.2.2.1. Load code
The transaction load consists of binary encoded data that the verifier does not need to care about. The logic of encoding and decoding (essentially the mutual conversion of byte arrays) is completely realized by a specific transaction processor. Therefore, the decision on which format to use for coding is up to the transaction processor. Therefore, it is necessary to clarify the relevant definitions of the transaction processor before coding. For the Integer Key chain code family, it uses CBOR (concise binary object presentation, a data exchange format) to encode its payload.
import cbor payload = { 'Verb': 'set', 'Name': 'foo', 'Value': 42} payload_bytes = cbor.dumps(payload)
2.2.2.2. Create transaction header
The transaction header contains information routed to the correct transaction processor, transaction family_name and transaction family version family_version, etc. (which can correspond to the metadata of the transaction processor), the read or write address (to assist the transaction processor to read and write data more efficiently?), and the public key signer corresponding to the signature_ public_ Key (for later signature verification), and payload byte for summary calculation using SHA-512_ sha512.
The code is as follows:
from hashlib import sha512 from sawtooth_sdk.protobuf.transaction_pb2 import TransactionHeader txn_header_bytes = TransactionHeader( family_name='intkey', family_version='1.0', inputs=['1cf1266e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7'], outputs=['1cf1266e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7'], signer_public_key=signer.get_public_key().as_hex(), # In this example, we're signing the batch with the same private key, # but the batch can be signed by another party, in which case, the # public key will need to be associated with that key. batcher_public_key=signer.get_public_key().as_hex(), # In this example, there are no dependencies. This list should include # an previous transaction header signatures that must be applied for # this transaction to successfully commit. # For example, # dependencies=['540a6803971d1880ec73a96cb97815a95d374cbad5d865925e5aa0432fcf1931539afe10310c122c5eaae15df61236079abbf4f258889359c4d175516934484a'], dependencies=[], payload_sha512=sha512(payload_bytes).hexdigest() ).SerializeToString()
2.2.2.3. Assembly transaction
After the transaction header is created, its byte array will be used to create a signature, which will also be used as the transaction ID. the transaction header byte array, transaction header signature and payload byte array are used to construct a complete transaction.
from sawtooth_sdk.protobuf.transaction_pb2 import Transaction signature = signer.sign(txn_header_bytes) txn = Transaction( header=txn_header_bytes, header_signature=signature, payload=payload_bytes )
2.2.2.4. Code transaction (optional)
If the operations of creating transactions and packaging into batches are completed on the same machine, the encoding operation can be omitted. However, if the processing component of the packaging batch is external, the transactions need to be serialized and passed, and then packaged on the external packer. Python SDK provides two methods: one is to store multiple transactions in the TransactionList and serialize them together, and the other is to serialize a single transaction. The code is as follows:
from sawtooth_sdk.protobuf.transaction_pb2 import TransactionList txn_list_bytes = TransactionList( transactions=[txn1, txn2] ).SerializeToString() txn_bytes = txn.SerializeToString()
2.2.3. Build batch
Batch is the smallest unit of transaction processing in sawtooth, that is, when submitting several transactions to the verifier, it must be included in one batch. A batch is an atomic operation in which one or more transactions are either executed or not executed. This is somewhat similar to the concept of transactions in a database, but all transactions in a batch may not have dependencies.
2.2.3.1. Create batch header
Similar to the transaction header, each batch needs to have a batch header, but the batch structure is simpler than the transaction. A batch header only needs the signer's public key and the transaction id list (the id is also the signature of the transaction, so the verification can be completed only with the batch header), and its order is consistent with their order in the batch.
from sawtooth_sdk.protobuf.batch_pb2 import BatchHeader txns = [txn] batch_header_bytes = BatchHeader( signer_public_key=signer.get_public_key().as_hex(), transaction_ids=[txn.header_signature for txn in txns], ).SerializeToString()
2.2.3.2. Create batch
Similar to creating a transaction process, the header signature is used as the batch id, and then the batch is composed of the transaction header byte array, batch id and transaction.
from sawtooth_sdk.protobuf.batch_pb2 import Batch signature = signer.sign(batch_header_bytes) batch = Batch( header=batch_header_bytes, header_signature=signature, transactions=txns )
2.2.3.3. Batch coded in batch list
In order to submit several batches to the verifier, they need to be collected in the BatchList first. Multiple related or unrelated batches can be submitted to the same BatchList. However, unlike batches, BatchList is not atomic. Batches from different clients may be inserted into the same BatchList during processing. The code is as follows:
from sawtooth_sdk.protobuf.batch_pb2 import BatchList batch_list_bytes = BatchList(batches=[batch]).SerializeToString()
2.2.4. Submit batch to verifier
The client and verifier can exist in parallel. Their communication is completed using the REST API. First, set the "content type" of the request header to "application / octet stream", and then set the request body to the serialized BatchList.
There are many ways to submit HTTP requests. The urllib library is used in the example on the official website:
import urllib.request from urllib.error import HTTPError try: request = urllib.request.Request( 'http://rest.api.domain/batches', batch_list_bytes, method='POST', headers={'Content-Type': 'application/octet-stream'}) response = urllib.request.urlopen(request) except HTTPError as e: response = e.file
If the batch list is stored as a file, you can use the following code to read it:
output = open('intkey.batches', 'wb') output.write(batch_list_bytes)
Or, instead of Python, submit directly with curl command:
curl --request POST \ --header "Content-Type: application/octet-stream" \ --data-binary @intkey.batches \ "http://rest.api.domain/batches
These are the key steps of development. I originally planned to implement a simple bank account application as a practice according to this process, but I found that there may be network problems in the docker environment. In addition, the official XO client also has a set of complex Command-line processing logic and packaging process, involving some advanced applications of Python, So it's cool for the time being. Later, I'll try to imitate the cat by reading the XO source code.