introduction
The previous article has gone over a lot of details, just to clarify the flow of tcc distributed transactions, and introduce the differences and application scenarios between it and the other two commonly used distributed transactions "reliable message queue" and "saga".
Next, let's introduce our protagonist "dtm", which is similar to Ali's distributed transaction framework seata. Unfortunately, due to the language limitations of seata, we. Neters are "deterred", but what's more pitiful is that we can't find a similar framework or middleware in our. net ecosystem. Seeing that the new order placing process designed by myself was aborted because there was no middleware to support the business, I suddenly received a message from group friends and threw me a link to the dtm project, saying that I could have a look. Here, I am very grateful to the group of friends for sending me this information, which makes me find the confidence to continue the transformation of orders. In the following pages, I will make a record of the problems and some details I encountered during the dtm trial.
Introduction to dtm
There are many articles on the official website on the introduction of dtm, a cross language tcc distributed transaction middleware. It is clear from a step-by-step view. The official website address is: https://dtm.pub/.
The c# sdk client officially recommended by dtm is provided by Zhang Shanyou. The address is: https://github.com/yedf/dtmcli-csharp
, corresponding demo address: https://github.com/yedf/dtmcli-csharp-sample . Captain Zhang "has few cruel words". Both sdk and examples are provided, but there are no detailed documents used. The subtext is to look at the source code combined with the examples. sdk source code is relatively simple. It is not difficult to master the use of dtm by running example debugging and combining with the database of dtm server. It's just that you will encounter all kinds of problems for the first time.
Use of dtm
dtm server deployment
About docker-compose.yml
My deployment environment is CentOS 7.6 and docker 20.10. The original docker-compose.yml is as follows:
Among them, some of the original versions framed by the red line are not available, because there are problems in use. They are added after finding the problems through communication with the author.
1. Function of red box 1: make the time and time zone in the container consistent with that used by the host computer. If it is not added, the mysql container will use Beijing time, while the dtm service program does use utc time, with a difference of 8 hours. In this way, the dtm server will fail to complete the correct scheduling of transactions due to misjudgment and query due to time confusion when managing transactions.
2. Function of red box 2: it is obviously for the interconnection between dtm server and mysql.
Note: the author said that he can use ubuntu without adding a red box, but CentOS 7.6 does.
On the starting order of dtm service program and mysql container
In fact, after using the modified docker-compose.yml to start the container, there is still a problem. The dtm service program will flash back and report the following error:
The error in reading the report should be that dtm is not connected to the database. Here, the author is really responsible. He helped me to see the reason. He said that dtm started to create the database before mysql was ready. Let me try to start the dtm container after mysql container is ready. After the instructions of the author, I manually restart the exited dtm container with docker start. Finally, the corresponding library on mysql is created by the dtm service program, and dtm will no longer report an error and exit. The environment is now set up.
Use of sdk
After setting up the dtm server environment, we can run c# examples to see the effect.
Registration of dtm service:
public void ConfigureServices(IServiceCollection services)
{
services.AddDtmcli(dtm => dtm.DtmUrl = "http://192.168.1.244:8080");
services.AddControllers();
services.AddSwaggerGen(c =>
{
c.SwaggerDoc("v1", new OpenApiInfo { Title = "DtmTccSample", Version = "v1" });
});
}
It's very simple, just one sentence: services. Adddtmcli (dtm = > dtm. DtmUrl =“ http://192.168.1.244:8080 ");， DtmUrl is the address of the dtm server.
There is only one piece of code in the sdk introduction documents and examples:
First of all, there is no problem that the sample code can run, but it is only a normal scenario in which the sub transaction is successfully committed when the try phase is successfully executed.
If the try phase is not successful or abnormal, the sub transaction should be rolled back. The example code should read as follows:
1 [HttpPost] 2 public async Task<RestResult> Demo() 3 { 4 var svc = "http://172.16.2.14:5000/api"; 5 TransRequest request = new TransRequest() { Amount = 30 }; 6 var cts = new CancellationTokenSource(); 7 try 8 { 9 var res1 = string.Empty; 10 var res2 = string.Empty; 11 var isTryOk = true; 12 await globalTransaction.Excecute(async (tcc) => 13 { 14 res1 = await tcc.CallBranch(request, svc + "/TransOut/Try", svc + "/TransOut/Confirm", svc + "/TransOut/Cancel", cts.Token); 15 res2 = await tcc.CallBranch(request, svc + "/TransIn/Try", svc + "/TransIn/Confirm", svc + "/TransIn/Cancel",cts.Token); 16 17 if (!res1.Contains("SUCCESS") || !res2.Contains("SUCCESS")) 18 { 19 //Exception log 20 logger.LogError($"{res1}{res2}"); 21 //Throw exception: the purpose is to make sdk Client catch exception, notification dtm Server, try The phase encountered an exception and all sub transactions were rolled back 22 throw new AccessViolationException($"{res1}{res2}"); 23 } 24 logger.LogInformation($"tcc returns: {res1}-{res2}"); 25 }, cts.Token); 26 if (res1.Contains("SUCCESS") || res2.Contains("SUCCESS")) 27 { 28 logger.LogError($"{res1}{res2}"); 29 return new RestResult() { Result = "FAILURE" }; 30 } 31 } 32 catch(Exception ex) 33 { 34 35 } 36 return new RestResult() { Result = "SUCCESS" }; 37 }
In fact, the core code is lines 17-23. To inform dtm of the failure of the try phase, we need to capture the return values of all sub transactions in the try phase and judge whether any return value is not what we want (the return values of sub transactions in the example are res1 and res2). If the return value is incorrect, an exception will be thrown in the globaltransaction.execute method. The thrown exception will be caught by the sdk client and told dtm that the sub transaction should be rolled back. The specific source code is as follows:
1 public async Task<string> Excecute(Func<Tcc,Task> tcc_cb, CancellationToken cancellationToken =default) 2 { 3 var tcc = new Tcc(this.dtmClient, await this.GenGid()); 4 5 var tbody = new TccBody 6 { 7 Gid = tcc.Gid, 8 Trans_Type ="tcc" 9 }; 10 11 12 try 13 { 14 await dtmClient.TccPrepare(tbody, cancellationToken); 15 16 await tcc_cb(tcc); 17 18 await dtmClient.TccSubmit(tbody, cancellationToken); 19 } 20 catch(Exception ex) 21 { 22 logger.LogError(ex,"submitting or abort global transaction error"); 23 await this.dtmClient.TccAbort(tbody, cancellationToken); 24 return string.Empty; 25 } 26 return tcc.Gid; 27 }
In the catch block, this.dtmClient.TccAbort is to tell dtm that the transaction should be returned.
About retry after failure of Confirm and Cancel phases
If the try phase is executed successfully, all sub transactions will execute the Confirm interface. If the try phase fails, all sub transactions will execute the Cancel phase interface. If any sub transaction fails in the Confirm or Cancel phase, the sub transaction will be retried continuously until it succeeds.
It should be noted here that the execution of Confirm or Cancel sub transactions is sequential. If any sub transaction is repeatedly retried due to execution failure, the remaining unexecuted sub transactions will also be blocked and cannot be executed.
The first retry interval is 10s, and the subsequent retry time will be doubled, with an interval of 20 40 80 160.
If the sub transactions are executed in the order of ab in the try phase, the cancel phase is executed in the order of ba.
dtm log
The dtm log is very detailed, including the call to the transaction interface, the execution of the database and error exceptions. The command to view the dtm log in the container:
docker container logs dtm-api-1 --follow
dtm correlation table
The table of tcc transaction design is as follows:
Where trans_global is the global transaction table, trans_branch is the branch sub transaction table. The notes are also clear at a glance. The sql is as follows:
1 CREATE TABLE `trans_global` ( 2 `id` int(11) NOT NULL AUTO_INCREMENT, 3 `gid` varchar(128) NOT NULL COMMENT 'Transaction Global id', 4 `trans_type` varchar(45) NOT NULL COMMENT 'Transaction type: saga | xa | tcc | msg', 5 `status` varchar(12) NOT NULL COMMENT 'Status of global transactions prepared | submitted | aborting | finished | rollbacked', 6 `query_prepared` varchar(128) NOT NULL COMMENT 'prepared Query of status transactions api', 7 `protocol` varchar(45) NOT NULL COMMENT 'communication protocol http | grpc', 8 `create_time` datetime DEFAULT NULL, 9 `update_time` datetime DEFAULT NULL, 10 `commit_time` datetime DEFAULT NULL, 11 `finish_time` datetime DEFAULT NULL, 12 `rollback_time` datetime DEFAULT NULL, 13 `next_cron_interval` int(11) DEFAULT NULL COMMENT 'Interval of next scheduled processing', 14 `next_cron_time` datetime DEFAULT NULL COMMENT 'Next scheduled processing time', 15 `owner` varchar(128) NOT NULL DEFAULT '' COMMENT 'Locks that are processing global transactions', 16 PRIMARY KEY (`id`), 17 UNIQUE KEY `gid` (`gid`), 18 KEY `owner` (`owner`), 19 KEY `create_time` (`create_time`), 20 KEY `update_time` (`update_time`), 21 KEY `status_next_cron_time` (`status`,`next_cron_time`) COMMENT 'This index is used to query global transactions that have timed out, and it can be indexed reasonably' 22 ) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4; 23 24 CREATE TABLE `trans_branch` ( 25 `id` int(11) NOT NULL AUTO_INCREMENT, 26 `gid` varchar(128) NOT NULL COMMENT 'Transaction Global id', 27 `url` varchar(128) NOT NULL COMMENT 'Action associated url', 28 `data` text COMMENT 'Data carried by the request', 29 `branch_id` varchar(128) NOT NULL COMMENT 'Transaction branch name', 30 `branch_type` varchar(45) NOT NULL COMMENT 'Transaction branch type saga_action | saga_compensate | xa', 31 `status` varchar(45) NOT NULL COMMENT 'Status of the step submitted | finished | rollbacked', 32 `finish_time` datetime DEFAULT NULL, 33 `rollback_time` datetime DEFAULT NULL, 34 `create_time` datetime DEFAULT NULL, 35 `update_time` datetime DEFAULT NULL, 36 PRIMARY KEY (`id`), 37 UNIQUE KEY `gid_uniq` (`gid`,`branch_id`,`branch_type`), 38 KEY `create_time` (`create_time`), 39 KEY `update_time` (`update_time`) 40 ) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8mb4;
Finally, the dtm c# communication wechat group is attached. If you have any questions, you can discuss them together:
