32 | context.Context type
In the last article, we talked about the sync.WaitGroup type: a synchronization tool that can help us implement one to many goroutine collaboration processes.
When using the WaitGroup value, we'd better use the standard pattern of "first unified Add, then concurrent Done, and finally Wait" to build the collaboration process.
If the Add method of the value is called concurrently in order to increase the value of its counter while calling the Wait method of the value, panic is likely to be raised.
This brings a problem. If we cannot determine the number of goroutines executing subtasks at the beginning, there is a certain risk to use the WaitGroup value to coordinate them with the goroutines distributing subtasks. One solution is to enable goroutine to execute subtasks in batches.
Leading content: supplementary knowledge of WaitGroup value
As we all know, WaitGroup value can be reused, but the integrity of its counting cycle needs to be guaranteed. Especially when it comes to calling its Wait method, its next counting cycle must Wait until the call of the Wait method corresponding to the current counting cycle is completed.
The possible panic situation I mentioned earlier is caused by not following this rule.
As long as we enable goroutine to execute subtasks in batches on the premise of strictly following the above rules, there will certainly be no problem. There are many specific implementation methods, and the simplest one is to use the for loop as an auxiliary. The code here is as follows:
func coordinateWithWaitGroup() {
total := 12
stride := 3
var num int32
fmt.Printf("The number: %d [with sync.WaitGroup]\n", num)
var wg sync.WaitGroup
for i := 1; i <= total; i = i + stride {
wg.Add(stride)
for j := 0; j < stride; j++ {
go addNum(&num, i+j, wg.Done)
}
wg.Wait()
}
fmt.Println("End.")
}The coordinateWithWaitGroup function shown here is a modified version of the function with the same name in the previous article. The addNum function invoked is a simplified version of the same name function in the previous article. Both functions have been placed in the demo67.go file.
package main
import (
"fmt"
"sync"
"sync/atomic"
"time"
)
func main() {
coordinateWithWaitGroup()
}
func coordinateWithWaitGroup() {
total := 12
stride := 3
var num int32
fmt.Printf("The number: %d [with sync.WaitGroup]\n", num)
var wg sync.WaitGroup
//fmt.Println("Start loop ...")
for i := 1; i <= total; i = i + stride {
//if i > 1 {
// fmt.Println("Next iteration:")
//}
wg.Add(stride)
for j := 0; j < stride; j++ {
go addNum(&num, i+j, wg.Done)
}
wg.Wait()
}
fmt.Println("End.")
}
// addNum is used to atomically increase the value of the variable referred to by numP once.
func addNum(numP *int32, id int, deferFunc func()) {
defer func() {
deferFunc()
}()
for i := 0; ; i++ {
currNum := atomic.LoadInt32(numP)
newNum := currNum + 1
time.Sleep(time.Millisecond * 200)
if atomic.CompareAndSwapInt32(numP, currNum, newNum) {
fmt.Printf("The number: %d [%d-%d]\n", newNum, id, i)
break
} else {
//fmt.Printf("The CAS operation failed. [%d-%d]\n", id, i)
}
}
}We can see that the modified coordinateWithWaitGroup function circularly uses the WaitGroup value represented by the variable wg. It still uses the mode of "first unified Add, then concurrent Done, and finally Wait", but it reuses it by using the for statement.
Well, by now, you should have understood the use of WaitGroup value. However, I now want you to use another tool to implement the above collaboration process.
Our question today is: how to use the program entities in the context package to realize one to many goroutine collaboration process?
More specifically, I need you to write a function called coordinateWithContext. This function should have the same function as the coordinateWithWaitGroup function above.
Obviously, you can no longer use sync.WaitGroup, but use the functions and context types in the context package as implementation tools. Note that it is not important to enable goroutine to execute subtasks in batches.
I'll give you a reference answer here.
func coordinateWithContext() {
total := 12
var num int32
fmt.Printf("The number: %d [with context.Context]\n", num)
cxt, cancelFunc := context.WithCancel(context.Background())
for i := 1; i <= total; i++ {
go addNum(&num, i, func() {
if atomic.LoadInt32(&num) == int32(total) {
cancelFunc()
}
})
}
<-cxt.Done()
fmt.Println("End.")
}In this function, I call the context.Background function and the context.WithCancel function successively, and get a revocable context.Context type value (represented by variable cxt), and a context.CancelFunc type revocation function (represented by variable cancelFunc).
In the following unique for statement, I asynchronously call the addNum function through a go statement in each iteration. The total number of calls is only based on the value of the total variable.
Notice the last parameter value I gave to the addNum function. It is an anonymous function that contains only one if statement. This if statement will "atomically" load the value of the num variable and determine whether it is equal to the value of the total variable.
If the two values are equal, the cancelfunction is called. This means that if all addNum functions are executed, the goroutine of the distribution subtask will be notified immediately.
Here, the goroutine of the distributed subtask is the goroutine that executes the coordinateWithContext function. After executing the for statement, it will immediately call the Done function of cxt variable and try to receive the channel returned by the function.
Once the cancelfunction is called, the receiving operation for the channel will end immediately. Therefore, this can realize the function of "waiting for all addNum functions to be executed".
Problem analysis
context.Context type (hereinafter referred to as context type) is added to the standard library only when Go 1.7 is published. Then, many other code packages in the standard library have been extended to support it, including os/exec package, net package, database/sql package, runtime/pprof package and runtime/trace package.
Context type is actively supported by many code packages in the standard library mainly because it is a very general synchronization tool. Its value can not only be diffused arbitrarily, but also be used to transmit additional information and signals.
More specifically, the Context type can provide a class of values representing the Context. This type of value is concurrency safe, that is, it can be propagated to multiple goroutine s.
Because the context type is actually an interface type, and all private types implementing the interface in the context package are pointer types based on a data type, such propagation will not affect the function and security of the type value.
The value of Context type (hereinafter referred to as Context value) can be multiplied, which means that we can generate any child value through a Context value. These child values can carry the attributes and data of their parent values, and can also respond to the signals we convey through their parent values.
Because of this, all context values together form a tree structure that represents the overall picture of the context. The tree root (or context root node) of this tree is a predefined context value in the context package, which is globally unique. We can get it by calling the context.Background function (that's what I did in the coordinateWithContext function).
Note here that this context root node is only a basic fulcrum, and it does not provide any additional functions. In other words, it can neither be cancel led nor carry any data.
In addition, the context package also contains four functions for multiplying context values, namely WithCancel, WithDeadline, WithTimeout and WithValue.
The type of the first parameter of these functions is context.Context, and the name is parent. As the name suggests, the parameters in this position correspond to the parent value of the context value they will generate.
The WithCancel function is used to generate a child value of a revocable parent. In the coordinateWithContext function, I call this function to obtain a Context value derived from the Context root node and a function used to trigger the revocation signal.
The WithDeadline function and WithTimeout function can be used to generate a child value of a parent that will be revoked regularly. As for the WithValue function, we can call it to generate a child value of the parent that will carry additional data.
By now, we have a basic understanding of the functions and context types in the context package. But that's not enough. Let's expand it.
Knowledge expansion
Question 1: what does "revocable" mean in the context package? What does "undo" a context value mean?
I believe many Go program developers who are new to the context package will have such questions. Indeed, the word "revocable" is more abstract here, which is easy to confuse. Let me explain here again.
This needs to start with the declaration of Context type. There are two methods in this interface that are closely related to undo. The Done method returns a receive channel with element type struct {}. However, the purpose of this receiving channel is not to pass the element value, but to let the caller perceive the signal that "cancels" the current Context value.
Once the current Context value is revoked, the receiving channel here will be closed immediately. As we all know, for a channel that does not contain any element value, its closing will immediately end any receiving operation against it.
Because of this, in the coordinateWithContext function, the receiving operation based on the call expression cxt.Done() can sense the cancellation signal.
In addition to letting the user of the Context value perceive the revocation signal and get the specific reason for "revocation", it is sometimes necessary. The latter is the function of the Err method of Context type. The result of this method is of type error, and its value can only be equal to the value of the context.Canceled variable or the value of the context.DeadlineExceeded variable.
The former is used to indicate manual revocation, while the latter represents revocation due to the expiration time given by us.
You may already feel that for the Context value, the word "revocation" refers to the signal used to express the state of "revocation"; If the verb is spoken, it refers to the communication of cancellation signal; "Revocable" refers to the ability to convey this cancellation signal.
As I mentioned earlier, when we call the context.WithCancel function to generate a revocable Context value, we will also get a function to trigger the revocation signal.
By calling this function, we can trigger the cancellation signal for the Context value. Once triggered, the cancellation signal is immediately transmitted to the Context value and expressed by the result value of its Done method (a receiving channel).
The revocation function is only responsible for triggering the signal, and the corresponding revocable Context value is only responsible for transmitting the signal. They will not care about the subsequent specific "revocation" operation. In fact, our code can perform arbitrary operations after sensing the revocation signal, and the Context value has no constraints on this.
Finally, if we Go deeper, the original meaning of "undo" here is to terminate the response of the program to a request (such as HTTP request) or cancel the processing of an instruction (such as SQL instruction). This is also the original intention of the Go language team when creating context code package and context type.
If we look at the API and source code of net package and database/sql package, we can understand their typical applications in this regard.
package main
import (
"context"
"fmt"
"sync/atomic"
"time"
)
func main() {
coordinateWithContext()
}
func coordinateWithContext() {
total := 12
var num int32
fmt.Printf("The number: %d [with context.Context]\n", num)
cxt, cancelFunc := context.WithCancel(context.Background())
for i := 1; i <= total; i++ {
go addNum(&num, i, func() {
if atomic.LoadInt32(&num) == int32(total) {
cancelFunc()
}
})
}
<-cxt.Done()
fmt.Println("End.")
}
// addNum is used to atomically increase the value of the variable referred to by numP once.
func addNum(numP *int32, id int, deferFunc func()) {
defer func() {
deferFunc()
}()
for i := 0; ; i++ {
currNum := atomic.LoadInt32(numP)
newNum := currNum + 1
time.Sleep(time.Millisecond * 200)
if atomic.CompareAndSwapInt32(numP, currNum, newNum) {
fmt.Printf("The number: %d [%d-%d]\n", newNum, id, i)
break
} else {
//fmt.Printf("The CAS operation failed. [%d-%d]\n", id, i)
}
}
}Question 2: how does the revocation signal propagate in the context tree?
As I mentioned earlier, the context package contains four functions for multiplying context values. WithCancel, WithDeadline and WithTimeout are all used to generate revocable child values based on the given context value.
The WithCancel function of the context package will produce two result values after being called. The first result value is the revocable context value, and the second result value is the function used to trigger the revocation signal.
After the Undo function is called, the corresponding Context value will first close its internal receiving channel, that is, the channel returned by its Done method.
It then sends a cancellation signal to all its child values (or child nodes). These subvalues will do the same and continue to propagate the cancellation signal. Finally, the Context value breaks its association with its parent value.
(propagate undo signal in context tree)
The context value generated by calling the WithDeadline function or WithTimeout function of the context package is also revocable. They can not only be revoked manually, but also be revoked automatically according to the expiration time given at the time of generation. Here, the function of timing cancellation is realized by means of their internal timer.
When the expiration time arrives, the behavior of these two Context values is almost the same as that when the Context value is manually revoked, but the former will stop and release its internal timer at the end.
Finally, note that the Context value obtained by calling the context.WithValue function is irrevocable. When cancellation signals are propagated, if they are encountered, they will cross directly and try to pass the signal directly to their child values.
Question 3: how to carry data through the Context value? How to get data from it?
Now that we have talked about the WithValue function of the context package, let's talk about how the context value carries data.
The WithValue function requires three parameters when generating a new Context value (hereinafter referred to as the Context value containing data), namely, parent value, key and value. Similar to "dictionary constraints on keys", the key type here must be decidable.
The reason is very simple. When we get data from it, it needs to find the corresponding value according to the given key. However, this Context value does not use a dictionary to store keys and values, and the latter two are simply stored in the corresponding fields of the former.
The Value method of Context type is used to obtain data. When we call the Value method of the Context Value containing data, it will first judge whether the given key is equal to the key stored in the current Value. If it is equal, it will directly return the Value stored in the Value, otherwise it will continue to search in its parent Value.
If the equal key is still not stored in its parent value, the method will look all the way along the direction of the context root node.
Note that except for the Context Value containing data, several other Context values cannot carry data. Therefore, when searching along the road, the Value method of Context Value will directly cross those values.
If the Value of the Value method we call itself does not contain data, the Value method of its parent or grandparent will be actually called. This is because the actual types of these Context values belong to structure types, and they all express the parent-child relationship by "embedding their parent values into themselves".
Finally, remind me that the Context interface does not provide a method to change data. Therefore, under normal circumstances, we can only store new data by adding a Context value containing data in the Context tree, or discard the corresponding data by revoking the parent value of this value. If the data you store here can be changed from the outside, you must ensure its own security.
package main
import (
"context"
"fmt"
"time"
)
type myKey int
func main() {
keys := []myKey{
myKey(20),
myKey(30),
myKey(60),
myKey(61),
}
values := []string{
"value in node2",
"value in node3",
"value in node6",
"value in node6Branch",
}
rootNode := context.Background()
node1, cancelFunc1 := context.WithCancel(rootNode)
defer cancelFunc1()
// Example 1.
node2 := context.WithValue(node1, keys[0], values[0])
node3 := context.WithValue(node2, keys[1], values[1])
fmt.Printf("The value of the key %v found in the node3: %v\n",
keys[0], node3.Value(keys[0]))
fmt.Printf("The value of the key %v found in the node3: %v\n",
keys[1], node3.Value(keys[1]))
fmt.Printf("The value of the key %v found in the node3: %v\n",
keys[2], node3.Value(keys[2]))
fmt.Println()
// Example 2.
node4, _ := context.WithCancel(node3)
node5, _ := context.WithTimeout(node4, time.Hour)
fmt.Printf("The value of the key %v found in the node5: %v\n",
keys[0], node5.Value(keys[0]))
fmt.Printf("The value of the key %v found in the node5: %v\n",
keys[1], node5.Value(keys[1]))
fmt.Println()
// Example 3.
node6 := context.WithValue(node5, keys[2], values[2])
fmt.Printf("The value of the key %v found in the node6: %v\n",
keys[0], node6.Value(keys[0]))
fmt.Printf("The value of the key %v found in the node6: %v\n",
keys[2], node6.Value(keys[2]))
fmt.Println()
// Example 4.
node6Branch := context.WithValue(node5, keys[3], values[3])
fmt.Printf("The value of the key %v found in the node6Branch: %v\n",
keys[1], node6Branch.Value(keys[1]))
fmt.Printf("The value of the key %v found in the node6Branch: %v\n",
keys[2], node6Branch.Value(keys[2]))
fmt.Printf("The value of the key %v found in the node6Branch: %v\n",
keys[3], node6Branch.Value(keys[3]))
fmt.Println()
// Example 5.
node7, _ := context.WithCancel(node6)
node8, _ := context.WithTimeout(node7, time.Hour)
fmt.Printf("The value of the key %v found in the node8: %v\n",
keys[1], node8.Value(keys[1]))
fmt.Printf("The value of the key %v found in the node8: %v\n",
keys[2], node8.Value(keys[2]))
fmt.Printf("The value of the key %v found in the node8: %v\n",
keys[3], node8.Value(keys[3]))
}summary
Today, we mainly discuss the functions and context types in the context package. The functions in the package are used to generate new context type values. Context type is a synchronization tool that can help us realize multi goroutine collaborative processes. Not only that, we can also communicate cancellation signals or pass data through this type of value.
The actual values of Context type are generally divided into three types: root Context value, revocable Context value and Context value with data. All Context values together form a Context tree. The scope of this tree is global, and the root Context value is the root of this tree. It is globally unique and does not provide any additional functionality.
Revocable Context values can be divided into: Context values that can only be manually revoked and Context values that can be revoked regularly.
We can manually undo them by the Undo function obtained when they are generated. For the latter, the timing reversal time must be completely determined at the time of generation and cannot be changed. However, we can manually undo it before the expiration time is reached.
Once the Undo function is called, the undo signal will be immediately transmitted to the corresponding Context value and expressed by the receiving channel returned by the Done method of the value.
The operation of "undo" is the key for the Context value to coordinate multiple goroutine s. The undo signal always propagates in the direction of the leaf node of the Context tree.
The Context Value containing data can carry data. Each Value can store a pair of keys and values. When we call its Value method, it will find the Value one by one along the direction of the root node of the Context tree. If an equal key is found, it will immediately return the corresponding Value, otherwise it will return nil at the end.
The Context value containing data cannot be revoked, and the revocable Context value cannot carry data. However, because they form an organic whole (i.e. Context tree), they are much more powerful than sync.WaitGroup in function.
Thinking questions
Today's question is: is the Context value breadth first or depth first when conveying the cancellation signal? What are its strengths and weaknesses?