reflex
Go language provides a mechanism to update variables or check their values, call their methods and the internal operations they support at run time, without knowing the specific types of these variables at compile time. This mechanism is called reflection (the definition of reflection here is roughly the same as that in other languages). Reflection also allows us to treat the type itself as a value type of the first type.
The author adds: The first type of value does not necessarily refer to the object in object-oriented programming, but refers to all entities in the program (such as variables, functions, queues, dictionaries, etc.). Generally, the first type of value has the following characteristics:
- Can be stored in variables or other structures
- Can be passed as parameters to other methods / functions
- Can be used as the return value of a method / function
- Can be created at execution time without having to write it all out at design time
- Have a fixed identity
"Inherent identity" means that the entity has an internal representation rather than being identified by name. For example, anonymous functions can also call any name by assignment. The basic types of values (int, float) in most languages are the first type of values; But the array is not necessarily. For example, when the array in C is used as a function parameter, the address of the first element is passed, and the array length information is lost. For most dynamic languages, functions / methods are first-class values, such as Python and Go, but Ruby is not because it cannot return a method. The first type of function is necessary for functional programming languages.
In this chapter, we will explore the reflection characteristics of Go language, see what expressive power it can add to the language, and how to use the reflection mechanism in two crucial APIs: one is the string formatting function provided by fmt package, and the other is the protocol specific encoding and decoding function similar to encoding/json and encoding/xml. For the text/template and html/template packages we saw in section 4.6, their implementation also depends on reflection technology. However, reflection is a complex introspective technique and should not be used arbitrarily. Therefore, although the above packages are internally implemented by reflection technology, their own APIs do not expose reflection related interfaces.
Why reflection?
Sometimes we need to write a function that can handle the values of a class of types that do not meet the common public interface. It may also be because they have no definite representation (the uncertain type may be one of multiple types), or these types may not exist when we design the function
A familiar example is the string formatting processing logic provided by fmt.Fprintf function, which can be used to format and print any type of value, and even support user-defined types. Let's try to implement a similar function. For simplicity, our function takes only one parameter and returns a formatted string similar to fmt.Sprint. The function name we implemented is also called sprint.
We first use the switch type branch to test whether the input parameter implements the string method. If so, call the method. Then continue to add the type test branch, check whether the dynamic type of the value is the basic type such as string, int and bool, and perform the corresponding formatting operation in each case.
func Sprint(x interface{}) string { // Author's note: interface {} in Go language can be used as generics
type stringer interface {
String() string
}
switch x := x.(type) { // Note to the author: if there is an outsourcing type, the type assertion will be restored to the underlying type
case stringer:
return x.String()
case string:
return x
case int:
return strconv.Itoa(x)
// ...similar cases for int16, uint32, and so on...
case bool:
if x {
return "true"
}
return "false"
default:
// array, chan, func, map, pointer, slice, struct author's note: they are all reference types and printing is meaningless
return "???"
}
}But how do we handle other types like [] float64, map[string][]string? We can certainly add more test branches, but the number of these composite types is basically infinite. And how to handle named types like URL. Values? Even if the type branch can recognize that the underlying basic type is map[string][]string, it does not match the url.Values type because they are two different types, and the switch type branch cannot contain each type similar to url.Values, which will lead to dependence on these libraries.
There is no way to check the representation of unknown types, so let's stop here. That's why we need reflection.
reflect.Type and reflect.Value
The implementation of reflection mechanism in Go language is provided by the reflect package. It defines two important types, Type and Value. A Type represents a Go Type, which is an interface. There are many ways to distinguish types and check their components, such as members of a structure or parameters of a function. The only thing that can reflect the implementation of reflect.Type is the Type description information of the interface (§ 7.5), and it is this entity that identifies the dynamic Type of the interface Value.
The function reflect.TypeOf accepts any interface {} type and returns its dynamic type as reflect.Type:
t := reflect.TypeOf(3) // A reflect.type variable is the author's supplement fmt.Println(t.String()) // int fmt.Println(t) // int
The TypeOf(3) call passes the value 3 (argument) to the interface {} parameter (formal parameter). Returning to section 7.5, converting a specific value to an interface type will have an implicit interface conversion operation. This operation creates an interface value containing two information: the dynamic type of the operand (int in this case) and its dynamic value (3 in this case)
Because reflect.TypeOf returns a dynamically typed interface value, it always returns a specific type. Therefore, the following code will print "* os.File" instead of "io.Writer" (* os.File is the declared type and io.Writer is the actual type). Later, we'll see the reflect.Type that can express the interface type.
var w io.Writer = os.Stdout fmt.Println(reflect.Typeof(w)) // *os.File is the declaration type and io.Writer is the underlying type
Note that the reflect.Type interface satisfies the fmt.Stringer interface. Because printing the dynamic type of an interface is helpful for debugging and logging, fmt.Printf provides an abbreviated% T parameter, which internally uses reflect.TypeOf to output:
fmt.Printf("%T\n", 3) // int is equivalent to output reflect.Typeof(3) + "\n"Another important type in the reflect package is value. A reflect. Value can load any type of value. The function reflect.ValueOf accepts any interface {} type and returns a reflect.Value loaded with its dynamic value. Similar to reflect.TypeOf, the result (type) returned by reflect.ValueOf is also a specific type, but reflect.Value can also hold an interface value.
v := reflect.ValueOf(3) // a reflect.Value
fmt.Println(v) // 3
fmt.Println("%v\n", v) // 3
fmt.Println(v.String()) // Note: "< int value >" note that v.String() returns the interface typeSimilar to reflect.Type, reflect.Value also satisfies the fmt.Stringer interface, but unless the Value holds a String, the String method returns only its type. Using the% v flag parameter of the FMT package will give special treatment to reflect.Values.
Calling the Type method on Value will return the reflection.Type corresponding to the specific Type:
t := v.Type() // A reflect.type variable is the author's supplement fmt.Println(t.String()) // int
The inverse operation of reflect.ValueOf is the reflect.Value.Interface method, which returns an interface {} type and loads the same specific value as reflect.Value:
v := reflect.ValueOf(3) // A reflect.value variable
x := v.Interface() // An interface {} variable
i := x.(int) // an int variable variable complements the author. In addition, interface {} can be asserted as any type
fmt.Printf("%d\n", i) // "3"Both reflect.Value and interface {} can load arbitrary values. The difference is that an empty interface hides the internal representation of the Value and all methods. Therefore, only when we know the specific dynamic type can we use type assertion to access the internal Value (as above), otherwise we can't access the internal Value. In contrast, a Value has many ways to check its content, regardless of its specific type. Let's try to implement our format function format.Any again.
We use the kind method of reflect. Value to replace the previous type switch. Although there are still infinite types, their kind types are limited: Bool, String and the basic types of all numeric types; Aggregation types corresponding to Array and Struct; Chan, Func, Ptr, Slice, and reference types corresponding to Map; interface type; There is also an Invalid type that represents a null value. (the kind of an empty reflect.Value is Invalid.)
// gopl.io/ch12/format
package format
import (
"reflect"
"strconv"
)
// Any formats any value as a string
func Any(value interface{}) string {
return formatAtom(reflect.ValueOf(value))
}
// formatAtom formats a value without inspecting its internal structure
func formatAtom(v reflect.Value) string {
switch v.Kind() {
case reflect.Invalid:
return "invalid"
case reflect.Int, reflect.Int8, reflect,Int16,
reflect.Int32, reflect.Int64:
return strconv.FormatInt(v.Int(), 10)
case reflect.Uint, reflect.Uint8, reflect.Uint16,
refelct.Uint32, reflect.Uint64, reflect.Uintptr:
return strconv.FormatUint(v.Uint(), 10)
// ...floating-point and complex cases omitted for brevity...
case reflect.Bool:
return strconv.FormatBool(v.Bool())
case reflect.String:
return strconv.Quote(v.String())
case reflect.Chan, reflect.Func, reflect.Ptr, reflect.Slice, reflect.Map:
return v.Type().String() + " 0x" + strconv.FormatUint(uint64(v.Pointer()), 16)
default: // reflect.Array, reflect.Struct, reflect.Interface
return v.Type().String() + " value"
}
}So far, our function treats each value as an indivisible object without internal structure, so it is called formaatom. For aggregate types (structures and arrays) and interfaces, only the type of value is printed. For reference types (channels, functions, pointers, slices, and maps), the type and hexadecimal reference address are printed. Although it is not ideal, it is still a significant progress, and Kind only cares about the underlying representation, and format.Any also supports named types. For example:
var x int64 = 1
var d time.Duration = 1 * time.Nanosecond
fmt.Println(format.Any(x)) // "1"
fmt.Println(format.Any(d)) // "1"
fmt.Println(format.Any([]int64{x})) // "[]int64 0x8202b87b0"
fmt.Println(format.Any([]time.Duration{d})) // "[]time.Duration 0x8202b87e0"Display, a recursive value printer
Next, let's look at how to improve the Display of aggregate data types. We don't want to completely copy an fmt.Sprint function. We just build a Display function for debugging: given any complex type x, print the complete structure corresponding to this value, and mark the discovery path of each element.
Let's start with the following example:
e, _ := eval.Parse("sqrt(A / pi)")
Display("e", e)In the above call, the parameter passed into the Display function is the syntax tree returned by an expression evaluation function in section 7.9. The output of the Display function is as follows:
Display e (eval.call): e.fn = "sqrt" e.args[0].type = eval.binary e.args[0].value.op = 47 e.args[0].value.x.type = eval.Var e.args[0].value.x.value = "A" e.args[0].value.y.type = eval.Var e.args[0].value.y.value = "pi"
You should try to avoid exposing interfaces involving reflection in a package's API. We will define an unexported display function for recursive processing. The exported display function is just a simple wrapper of the display function to accept parameters of interface {} type:
// gopl.io/ch12/display
func Display(name string, x interface{}) {
fmt.Printf("Display %s (%T):\n", name, x)
display(name, reflect.ValueOf(x))
}In the display function, we use the previously defined formaatom function to print the element value of the basic type - basic type, function and chan, etc., but we will use the method of reflect.Value to recursively display each member of a complex type. In the recursive descent process, the path string, the starting value passed in from the beginning (here "e"), will gradually increase to indicate how to reach the current value (for example, "e.args[0].value").
Because we no longer simulate the fmt.Sprint function, we will directly use the fmt package to simplify our example implementation.
func display(path string, v reflect.Value) {
switch v.Kind() {
case reflect.Invalid:
fmt.Printf("%s = invalid\n", path)
case reflect.Slice, reflect.Array:
for i:=0; i<v.Len(); i++ {
display(fmt.Spritf("%s[%d]", path, i), v.Index(i))
}
case reflect.Struct:
for i:=0; i<v.NumField(); i++ {
filedPath := fmt.Sprintf("%s.%s", path, v.Type().Field(i).Name)
display(fieldPath, v.Filed(i))
}
case reflect.Map:
for _, key := range v.MapKeys() {
display(fmt.Sprintf("%s[%s]", path,
formatAtom(key)), v.MapIndex(key))
}
case reflect.Ptr:
if v.IsNil() {
fmt.Printf("%s = nil\n", path)
} else {
display(fmt.Sprintf("(*%s)", path), v.Elem())
}
case reflect.Interface:
if v.IsNil {
fmt.Printf("%s = nil\n", path)
} else {
fmt.Printf("%s.type = %s\n", path, v.Elem().Type())
display(path+"value", v.Elem())
}
default: // basic types, channels, funcs
fmt.Printf("%s = %s\n", path, formatAtom(v))
}
}Let's discuss different types: Slice and array: the processing logic of these two data types is the same. The Len method returns the number of elements in slice or array value. Index(i) obtains the element corresponding to index I and returns a reflect.Value; If the index I is out of range, it will cause a panic exception, which is similar to the built-in len(a) and a[i] operations of array or slice types. display recursively calls its own processing for each element in the sequence. We indicate the access path by appending "[i]" to the path during recursive processing.
Although the reflect.Value type has many methods, only a few methods can safely call any value. For example, the Index method can only be called on Slice, array or string type values. If it is called on other types, it will cause panic exception.
Struct: the NumField method reports the number of members in the struct. Field(i) returns the value of the ith member with the type reflect.Value. The member list also includes members promoted through anonymous fields. In order to add ". f" to the path to represent the member path, we must obtain the reflection.type type information corresponding to the structure, and then access the name of the ith member of the structure.
Maps: the Mapkeys method returns a slice of type reflect.Value, and each element corresponds to a key of the map. As usual, the order is random when traversing the map. MapIndex(key) returns the value corresponding to the key in the map. We add "[key]" to the path to represent the access path. (we have an unfinished work here. In fact, the key type of map is not limited to the type that formatAtom can handle perfectly; arrays, structures and interfaces can be used as the key of map.)
Pointer: the variable pointed to by the pointer returned by the Elem method is still of type reflect.Value. Even if the pointer is nil, this operation is safe. In this case, the pointer is of Invalid type, but we can explicitly test a null pointer with the IsNil method, so that we can print more appropriate information. We add "*" before the path and include it in parentheses to avoid ambiguity.
Interface: again, we use the IsNil method to test whether the interface is nil. If not, we can call v.Elem() to obtain the dynamic value corresponding to the interface and print the corresponding type and value.
Now our Display function is finally completed. Let's see its performance. The following Movie types are changed from the Movie types in section 4.5:
type Movie struct {
Title, Subtitle string
Year int
Color bool
Actor map[string]string
Oscars []string
Sequel *string
}Let's declare a variable of this type and see how the Display function displays it:
strangelove := Movie{
Title: "Dr. Strangelove",
Subtitle: "How I Learned to Stop Worrying and Love the Bomb",
Year: 1964,
Color: false,
Actor: map[string]string{
"Dr. Strangelove": "Peter Sellers",
"Grp. Capt. Lionel Mandrake": "Peter Sellers",
"Pres. Merkin Muffley": "Peter Sellers",
"Gen. Buck Turgidson": "George C. Scott",
"Brig. Gen. Jack D. Ripper": "Sterling Hayden",
`Maj. T.J. "King" Kong`: "Slim Pickens",
},
Oscars: []string{
"Best Actor (Nomin.)",
"Best Adapted Screenplay (Nomin.)",
"Best Director (Nomin.)",
"Best Picture (Nomin.)",
},
}The Display("strange love", strange love) call will display (the Chinese name of the strange love movie is Dr. qi'ai):
Display strangelove (display.Movie): strangelove.Title = "Dr. Strangelove" strangelove.Subtitle = "How I Learned to Stop Worrying and Love the Bomb" strangelove.Year = 1964 strangelove.Color = false strangelove.Actor["Gen. Buck Turgidson"] = "George C. Scott" strangelove.Actor["Brig. Gen. Jack D. Ripper"] = "Sterling Hayden" strangelove.Actor["Maj. T.J. \"King\" Kong"] = "Slim Pickens" strangelove.Actor["Dr. Strangelove"] = "Peter Sellers" strangelove.Actor["Grp. Capt. Lionel Mandrake"] = "Peter Sellers" strangelove.Actor["Pres. Merkin Muffley"] = "Peter Sellers" strangelove.Oscars[0] = "Best Actor (Nomin.)" strangelove.Oscars[1] = "Best Adapted Screenplay (Nomin.)" strangelove.Oscars[2] = "Best Director (Nomin.)" strangelove.Oscars[3] = "Best Picture (Nomin.)" strangelove.Sequel = nil
We can also use the Display function to Display the internal structure of types in the standard library, such as * os.File type:
Display("os.Stderr", os.Stderr)The results are as follows:
Output: Display os.Stderr (*os.File): (*(*os.Stderr).file).fd = 2 (*(*os.Stderr).file).name = "/dev/stderr" (*(*os.Stderr).file).nepipe = 0
As you can see, reflection has access to members in the structure that are not exported. It should be noted that the output of this example may be different on different operating systems, and the results may be different with the development of the standard library. (this is one of the reasons why these members are defined as private members!)
We can even use the Display function to Display the internal structure of reflect.Value (set here as the type descriptor of * os.File). The output of the Display("rV", reflect.ValueOf(os.Stderr)) call is as follows. Of course, the results obtained in different environments may be different:
Display rV (reflect.Value): (*rV.typ).size = 8 (*rV.typ).hash = 871609668 (*rV.typ).align = 8 (*rV.typ).fieldAlign = 8 (*rV.typ).kind = 22 (*(*rV.typ).string) = "*os.File" (*(*(*rV.typ).uncommonType).methods[0].name) = "Chdir" (*(*(*(*rV.typ).uncommonType).methods[0].mtyp).string) = "func() error" (*(*(*(*rV.typ).uncommonType).methods[0].typ).string) = "func(*os.File) error" ...
var i interface{} = 3
Display("i", i)The results are as follows:
Output: Display i (int): i = 3
But if it is changed to:
Display("&i", &i)The results are as follows
Output:
Display &i (*interface {}):
(*&i).type = int
(*&i).value = 3In the first example, the Display function calls reflect.ValueOf(i) (author's note: called by the formatAtom function), which returns a Value of type Int. As we mentioned in section 12.2, reflect.ValueOf always returns a specific type of Value because it is extracted from an interface Value.
In the second example, the display function calls reflect. Valueof (& i), which returns a pointer to i, corresponding to Ptr type. In the Ptr branch of switch, call the Elem method on this Value and return a Value to represent the variable i itself, corresponding to the Interface type. Such an indirectly obtained Value may represent any type of Value, including Interface type. The display function recursively calls itself. This time, it prints the dynamic type and Value of the Interface respectively.
For the current implementation, if the object graph contains loops, the Display will fall into an endless loop (Note: because of recursive calls and no explicit termination conditions), such as the following end-to-end linked list:
// a struct that points to itself
type Cycle struct{ Value int; Tail *Cycle } // Note: you can replace Cycle with ListNode, which is easier to understand
var c Cycle
c = Cycle{42, &c} // Author's note: &c is a pointer to itself
DisPlay("c", c)Display will always perform deep recursive printing:
Display c (display.Cycle): c.Value = 42 (*c.Tail).Value = 42 (*(*c.Tail).Tail).Value = 42 (*(*(*c.Tail).Tail).Tail).Value = 42 ...ad infinitum...
Many Go language programs contain some cyclic data. Making the Display support this kind of ring data structure requires some skills. It needs to record the paths accessed so far, which will bring some costs. The general solution is to adopt the language feature of unsafe. We will see the specific solution in section 13.3.
Looped data structures rarely cause problems for the fmt.Sprint function because it rarely attempts to print a complete data structure. For example, when it encounters a pointer, it simply prints the numeric value of the pointer. It may get stuck when printing a slice or map containing itself, but this is rare and is not worth the cost of processing loopback.
Example: encoded as S expression
Display is a debugging tool for displaying structured data, but it cannot encode any Go language object into general messages and then use them for interprocess communication.
As we can see in section 4.5, the standard library of Go language supports a variety of coding formats, including JSON, XML and ASN.1. Another format that is still widely used is the S expression format, which uses the syntax of Lisp language. However, different from other coding formats, the standard library of Go language does not support S expressions, mainly because it does not have a recognized standard specification.
In this section, we will define a package for encoding arbitrary Go language objects into S expression format, which supports the following structures:
42 integer "hello" string (have Go Style quotes) foo symbol (Names not enclosed in quotation marks) (1 2 3) list
Booleans traditionally use the t symbol to represent true and the empty list or nil symbol to represent false, but for simplicity, we ignore Booleans for the time being. Also ignored are chan pipes and functions, because their exact state cannot be known through reflection. We also ignore floating point numbers, complex numbers, and interface s.
We encode the type of Go language into S expression as follows. Integers and strings are encoded in an obvious way. Null values are encoded as nil symbols. Arrays and slice s are encoded as lists.
The structure is encoded as a list of member objects. Each member object corresponds to a sub list with two elements. The first element of the sub list is the name of the member and the second element is the value of the member. The Map is encoded as a list of key value pairs. Traditionally, S expressions use a dot symbol list (key. Value) structure to represent key/value pairs instead of a list with two elements, but we ignore the dot symbol list for simplicity.
The encoding is completed by an encode recursive function, as shown below. Its structure is essentially similar to the previous Display function:
// gopl.io/ch12/sexpr
func encode(buf *bytes.Buffer, v reflect.Value) error {
switch v.Kind() {
case reflect.Invalid:
buf.WriteString(nil)
case reflect.Int, reflect.Int8, reflect.Int16,
reflect.Int32, reflect.Int64:
fmt.Fprintf(buf, "%d", v.Int())
case reflect.Uint, reflect.Uint8, reflect.Uint16,
reflect.Uint32, reflect.Uint64, reflect.Uintptr:
fmt.Fprintf(buf, "%d", v.Int())
case reflect.String:
fmt.Fprintf(buf, "%q", v.String())
case reflect.Ptr:
return encode(buf, v.Elem())
case reflect.Array, reflect.Slice: // (value...) author's note: compound data type recursive call, the same below
buf.WriteByte('(')
for i:=0; i<v.Len(); i++ {
if i>0 {
buf.WriteByte(' ') // Author's note: multiple elements use spaces, the same below
}
if err := encode(buf, v.Index(i)); err != nil {
return err
}
}
buf.WriteByte(')')
case reflect.Struct: // (name value ...)
buf.WriteByte('(')
for i:=0; i<v.numField(); i++ {
if i>0 {
buf.WriteByte(' ')
}
fmt.Fprintf(buf, "(%s ", v.Type().Filed(i).Name) // Note to the author: the value corresponding to the next recursive printing of "(type)" in this printing
if err := encode(buf, v.Field(i)); err != nil {
return err
}
buf.WriteByte(')')
}
buf.WriteByte(')')
case reflect.Map: // ((key value) ...)
buf.WriteByte('(')
for i, key := range v.MapKeys() {
if i > 0 {
buf.WriteByte(' ')
}
buf.WriteByte('(')
if err := encode(buf, key); err != nil {
return err
}
buf.WriteByte(' ')
if err := encode(buf, v.MapIndex(key)); err != nil {
return err
}
buf.WriteByte(')')
}
buf.WriteByte(')') // Note to the author: buf.WriteByte('(') and buf.WriteByte('') always appear in pairs. Don't omit, because parentheses appear in pairs, and the code simulates this process
default:
return fmt.Errof("unsupport type: %s", v.Type())
}
return nil
}Marshal function is a wrapper for encode to keep the API similar to other packages under encoding /
// Marshal encodes a Go value in S-expression form
func Marshal(v interface{}) ([]byte, error) {
var buf bytes.Buffer
if err := encode(&buf, reflect.ValueOf(v)); err != nil { // Author's note: err in any layer of recursive call will be backtracked layer by layer
return nil, err
}
return buf.Bytes(), nil
}The following is the result of Marshal l encoding the strangelove variable in Section 12.3:
((Title "Dr. Strangelove") (Subtitle "How I Learned to Stop Worrying and Lo
ve the Bomb") (Year 1964) (Actor (("Grp. Capt. Lionel Mandrake" "Peter Sell
ers") ("Pres. Merkin Muffley" "Peter Sellers") ("Gen. Buck Turgidson" "Geor
ge C. Scott") ("Brig. Gen. Jack D. Ripper" "Sterling Hayden") ("Maj. T.J. \
"King\" Kong" "Slim Pickens") ("Dr. Strangelove" "Peter Sellers"))) (Oscars
("Best Actor (Nomin.)" "Best Adapted Screenplay (Nomin.)" "Best Director (N
omin.)" "Best Picture (Nomin.)")) (Sequel nil))The entire output is encoded in one line to reduce the size of the output, but it is also difficult to read. The following is the result of manual formatting of the S expression. Writing a beautifying formatting function of S expression will be a challenging exercise task; however http://gopl.io A simple version is also provided.
((Title "Dr. Strangelove")
(Subtitle "How I Learned to Stop Worrying and Love the Bomb")
(Year 1964)
(Actor (("Grp. Capt. Lionel Mandrake" "Peter Sellers")
("Pres. Merkin Muffley" "Peter Sellers")
("Gen. Buck Turgidson" "George C. Scott")
("Brig. Gen. Jack D. Ripper" "Sterling Hayden")
("Maj. T.J. \"King\" Kong" "Slim Pickens")
("Dr. Strangelove" "Peter Sellers")))
(Oscars ("Best Actor (Nomin.)"
"Best Adapted Screenplay (Nomin.)"
"Best Director (Nomin.)"
"Best Picture (Nomin.)"))
(Sequel nil))Similar to fmt.Print, json.Marshal and Display functions, sexpr.Marshal functions will also fall into a dead loop when dealing with data structures with rings (note to the author: because they are essentially implemented by recursion, and there is no explicit recursion termination condition).
In section 12.6, we will give the implementation steps of S expression decoder, but before that, we also need to understand how to update the program variables through reflection technology.
Modify the value through reflect.Value
So far, reflection is just another way to read variables in a program. However, in this section, we will focus on how to modify variables through the reflection mechanism.
Recall that expressions like x, x.f[1] and * p in Go language can represent variables, but others such as x+1 and f(2) are not variables (they are expressions, and the calculation result will return a value). (broadly speaking) a variable (in Implementation) is an addressable memory space in which a value is stored, and its stored value can be updated through the memory address.
There is a similar difference for reflect.Values. Some reflect.Values are desirable addresses; Others cannot. Consider the following declaration statement:
x := 2 // value type variable a := reflect.ValueOf(2) // 2 int no b := reflect.ValueOf(x) // 2 int no c := reflect.ValueOf(&x) // &x *int no d := c.Elem() // 2 int yes(x)
The address of the variable corresponding to a is not acceptable. Because the Value in a is only a copy of the integer 2. The Value in b is also not an address. The Value in c is still not an address, it is just a copy of pointer &x. In fact * *, all reflect. Values returned through reflect.ValueOf(x) are undesirable addresses * *. However, for d, it is generated by dereference of c and points to another variable, so it is addressable. We can get the Value of the desirable address corresponding to any variable x by calling reflect. Valueof (& x). Elem().
We can call the CanAddr method of reflect.Value to determine whether it can be addressed:
fmt.Println(a.CanAddr()) // false fmt.Println(b.CanAddr()) // false fmt.Println(c.CanAddr()) // false fmt.Println(d.CanAddr()) // true
Every time we get the reflect.Value indirectly through the pointer, it is a desirable address, even if it starts with a Value with a non desirable address. In the reflection mechanism, all the rules about whether to support address fetching are similar. For example, slice's index expression e[i] will implicitly contain a pointer, which is a desirable address, even if the initial e expression is not supported. By analogy, the Value corresponding to reflect.ValueOf(e).Index(i) can also be addressed, even if the original reflect.ValueOf(e) does not support it.
(note to the author: a simple value is not addressable. It is copied from the values of other variables. Only the value corresponding to a variable is addressable. The address can be obtained by using the reflection mechanism or the index of aggregate data type)
There are three steps to access a variable from the reflect.Value of its corresponding address:
- The first step is to call the Addr() method, which returns a Value containing a pointer to the variable.
- Then, the Interface() method is called on the Value, that is, an interface {} is returned, which contains a pointer to the variable. 3. Finally, if we know the type of variable, we can use the type assertion mechanism to forcibly convert the interface of the obtained interface {} type to an ordinary type pointer. So we can update the variable through this ordinary pointer.
Examples are as follows:
x := 2 d := reflect.ValueOf(&x).Elem() // D references to the variable note: but it is a reflect.Value value value px := d.Addr().Interface().(*int) // px := &x *px = 3 // x = 3 fmt.Println(x) // 3
Alternatively, instead of using a pointer, update the corresponding value by calling the reflect.Value.Set method with the desired address:
d.Set(reflect.ValueOf(4)) // Note to the author: Set also receives the value of reflect.Value fmt.Println(x) // 4
The Set method performs similar checks for assignability constraints at compile time and runtime. In the above code, variables and values are of type int, but if the variable is of type int64, the program will throw a panic exception. Therefore, the key problem is to ensure that the variable of type change can accept the corresponding value:
d.Set(reflect.ValueOf(int64(5)))
Similarly, calling the Set method on a reflection.value with an undesirable address will also cause a panic exception:
x := 2 b := reflect.ValueOf(x) b.Set(reflect.ValueOf(3)) // panic: Set using unadressable value
There are many Set methods dedicated to basic data types: SetInt, SetUint, SetString and SetFloat.
d := reflect.ValueOf(&x).Elem() d.SetInt(3) fmt.Println(x) // 3
To some extent, these Set methods always complete the task as much as possible. Taking SetInt as an example, as long as the variable is a signed integer of some type, it can work, even some named types, or even as long as the underlying data type is a signed integer, and it will be automatically truncated if the value of the variable type is too large. But be careful: calling SetInt for a reflect.Value that references interface {} type will cause panic exception, even if the interface {} variable is not valid for integer type.
x := 1
rx := reflect.ValueOf(&x).Elem()
rx.SetInt(2) // OK, x=2
rx.Set(reflect.ValueOf(3)) // OK, x=3
rx.Setstring("hello") // pannic: string is not assignable to int
rx.Set(reflect.ValueOf("hello")) // panic: string is not assignable to int
var y interface{}
ry := reflect.ValueOf(&y).Elem()
ry.SetInt(2) // panic: SetInt called on interface Value
ry.Set(reflect.ValueOf(3)) // OK, y = int(3)
ry.SetString("hello") // panic: SetString called on interface Value
ry.Set(reflect.ValueOf("hello")) // OK, y = "hello"When we use Display to Display the os.Stdout structure, we find that reflection can read the non exported members in the structure beyond the restrictions of the Go language's export rules, such as the fd int member in the os.File structure on Unix like systems. However, the reflection mechanism does not modify these non exported members:
stdout := reflect.ValueOf(os.Stdout).Elem() // *os.Stdout, an os.File var
fmt.Println(stdout.Type()) // os.File
fd := stdout.FiledByName("fd")
fmt.Println(fd.Int()) // 1
fd.SetInt(2) // panic: unexported fieldA reflect.Value with a desirable address will record whether a structure member is an unexported member. If so, the modification operation will be rejected. Therefore, the CanAddr method does not correctly reflect whether a variable can be modified. Another related method CanSet is used to check whether the corresponding reflect.Value is a desirable address and can be modified:
fmt.Println(fd.CanAddr(), fd.CanSet()) // true false
The author adds: what is the difference between reflect.ValueOf() and Value.Elem()?
This example is also added to help readers deeply understand the above content. reference resources: https://www.it1352.com/2102902.html
reflect.ValueOf() returns a new Value, which is initialized to a specific Value stored in the interface. In particular, reflect.ValueOf(nil) returns a zero Value.
Value.Elem() can be called on an interface or pointer. If calling an interface, return the value contained in the interface; If called on a pointer, returns the value pointed to by the pointer. If you call a type that is neither an interface nor a pointer, a panic is generated. Similarly, Value.Elem(nil) returns nil.
Note: the value returned by value. Elem() contains type information. You can use Value.Elem().Type() to get the type corresponding to the value.
reflect.ValueOf () is a function that can be seen as an entry to reflection. When we have a common value without reflection function, such as int or string, we can use reflect.ValueOf() to get a descriptor corresponding to the common value (type: reflect.ValueOf value), which has reflection function.
Value.Elem() is a method for a value of type reflect. Value. Therefore, it can only be used if there is already a reflect.Value value value. You can use Value.Elem() to get the value pointed to by the reflect. Value value value (the above description has explained that reflect. Value is a descriptor corresponding to an original value, and Value.Elem gets the original value). Note that reflect.Indirect() can also achieve this. Value.Elem () has another rare high-level usage, which will be introduced at the end of the article.
If you want to preserve the reflection function of the original value, you can use the conventional Value.Interface() method, which returns the wrapped value in the form of interface {}. Examples are as follows:
var i int = 3 var p *int = &i fmt.Println(p, i) // 0x414020 3 v := reflect.ValueOf(p) fmt.Println(v.Interface()) // 0x414020 result or pointer v2 := v.Elem() fmt.Println(v2.Interface()) // 3 the result is a value
reflect.Elem() goes a step further
reflect.Elem returns the value contained in the interface or the value pointed to by the pointer.
Therefore, if reflect.Value wraps an interface value, Value.Elem() can also be used to get the specific value wrapped in the interface value.
Basically, no matter what value is passed to reflect.ValueOf(), it will be converted to the interface value. If it is not yet an interface value, it will be implicitly wrapped in interface {}. If the value passed is already an interface value, the specific value stored therein will be passed as interface {}. If a pointer is passed, it is also wrapped in the interface {} value. You can still use Value.Elem() to get the value pointed to, which will be the interface value (not the concrete value), and you can get the concrete value by using Value.Elem() again.
var r io.Reader = os.Stdin // os.Stdin is a * os.File type that implements the io.Reader interface v := reflect.ValueOf(r) // r is an interface value wrapped with * os.File fmt.Println(v.Type()) // *os.File v2 := reflect.Valueof(&r) // Note: what is passed is a pointer to r, which will be wrapped as an interface value fmt.Println(v2.Type()) // Value saves the type information, i.e.: * io.Reader fmt.Println(v2.Elem().Type()) // Get the value corresponding to the address, that is, the interface value of io.Reader type fmt.Println(v2.Elem().Elem().Type()) // Use Elem twice, and get the specific value of the interface value the second time, that is: * os.File
It may be a little windy. If you don't understand it, first review the concept of interface value: Conceptually, the value of an interface consists of two parts, a specific type and a value of that type. They are called dynamic types and dynamic values of interfaces.
Example: decoding S expression
The Marshal encoding function provided in each package under encoding /... In the standard library has a corresponding Unmarshal function for decoding. For example, as we saw in section 4.5, to decode byte slice data containing JSON encoding format into our own Movie type (§ 12.3), we can do this:
data := []byte{/* ... */} // The data content is not specified
var movie Movie
err := json.Unmarshal(data, &movie)The Unmarshal function uses the reflection mechanism class to modify each member of the movie variable and create the corresponding map, structure and slice for the movie member according to the input content.
Now let'S implement a simple Unmarshal for S expression coding, which is similar to the previous json.Unmarshal standard library function, corresponding to the inverse operation of the sexpr.Marshal function we implemented earlier. We must remind here that a robust and general-purpose implementation usually needs more code than the following example. The example adopts a relatively simplified implementation version for ease of demonstration. We only support a limited subset of S expressions, and the way to deal with errors is rough. The purpose of the code is to demonstrate the use of reflection, not to construct a practical decoder of S expressions.
lexer uses the text/scanner package in the standard library to parse the byte data of the input stream into identifiers such as annotation, identifier, string face value and number face value. The Scan method of the input scanner scanner will Scan ahead and return the next identifier, for the run type. Most identifiers, such as "(", correspond to a Unicode character that can be represented by a single run, but text/scanner can also use a small negative number to represent identifiers, strings and other identifiers composed of multiple characters. Calling Scan method will return the types of these identifiers, and then calling TokenText method will return the text content corresponding to the token.
Because each parser may need to use the current token multiple times, but Scan will Scan forward all the time, we wrap a scanner helper type called lexer to track the identifier recently returned by the Scan method.
// gopl.io/ch12/sexpr
type lexer struct {
scan scanner.Scanner
token rune // the current token
}
func (lex *lexer) next() { lex.token = len.scan.Scan() } // Author's note: move to the next symbol
func (lex *lexer) text() string { return lex.scan.TokenText() } // Author's note: the text corresponding to the current character
func (lex *lexer) consume(want rune) {
if lex.token != want { // note: not an example of good error handling
panic(fmt.Sprintf("got %q, want %q", lex.text(), want))
}
lex.next()
}Now let'S go to the syntax parser. It mainly contains two functions. The first is the read function, which is used to read the current identifier of the S expression, and then update the variable v corresponding to the reflection.value of the desirable address according to the current identifier of the S expression.
func read(lex *lexer, v reflect.Value) {
switch lex.token {
case scanner.Ident:
// The only valid identifiers are "nil" and struct field names
if lex.text() == "nil" {
v.Set(reflect.Zero(v.Type())) // Note: set to the zero value of the corresponding type
lex.next()
return
}
case scanner.String:
s, _ := strconv.Unquote(lex.text()) // note: ignoring errors
v.SetString(s)
lex.next()
return
case scanner.Int:
i, _ := strconv.Atoi(lex.text()) // note: ignoring errors
v.SetInt(int64(i)
lex.next()
return
case '(':
lex.next()
readList(lex, v)
lex.next() // consume ')'
return
}
panic(fmt.Sprintf("unexpected token %q", lex.text()))
}Our S expression uses identifiers to distinguish two different types, structure member name and pointer to nil value. The read function value handles nil type identifiers. When scanner.Ident is "nil", use the reflect.Zero function to set the variable v to zero value. For any other type of identifier (Note: refers to nil or structure member name) , we all handle them as errors. The following readList function will handle the member names of the structure.
The most interesting part is recursion. The simplest is the processing of array types. Until the end tag of ")" is encountered, we use the Index function to obtain the address of each element of the array, and then recursively call the read function. Like other errors, if the input data causes the decoder's reference to exceed the range of the array, the decoder will throw a panic exception. Slice is also parsed in a similar way. The difference is that we will create a new variable for each element, and then add the element to the end of slice.
When processing each element of structure and map in a loop, you must decode a corresponding sublist in (key value) format. For structures, the key part is the name of the member. Similar to arrays, we use FieldByName to find the variables of the corresponding members of the structure, and then recursively call the read function. For a map, the key may be of any type. The processing of elements is similar to slice. We create a new variable, then recursively fill it, and finally add the newly parsed key/value pair to the map.
func readList(lex *lexer, v reflect.Value) {
switch v.Kind() {
case reflect.Array: // (item ...)
for i:=0; !endList(lex); i++ {
read(lex, v.Index(i))
}
}
case reflect.Slice: // (item ...)
for !endlist(lex) {
item := reflect.new(v.Type().Elem()).Elem() // Note to the author: note that after calling Elem twice, you will get the specific value of the interface value. See my supplementary content in the previous section
read(lex, item)
v.Set(reflect.Append(v, item))
}
case reflect.Struct: // ((name value))
for !endList(lex) {
lex.consume('(')
if lex.token != scanner.Ident {
panic(fmt.Sprintf("got token %q, want file name", lex.text()))
}
name := lex.text()
lex.next()
read(lex, v.FieldByName(name))
lex.comsume(')')
}
case reflect.Map: // ((key value) ...)
v.Set(reflect.MakeMap(v.Type())) // Note to the author: set a map of the corresponding type
for !endList(lex) {
lex.consume('(')
key := reflect.New(v.Type().Key()).Elem() // Create a new key
read(lex, key)
value := reflect.New(v.Type().Elem()).Elem() // Create a new value
read(lex, value)
v.SetMapIndex(key, value) // Fill in key and value
lex.consume(')')
}
default:
panic(fmt.Sprintf("cannot decode list into %v", v.Type()))
}
}
func endList(lex *lexer) bool {
switch lex.token {
case scanner.EOF:
panic("end of file")
case ')':
return true
}
return false
}Finally, we wrap the parser as an exported Unmarshal decoding function, hiding some edge processing logic such as initialization and cleaning. The internal parser throws an error in the form of panic, but the Unmarshal function captures the internal panic (§ 5.10) by calling the recover function in the defer statement, and then returns an error message corresponding to the panic.
// Unmarshal parses S-expression data and populates the variable whose address is in the non-nil pointer out
func Unmarshal(data []byte, out interface{}) (err error) {
lex := &lexer{scan: scanner.Scanner{Mode: scanner.GoTokens}}
lex.scan.Init(bytes.NewReader(data))
lex.next()
defer func() {
// note: this is not an example of ideal error handing
if x:=recover(); x!=nil {
err = fmt.Errof("error at %s: %v", lex.scan.Position, x)
}
}()
read(lex, reflect.ValueOf(out).Elem())
return nil
}The implementation in the production environment should not report any input problems in the form of panic, but should report some error related information, such as the line number and location of the wrong input. Nevertheless, we hope to use this example to show the implementation idea of the underlying code of packages such as encoding/json, and how to use reflection mechanism to fill the data structure.
Get structure field label
In section 4.5, we use the constructor member tag to set the corresponding name of the corresponding JSON. The JSON member tag allows us to select the name of the member and suppress the output of zero valued members. In this section, we will see how to get the member tag through the reflection mechanism class.
For a web service, the first thing most HTTP handlers do is to expand the parameters in the request to local variables. We define a tool function called params.Unpack, which makes it easier for the HTTP processing function to parse request parameters by using the structure member label mechanism.
First, let's see how to use it. The following search function is an HTTP request processing function. It defines a variable of anonymous structure type, and each member of the structure is used to represent the parameters of HTTP request. The structure member tag indicates the name of the request parameters. In order to reduce the length of the URL, these parameter names are usually mysterious abbreviations. Unpack populates the request parameters into the appropriate structure members so that we can easily access these parameters through the appropriate type classes.
// gopl.io/ch12/search
import "gopl.io/ch12/parms"
// search implements the /search URL endpoint.
func search(resp http.ResponserWriter, req *http.Request) {
var data struct {
Labels []string `http:"l"`
MaxResults int `http:max`
Exact bool `http:x`
}
data.MaxResult = 10 // set default
if err := params.Upack(req, &data); err != nil {
http.Error(resp, err.Error(), http.StatusBadRequest)
}
// ... rest of handler ...
fmt.Fprintf(resp, "Search: %+v\n", data)
}Note to the author: "search:% + V \ n" means to keep the output value of the structure field name. Similar usage also #v, not only keep the structure field name, but also add double quotation marks on the field value.
Next, the Unpack function will build the mapping from the valid parameter name of each structure member to the member variable. If the structure member has a member label, the valid parameter name may be different from the actual member name. The Field method of reflect.Type will return a reflect.StructField containing the name, type and optional member label of each member. The member tag information corresponds to a string of type reflect.StructTag, and a Get method is provided to parse and extract a substring according to a specific key, such as a substring in the form of http: "..." here.
// gopl.io/ch12/params
// Unpack populates the fields of the struct pointed to by ptr from the HTTP request parameterss in req.
func Unpack(req *http.Request, ptr interface{}) error {
if err := req.ParseForm(); err != nil {
return err
}
// Build map of fileds keyed by effective name
fields := make(map[string]reflect.Value)
v := relfect.ValueOf(ptr).Elem() // the struct variable
for i:=0; i<v.NumField(); i++ {
fieldInfo := v.Type().Field(i) // a reflect.StructFiled
tag := fieldInfo.Tag // a reflect.StructTag is equivalent to v.Type().Field(i).Tag gets the contents of ` `
name = tag.Get("http")
if name == "" {
name = string.ToLower(fieldInfo.Name)
}
fields[name] = v.Field(i)
}
// Update struct field for each parameter in the request
for name, values := range req.Form {
f = fields[name]
if !.IsValid() {
continue // ignore unrecognized HTTP parameters
}
for _, value := range values {
if f.Kind() == reflect.Slice {
elem := reflect.New(f.Type().Elem()).Elem() // Author's note: pay attention to the position of brackets
if err := populate(elem, value); err != nil {
return fmt.Errof("%s: %v", name, err)
}
f.Set(reflect.Append(f, elem))
} else {
if err := populate(f, value); err != nil {
return fmt.Errof("%s: %v", name, err)
}
}
}
}
return nil
}Finally, Unpack traverses the name / value parameter key value pair of the HTTP request and updates the corresponding structure members according to the. Recall that parameters with the same name can appear multiple times. If this happens and the corresponding structure member is a slice, all parameters are added to the slice. In other cases, the corresponding member value will be overwritten, and only the last parameter value will work.
The populate function carefully populates a single member v (or a single element in a slice type member) with the requested string type parameter value. Currently, it only supports strings, signed integers, and Booleans.)
func populate(v reflect.Value, value string) error {
switch v.Kind() {
case reflect.String:
v.SetString(value)
case reflect.Int:
i, err := strconv.ParseInt(value, 10, 64)
if err != nil {
return err
}
v.SetInt(i)
case reflect.Bool:
b, err := strconv.ParseBool(value)
if err != nil {
return nil
}
v.SetBool(b)
default:
return fmt.Errorf("unsupported kind %s", v.Type())
}
return nil
}If we add the above handler to a web server, we can generate the following sessions:
$ go build gopl.io/ch12/search
$ ./search &
$ ./fetch 'http://localhost:12345/search'
Search: {Labels:[] MaxResults:10 Exact:false}
$ ./fetch 'http://localhost:12345/search?l=golang&l=programming'
Search: {Labels:[golang programming] MaxResults:10 Exact:false}
$ ./fetch 'http://localhost:12345/search?l=golang&l=programming&max=100'
Search: {Labels:[golang programming] MaxResults:100 Exact:false}
$ ./fetch 'http://localhost:12345/search?x=true&l=golang&l=programming'
Search: {Labels:[golang programming] MaxResults:10 Exact:true}
$ ./fetch 'http://localhost:12345/search?q=hello&x=123'
x: strconv.ParseBool: parsing "123": invalid syntax
$ ./fetch 'http://localhost:12345/search?q=hello&max=lots'
max: strconv.ParseInt: parsing "lots": invalid syntaxDisplays the method set of a type
Our last example is to use reflect.Type to print the type of any value and the method of enumerating it:
// Print prints the method set of the value x
func Print(x interface{}) {
v := reflect.ValueOf(x)
t := v.Type()
fmt.Printf("type %s\n", t)
for i:=0; i<v.NumMethod(); i++ {
methType := v.Method(i).Type()
fmt.Printf("func (%s) %s%s\n", t, t.Method(i).Name,
strings.TrimPrefix(methType.String(), "func"))
}
}Both reflect.Type and reflect.Value provide a method method method. Each t.Method(i) call will return an instance of reflect.Method, which corresponds to a structure used to describe the name and type of a method. Each Method(i) method call returns a reflect.Value to represent the corresponding value (§ 6.4), that is, the receiver of a method binding. Using the reflect.Value.Call method (not shown here), you can call a value of Func type, but only its type is used in this example.
This is a method of type time.Duration and * strings. Replace:
methods.Print(time.Hour)
The printing results are as follows:
Output: type time.Duration func (time.Duration) Hours() float64 func (time.Duration) Minutes() float64 func (time.Duration) Nanoseconds() int64 func (time.Duration) Seconds() float64 func (time.Duration) String() string
methods.Print(new(strings.Replacer))
Output: type *strings.Replacer func (*strings.Replacer) Replace(string) string func (*strings.Replacer) WriteString(io.Writer, string) (int, error)
Some advice
Although reflection provides much more API s than we mentioned, our previous examples mainly give a direction and what functions can be realized through reflection.
Reflection is a powerful and expressive tool, but it should be used carefully for three reasons:
- Reflection based code is fragile
- Even if the corresponding type provides documents, the reflected operation cannot be statically type checked, and a large number of reflected code is usually difficult to understand.
- Reflection based code usually runs one or two orders of magnitude slower than normal code.
The first reason is that reflection based code is fragile. For each problem that will cause the compiler to report type errors, there are corresponding misuse problems in reflection. The difference is that the compiler will report errors immediately during construction, while reflection will throw panic exceptions only when it really runs. It may be a long time after writing the code, and the program may run for a long time.
Taking the previous readList function (§ 12.6) as an example, the reflect.Value.SetString method called to read a string from the input and fill in a variable of type int may cause a panic exception. Most programs that use reflection have similar risks. You need to carefully check the type of the corresponding value of each reflect.Value, whether it can be addressed, and whether it can be modified.
The best way to avoid this vulnerability caused by reflection is to control all reflection related uses within the package. If possible, avoid directly exposing the reflection.value type in the package API, so as to limit some illegal input; If this is not possible, perform additional type checks before each risky operation. Take the code in the standard library as an example. When fmt.Printf receives an illegal operand, it will not throw a panic exception, but print the relevant error message. Although there are bugs in the program, it will be easier to diagnose.
fmt.Printf("%d %s\n", "hello", 42) // "%d(string=hello) %s(int=42)"Reflection also reduces the security of programs and affects the accuracy of automated refactoring and analysis tools because they cannot recognize the type information that can only be confirmed at run time.
The second reason is that even if documents are provided for the corresponding types, reflected operations cannot be statically type checked, and a large number of reflected code is usually difficult to understand. Therefore, you always need to carefully maintain documentation for each exported type and other functions that accept parameters of interface {} or reflect.Value type.
The third reason is that reflection based code usually runs one or two orders of magnitude slower than normal code. For a typical project, the performance of most functions has little to do with the overall performance of the program, so it can be considered when reflection can make the program clearer. Testing is a particularly suitable scenario for using reflection because the data set of each test is very small. However, for performance critical path functions, it is best to avoid reflection.