preface

The previous article introduced how to achieve gRPC load balancing, but at present, only pick is officially provided_ First and round_ Two load balancing strategies of Robin, round_robin can't meet the requirements of different server configurations, so this article will introduce how to implement a custom load balancing strategy called weighted random method.

The weighted random method can assign different weights according to the processing ability of the server, so that the server with high processing ability can bear more requests, and the server with low processing ability can bear less requests.

Custom load balancing policy

gRPC provides V2PickerBuilder and V2Picker interfaces for us to implement our own load balancing strategy.

type V2PickerBuilder interface {
	Build(info PickerBuildInfo) balancer.V2Picker
}

V2PickerBuilder interface: creates a V2 version of the child connection selector.

Build method: returns a V2 selector that will be used for gRPC to select child connections.

type V2Picker interface {
	Pick(info PickInfo) (PickResult, error)
}

V2Picker interface: used for gRPC to select sub connections to send requests.
Pick method: sub connection selection

The problem is, we need to add the weight of the server address, but the address resolver.Address There is no attribute that provides weight. The official reply is: store the weight in the metadata of the address.

// attributeKey is the type used as the key to store AddrInfo in the Attributes
// field of resolver.Address.
type attributeKey struct{}

// AddrInfo will be stored inside Address metadata in order to use weighted balancer.
type AddrInfo struct {
	Weight int
}

// SetAddrInfo returns a copy of addr in which the Attributes field is updated
// with addrInfo.
func SetAddrInfo(addr resolver.Address, addrInfo AddrInfo) resolver.Address {
	addr.Attributes = attributes.New()
	addr.Attributes = addr.Attributes.WithValues(attributeKey{}, addrInfo)
	return addr
}

// GetAddrInfo returns the AddrInfo stored in the Attributes fields of addr.
func GetAddrInfo(addr resolver.Address) AddrInfo {
	v := addr.Attributes.Value(attributeKey{})
	ai, _ := v.(AddrInfo)
	return ai
}

Define the AddrInfo structure and add the Weight attribute. The Set method stores the Weight in the resolver.Address , get method from resolver.Address Get Weight.

After solving the problem of weight storage, we implement the load balancing strategy of weighted random method.

First, implement the V2PickerBuilder interface and return the sub connection selector.

func (*rrPickerBuilder) Build(info base.PickerBuildInfo) balancer.V2Picker {
	grpclog.Infof("weightPicker: newPicker called with info: %v", info)
	if len(info.ReadySCs) == 0 {
		return base.NewErrPickerV2(balancer.ErrNoSubConnAvailable)
	}
	var scs []balancer.SubConn
	for subConn, addr := range info.ReadySCs {
		node := GetAddrInfo(addr.Address)
		if node.Weight <= 0 {
			node.Weight = minWeight
		} else if node.Weight > 5 {
			node.Weight = maxWeight
		}
		for i := 0; i < node.Weight; i++ {
			scs = append(scs, subConn)
		}
	}
	return &rrPicker{
		subConns: scs,
	}
}

In the weighted random method, I use the method of space for time to convert the weight into the number of addresses (for example, if the weight of addr1 is 3, then add 3 sub links to the slice; if the weight of addr2 is 1, then add 1 sub link; when selecting the sub link, generate the random number according to the length of the sub link slice, using the random number as the subscript is the selected sub link), so as to avoid repeated calculation of the weight. Considering the memory usage, the weight is defined from 1 to 5.

Next, the sub connection selection is realized, the random number is obtained, and the sub connection is selected

type rrPicker struct {
	subConns []balancer.SubConn
	mu sync.Mutex
}

func (p *rrPicker) Pick(balancer.PickInfo) (balancer.PickResult, error) {
	p.mu.Lock()
	index := rand.Intn(len(p.subConns))
	sc := p.subConns[index]
	p.mu.Unlock()
	return balancer.PickResult{SubConn: sc}, nil
}

After the completion of the key code, we name the weighted random load balancing strategy as weight and register it in the load balancing strategy of gRPC.

// Name is the name of weight balancer.
const Name = "weight"
// NewBuilder creates a new weight balancer builder.
func newBuilder() balancer.Builder {
	return base.NewBalancerBuilderV2(Name, &rrPickerBuilder{}, base.Config{HealthCheck: false})
}

func init() {
	balancer.Register(newBuilder())
}

Full code weight.go

Finally, we only need to attach the weight when the server registers the service, and then the client sets the weight to resolver.Address Finally, the client changes the load balancing strategy to weight.

//SetServiceList set service address
func (s *ServiceDiscovery) SetServiceList(key, val string) {
	s.lock.Lock()
	defer s.lock.Unlock()
	//Get service address
	addr := resolver.Address{Addr: strings.TrimPrefix(key, s.prefix)}
	//Get service address weight
	nodeWeight, err := strconv.Atoi(val)
	if err != nil {
		//The default weight of non numeric characters is 1
		nodeWeight = 1
	}
	//Store service address weight in resolver.Address In the metadata of
	addr = weight.SetAddrInfo(addr, weight.AddrInfo{Weight: nodeWeight})
	s.serverList[key] = addr
	s.cc.UpdateState(resolver.State{Addresses: s.getServices()})
	log.Println("put key :", key, "wieght:", val)
}

Client uses weight load balancing strategy

func main() {
	r := etcdv3.NewServiceDiscovery(EtcdEndpoints)
	resolver.Register(r)
	// Connect to server
	conn, err := grpc.Dial(
		fmt.Sprintf("%s:///%s", r.Scheme(), SerName),
		grpc.WithBalancerName("weight"),
		grpc.WithInsecure(),
	)
	if err != nil {
		log.Fatalf("net.Connect err: %v", err)
	}
	defer conn.Close()

Operation effect:

Run service 1 with weight of 1

Run service 2 with a weight of 4

Run client

Check the load of the first 50 requests on service 1 and server 2. Service 1 allocated 9 requests, and service 2 allocated 41 requests, close to the weight ratio.

Disconnect service 2, all requests flow to service 1

Restart service 2 with weight 4, and request flows to two servers with weight random method

summary

This paper takes the weighted random method as an example, introduces how to implement the custom load balancing strategy of gRPC to meet our needs.

Source address: https://github.com/Bingjian-Zhu/etcd-example