In order to know more clearly about a distributed Web pressure measurement implementation, we gradually implement it from single user - > single user - > multi user - > distributed.
(1) What is web stress testing?
(2) Several Important Indicators in Pressure Testing
(3) Single-machine and single-user pressure measurement with Go language
(4) GO Language for Single Computer Multi-user Pressure Measurement
(5) Distributed Pressure Measurement with Go Language
(6) Relevant references
First, what is web stress testing?
Simply put, test how many requests a web site can support (that is, the maximum concurrency of a web site)
2. Several Important Indicators of Stress Testing
1) What are the indicators?
2) Explanation of the Meaning of Indicators
https://blog.csdn.net/adparking/article/details/45673315
3. The Realization of Single Computer and Single User Pressure Measurement Go
1) Some Concepts to Know
Number of concurrent connections:
Understanding: The number of concurrent connections is not equal to the number of concurrent connections. The real number of concurrent connections can only be calculated in the server. The number of concurrent connections on this side is equal to the sum of the number of concurrent connections in the state from the request to the information received from the server.
Total number of requests
response time
Average response time
Number of successes
Number of failures
2) Code implementation
package main import ( "fmt" "log" "net/http" "os" "strconv" "sync" "time" ) var ( SBCNum int // Number of concurrent connections QPSNum int // Total number of requests RTNum time.Duration // response time RTTNum time.Duration // Average response time SuccessNum int // Number of successes FailNum int // Number of failures BeginTime time.Time // start time SecNum int // Number of seconds RQNum int // Maximum concurrency, passed in from the command line Url string // url, passed in from the command line controlNum chan int // Control the number of concurrencies ) var mu sync.Mutex // Must be locked func init() { if len(os.Args) != 3 { log.Fatal("Number of requests url") } RQNum, _ = strconv.Atoi(os.Args[1]) controlNum = make(chan int, RQNum) Url = os.Args[2] } func main() { go func() { for range time.Tick(1 * time.Second) { SecNum++ fmt.Printf("Concurrent number:%d,Number of requests:%d,Average response time:%s,Number of successes:%d,Number of failures:%d\n", len(controlNum), SuccessNum+FailNum, RTNum/(time.Duration(SecNum)*time.Second), SuccessNum, FailNum) } }() requite() } func requite() { for { controlNum <- 1 go func(c chan int) { var tb time.Time var el time.Duration for { tb = time.Now() _, err := http.Get(Url) if err == nil { el = time.Since(tb) mu.Lock() // Lock up SuccessNum++ RTNum += el mu.Unlock() // Unlock } else { mu.Lock() // Lock up FailNum++ mu.Unlock() // Unlock } time.Sleep(1 * time.Second) } <- c }(controlNum) time.Sleep(45 * time.Millisecond) } }
4. Realization of Single Computer Multi-user Pressure Measurement Go
package main import ( "fmt" "log" "net/http" "os" "strconv" "sync" "time" ) var ( BeginTime time.Time // start time SecNum int // Number of seconds RQNum int // Maximum concurrency, passed in from the command line Url string // url, passed in from the command line userNum int // Number of users ) var users []User type User struct { UserId int // User id SBCNum int // Number of concurrent connections QPSNum int // Total number of requests RTNum time.Duration // response time RTTNum time.Duration // Average response time SuccessNum int // Number of successes FailNum int // Number of failures mu sync.Mutex } func (u *User) request(url string) { var tb time.Time var el time.Duration for i := 0;i < u.QPSNum;i++ { u.SBCNum++ go func(u *User) { for { tb = time.Now() _, err := http.Get(Url) if err == nil { el = time.Since(tb) u.mu.Lock() // Lock up u.SuccessNum++ u.RTNum += el u.mu.Unlock() // Unlock } else { u.mu.Lock() // Lock up u.FailNum++ u.mu.Unlock() // Unlock } time.Sleep(1 * time.Second) } }(u) } } func (u *User) show() { fmt.Printf("user id: %d,Concurrent number:%d,Number of requests:%d,Average response time:%s,Number of successes:%d,Number of failures:%d\n", u.UserId, u.SBCNum, u.SuccessNum + u.FailNum, u.RTNum/(time.Duration(SecNum)*time.Second), u.SuccessNum, u.FailNum) } func showAll(us []User) { uLen := len(us) var SBCNum int // Number of concurrent connections var RTNum time.Duration // response time var SuccessNum int // Number of successes var FailNum int // Number of failures for i := 0;i < uLen;i++ { SBCNum += us[i].SBCNum SuccessNum += us[i].SuccessNum FailNum += us[i].FailNum RTNum += us[i].RTNum us[i].show() } fmt.Printf("Concurrent number:%d,Number of requests:%d,Average response time:%s,Number of successes:%d,Number of failures:%d\n", SBCNum, SuccessNum+FailNum, RTNum/(time.Duration(SecNum)*time.Second), SuccessNum, FailNum) fmt.Println() } func init() { if len(os.Args) != 4 { log.Fatal("Number of user requests url") } userNum, _ = strconv.Atoi(os.Args[1]) RQNum, _ = strconv.Atoi(os.Args[2]) Url = os.Args[3] users = make([]User, userNum) } func main() { go func() { for range time.Tick(2 * time.Second) { SecNum += 2 showAll(users) } }() for range time.Tick(1 * time.Second) { requite() } } func requite() { c := make(chan int) temp := 0 for i := 0;i < userNum;i++ { if RQNum % userNum != 0 && i < RQNum % userNum { temp = 1 } else { temp = 0 } users[i].UserId = i users[i].QPSNum = RQNum / userNum + temp go users[i].request(Url) time.Sleep(45 * time.Millisecond) } <- c // block }
V. Distributed Pressure Measurement Go
Divide the master node and the slave node, and now implement the following functions separately
1) Main Node Function
Collect pressure measurement information from slave nodes
Display piezometric information
2) Slave Node Function
Send the piezometric information to the master node
3) The whole working principle
A master node starts, sets up a listening port, and uses TCP mode to start several slave nodes. Each slave node connects to the master node through an IP + port. After that, the master node records the information of the slave nodes connected to it. The slave node sends the relevant information to the master node, which displays the information at a set time.
Code implementation:
Main Node Code Implementation
package main import ( "log" "net" "time" "os" "encoding/json" "fmt" ) var ip string var port string var slaves []*slave type slave struct { UserId string SBCNum int // Number of concurrent connections QPSNum int // Total number of requests RTNum time.Duration // response time RTTNum time.Duration // response time SecNum int // time SuccessNum int // Number of successes FailNum int // Number of failures Url string conn net.Conn } func (s *slave) Run() { var v interface{} buf := make([]byte, 1024) for { n, err := s.conn.Read(buf) if err != nil { log.Println(err) break } err = json.Unmarshal(buf[:n], &v) if err != nil { log.Println(err) continue } s.SBCNum = int(v.(map[string]interface{})["SBCNum"].(float64)) // Number of concurrent connections s.RTNum = time.Duration(v.(map[string]interface{})["RTNum"].(float64)) // response time s.SuccessNum = int(v.(map[string]interface{})["SuccessNum"].(float64)) //Success Num int//Number of Success s.FailNum = int(v.(map[string]interface{})["FailNum"].(float64)) //FailNum int//Number of failures s.SecNum = int(v.(map[string]interface{})["SecNum"].(float64)) } } func init() { if len(os.Args) != 3 { log.Fatal(os.Args[0] + " ip port") } ip = os.Args[1] port = os.Args[2] } func main() { s, err := net.Listen("tcp", ip + ":" + port) if err != nil { log.Fatal(err) } defer s.Close() buf := make([]byte, 128) fmt.Println("Run...") go func() { for range time.Tick(2 * time.Second) { show(slaves) } }() for { conn, err := s.Accept() if err != nil { log.Println(err) continue } n, err := conn.Read(buf) tempC := slave{conn:conn,UserId:conn.RemoteAddr().String(), Url:string(buf[:n])} go tempC.Run() slaves = append(slaves, &tempC) } } func show(clients []*slave) { if len(clients) == 0 { return } temp := slave{} num := 0 for _, client := range clients { if client.SecNum == 0 { continue } num++ fmt.Printf("user id: %s,url: %s,Concurrent number:%d,Number of requests:%d,Average response time:%s,Number of successes:%d,Number of failures:%d\n", client.UserId, client.Url, client.SBCNum, client.SuccessNum + client.FailNum, client.RTNum / (time.Duration(client.SecNum) * time.Second), client.SuccessNum, client.FailNum) temp.SBCNum += client.SBCNum temp.RTNum += client.RTNum / (time.Duration(client.SecNum) * time.Second) temp.SecNum += client.SecNum temp.SuccessNum += client.SuccessNum temp.FailNum += client.FailNum } if num == 0 { return } fmt.Printf("Concurrent number:%d,Number of requests:%d,Average response time:%s,Number of successes:%d,Number of failures:%d\n", temp.SBCNum, temp.SuccessNum + temp.FailNum, temp.RTNum / time.Duration(num), temp.SuccessNum, temp.FailNum) fmt.Println() } func heartbeat(clients []slave) []slave { // Label Coupling tempC := []slave{} for _, client := range clients { _, err := client.conn.Write([]byte("")) if err == nil { // delete tempC = append(tempC, client) } } return tempC }
Slave node
package main import ( "net" "github.com/lunny/log" "encoding/json" "time" "os" "net/http" "sync" "strconv" "fmt" ) type master struct { ip string port string conn net.Conn } var ( SBCNum int // Number of concurrent connections QPSNum int // Total number of requests RTNum time.Duration // response time RTTNum time.Duration // Average response time SuccessNum int // Number of successes FailNum int // Number of failures SecNum int mt master err error mu sync.Mutex // Must be locked RQNum int // Maximum concurrency, passed in from the command line Url string // url, passed in from the command line ) func init() { if len(os.Args) != 5 { log.Fatalf("%s Concurrent Number url ip port", os.Args[0]) } RQNum, err = strconv.Atoi(os.Args[1]) if err != nil { log.Println(err) } Url = os.Args[2] mt.ip = os.Args[3] mt.port = os.Args[4] } func main() { mt.conn, err = net.Dial("tcp", mt.ip + ":" + mt.port) if err != nil { log.Fatal(err) } fmt.Println("The connection to the server was successful...") _, err = mt.conn.Write([]byte(Url)) if err != nil { log.Println(err) } go func() { for range time.Tick(1 * time.Second) { sendToMaster(mt, map[string]interface{}{ "SBCNum": SBCNum, // Number of concurrent connections "RTNum": RTNum, // response time "SecNum": SecNum, // time "SuccessNum": SuccessNum, // Number of successes "FailNum": FailNum, // Number of failures }) } }() go func() { for range time.Tick(1 * time.Second) { SecNum++ } }() requite(RQNum, Url) } func requite(RQNum int, url string) { c := make(chan int) for i := 0;i < RQNum;i++ { SBCNum = i + 1 go func(url string) { var tb time.Time var el time.Duration for { tb = time.Now() _, err := http.Get(url) if err == nil { el = time.Since(tb) mu.Lock() // Lock up SuccessNum++ RTNum += el mu.Unlock() // Unlock } else { mu.Lock() // Lock up FailNum++ mu.Unlock() // Unlock } time.Sleep(1 * time.Second) } }(url) time.Sleep(45 * time.Millisecond) } <- c // block } func sendToMaster(mt master, data map[string]interface{}) { r, err := json.Marshal(data) if err != nil { log.Println(err) } _, err = mt.conn.Write(r) if err != nil { log.Println(err) os.Exit(1) } }
Reference link:
The Concept of Pressure Measurement Index
(1)https://www.cnblogs.com/shijingjing07/p/6507317.html
(2)https://blog.csdn.net/adparking/article/details/45673315
github link: https://github.com/laijinhang/WebRequest
If something goes wrong, please point out that the code has been uploaded to github. Welcome to modify and improve it.