Test purpose

Because of K8s, I dabbled in go language and found that there are many web frameworks of golang, which are called high-performance benchmarks. I have been committed to the research of c + + high-performance server framework before. Out of curiosity, I want to objectively compare the many web frameworks of go from the performance level alone. On the other hand, I want to see how different the implementation of c + + is from that of go language.

There are many evaluation indexes of high-performance service framework, but generally speaking, throughput and QPS are the key consideration indexes. Throughput measures bandwidth utilization. QPS mainly tests the scheduling performance of the framework (almost all service frameworks that can be called "high performance" have no throughput problem, after all, the network bottleneck is easily reached). Because it is the QPS test of the framework itself, in order to shield the differences in http protocol implementation, the most concise protocol header is selected (protocol processing generally does not have lock, which is cpu intensive). Therefore, the request / return message is required to be as small as possible. This test is based on http protocol and returns empty message.

In order to achieve the second purpose of the test, a self rolling c + + service framework is added to the comparison test as the "brick" of the c + + implementation. This framework has not yet been open-source, and its high-performance feature guarantee is reflected in the following design:

• Realize socket multiplexing across platforms, support: poll, epoll, kqueue, port, select, IOCP and other models
• Using lock free algorithm
• Thread pool design
• socket connection pool
• Multi level task queue
  ......

PS:

Well, this seems to be closer to testing the scheduling performance of the socket service framework...
Don't worry about keep alive, because wrk uses HTTP/1.1, which is always keep alive by default.

testing environment

Environment settings

ulimit -n 2000

Pressure measuring tool

wrk
Due to the environment limitation, only wrk client and the server to be tested can run on one machine

c + + self-developed framework

• Startup script: (maximum 2000 concurrent connections, 2 threads processing, http port 8080)
  ./proxy_server -i2000 -o2000 -w2 -x8080
• If you have conditions to test linux system, you can download the server (select the package of the corresponding platform): github.com/lazy-luo/smartate
• http return message:

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Content-Length: 0
Connection: keep-alive

• Pressure test results:

$wrk -d 100s -c 1024 -t 8 http://localhost:8080/
Running 2m test @ http://localhost:8080/
  8 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    13.03ms    3.80ms 100.73ms   86.97%
    Req/Sec     9.43k     1.64k   39.35k    88.23%
  7509655 requests in 1.67m, 444.03MB read
  Socket errors: connect 0, read 794, write 2, timeout 0
Requests/sec:  75018.11
Transfer/sec:      4.44MB

• Resource usage:
  

Go restful framework:

• main_go-restful.go

package main
import (
   "github.com/emicklei/go-restful"
   "net/http"
)
func main(){
    ws := new(restful.WebService)
    ws.Route(ws.GET("/").To(hello))
    restful.Add(ws)
    http.ListenAndServe(":8080",nil)
}
func hello(req *restful.Request,resp *restful.Response){
    resp.Write([]byte(""))
}

• http return message:

$curl -i http://localhost:8080/
HTTP/1.1 200 OK
Date: Mon, 21 Oct 2019 03:54:27 GMT
Content-Length: 0

• Pressure test results:

$wrk -d 100s -c 1024 -t 8 http://localhost:8080/
Running 2m test @ http://localhost:8080/
  8 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    19.72ms   10.57ms 331.94ms   87.67%
    Req/Sec     6.52k     1.24k   23.75k    80.42%
  5180908 requests in 1.67m, 370.57MB read
  Socket errors: connect 0, read 844, write 3, timeout 0
Requests/sec:  51757.61
Transfer/sec:      3.70MB

• Resource usage:
  

Go echo framework:

• main_go-echo.go

package main
import (
    "net/http"
    "github.com/labstack/echo"
)
func main() {
    e := echo.New()
    e.GET("/", func(c echo.Context) error {
        return c.String(http.StatusOK, "")
    })
    e.Logger.Fatal(e.Start(":8080"))
}

• http return message:

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Date: Mon, 21 Oct 2019 04:09:24 GMT
Content-Length: 0

• Pressure test results:

$ wrk -d 100s -c 1024 -t 8 http://localhost:8080/
Running 2m test @ http://localhost:8080/
  8 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    17.32ms    8.19ms 252.60ms   90.70%
    Req/Sec     7.52k     1.35k   39.96k    80.55%
  5974370 requests in 1.67m, 660.92MB read
  Socket errors: connect 0, read 431, write 67, timeout 0
Requests/sec:  59686.09
Transfer/sec:      6.60MB

• Resource usage:
  

Go iris framework:

• main_go-iris.go

package main
import(
    "time"
    "github.com/kataras/iris"
    "github.com/kataras/iris/cache"
)
func main(){
    app := iris.New()
    app.Logger().SetLevel("error")
    app.Get("/",cache.Handler(10*time.Second),writeMarkdown)
    app.Run(iris.Addr(":8080"))
}
func writeMarkdown(ctx iris.Context){
    ctx.Markdown([]byte(""))
}

• http return message:

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Date: Mon, 21 Oct 2019 04:11:59 GMT
Content-Length: 0

• Pressure test results:

$ wrk -d 100s -c 1024 -t 8 http://localhost:8080/
Running 2m test @ http://localhost:8080/
  8 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    22.03ms    7.99ms 140.47ms   84.58%
    Req/Sec     5.79k   775.23    19.31k    80.35%
  4608572 requests in 1.67m, 505.43MB read
  Socket errors: connect 0, read 726, write 22, timeout 0
Requests/sec:  46041.23
Transfer/sec:      5.05MB

• Resource usage:
  

go-gin framework

• main_go-gin.go

package main
import (
    "fmt"
    "net/http"
    "log"
    "github.com/julienschmidt/httprouter"
)
func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    fmt.Fprint(w, "")
}
func main() {
    router := httprouter.New()
    router.GET("/", Index)
    log.Fatal(http.ListenAndServe(":8080", router))
}

• http return message:

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Date: Mon, 21 Oct 2019 04:15:33 GMT
Content-Length: 0

• Pressure test results:

$ wrk -d 100s -c 1024 -t 8 http://localhost:8080/
Running 2m test @ http://localhost:8080/
  8 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.71ms    7.72ms 268.45ms   87.79%
    Req/Sec     7.71k     1.58k   21.27k    82.12%
  6130281 requests in 1.67m, 438.47MB read
  Socket errors: connect 0, read 693, write 36, timeout 0
Requests/sec:  61243.74
Transfer/sec:      4.38MB

• Resource usage:
  

Go Chi framework:

• main_go-chi.go

package main
import (
    "net/http"
    "github.com/go-chi/chi"
)
func main() {
    r := chi.NewRouter()
    r.Get("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte(""))
    })
    http.ListenAndServe(":8080", r)
}

• http return message:

$ curl -i http://localhost:8080/
HTTP/1.1 200 OK
Date: Mon, 21 Oct 2019 04:18:42 GMT
Content-Length: 0

• Pressure test results:

$ wrk -d 100s -c 1024 -t 8 http://localhost:8080/
Running 2m test @ http://localhost:8080/
  8 threads and 1024 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    17.17ms    8.47ms 253.47ms   90.07%
    Req/Sec     7.65k     1.42k   26.08k    79.76%
  6071695 requests in 1.67m, 434.28MB read
  Socket errors: connect 0, read 110, write 2, timeout 0
Requests/sec:  60658.49
Transfer/sec:      4.34MB

• Resource usage:
  

Conclusion:

• In the go language web framework, gin, chi and echo have the same performance, while gin has a slight advantage, while iris has a poor measurement effect.
• There is a certain performance gap between go and c + + network framework, but it is not decisive.
• go language consumes a lot of resources and c + + is light and efficient enough.
• go language is really easy to use and simple!! It's just that the three parties rely too much on each other. It's very sour and refreshing to solve problems... Of course, with the upgrade of dependency package, you will always be happy:)