01Gin source code interpretation

Keywords: Go github JSON IE Web Server

brief introduction

Gin source code interpretation, based on v1.5.0 Edition.

Process Overview

On the official document, an example of getting started is as follows:

package main

import "github.com/gin-gonic/gin"

func main() {
    r := gin.Default()
    r.GET("/ping", func(c *gin.Context) {
        c.JSON(200, gin.H{
            "message": "pong",
        })
    })
    r.Run() // Listen and start the service on 0.0.0.0:8080
}

It seems very simple. First, initialize gin.Default(), then define a route called / ping, and finally start r.Run()

Initialization

First, take a deep look at the process of gin.Default():

// Default returns an Engine instance with the Logger and Recovery middleware already attached.
func Default() *Engine {
    debugPrintWARNINGDefault()
    engine := New()
    engine.Use(Logger(), Recovery())
    return engine
}

With the annotation, we can see that the main function of Default is to initialize Engine, and then load two middleware for logging and recovery

Engine is actually a structure and the core of Gin framework. Take a look at its definition

// Engine is the framework's instance, it contains the muxer, middleware and configuration settings.
// Create an instance of Engine, by using New() or Default()
type Engine struct {
    RouterGroup

    // Enables automatic redirection if the current route can't be matched but a
    // handler for the path with (without) the trailing slash exists.
    // For example if /foo/ is requested but a route only exists for /foo, the
    // client is redirected to /foo with http status code 301 for GET requests
    // and 307 for all other request methods.
    RedirectTrailingSlash bool

    // If enabled, the router tries to fix the current request path, if no
    // handle is registered for it.
    // First superfluous path elements like ../ or // are removed.
    // Afterwards the router does a case-insensitive lookup of the cleaned path.
    // If a handle can be found for this route, the router makes a redirection
    // to the corrected path with status code 301 for GET requests and 307 for
    // all other request methods.
    // For example /FOO and /..//Foo could be redirected to /foo.
    // RedirectTrailingSlash is independent of this option.
    RedirectFixedPath bool

    // If enabled, the router checks if another method is allowed for the
    // current route, if the current request can not be routed.
    // If this is the case, the request is answered with 'Method Not Allowed'
    // and HTTP status code 405.
    // If no other Method is allowed, the request is delegated to the NotFound
    // handler.
    HandleMethodNotAllowed bool
    ForwardedByClientIP    bool

    // #726 #755 If enabled, it will thrust some headers starting with
    // 'X-AppEngine...' for better integration with that PaaS.
    AppEngine bool

    // If enabled, the url.RawPath will be used to find parameters.
    UseRawPath bool

    // If true, the path value will be unescaped.
    // If UseRawPath is false (by default), the UnescapePathValues effectively is true,
    // as url.Path gonna be used, which is already unescaped.
    UnescapePathValues bool

    // Value of 'maxMemory' param that is given to http.Request's ParseMultipartForm
    // method call.
    MaxMultipartMemory int64

    delims           render.Delims
    secureJsonPrefix string
    HTMLRender       render.HTMLRender
    FuncMap          template.FuncMap
    allNoRoute       HandlersChain
    allNoMethod      HandlersChain
    noRoute          HandlersChain
    noMethod         HandlersChain
    pool             sync.Pool
    trees            methodTrees
}

There are two ways to initialize the Engine in the note. Default() has already seen it, and look at New():

// New returns a new blank Engine instance without any middleware attached.
// By default the configuration is:
// - RedirectTrailingSlash:  true
// - RedirectFixedPath:      false
// - HandleMethodNotAllowed: false
// - ForwardedByClientIP:    true
// - UseRawPath:             false
// - UnescapePathValues:     true
func New() *Engine {
    debugPrintWARNINGNew()
    engine := &Engine{
        RouterGroup: RouterGroup{
            Handlers: nil,
            basePath: "/",
            root:     true,
        },
        FuncMap:                template.FuncMap{},
        RedirectTrailingSlash:  true,
        RedirectFixedPath:      false,
        HandleMethodNotAllowed: false,
        ForwardedByClientIP:    true,
        AppEngine:              defaultAppEngine,
        UseRawPath:             false,
        UnescapePathValues:     true,
        MaxMultipartMemory:     defaultMultipartMemory,
        trees:                  make(methodTrees, 0, 9),
        delims:                 render.Delims{Left: "{{", Right: "}}"},
        secureJsonPrefix:       "while(1);",
    }
    engine.RouterGroup.engine = engine
    engine.pool.New = func() interface{} {
        return engine.allocateContext()
    }
    return engine
}

Take a second look at the process of adding middleware

// Use attaches a global middleware to the router. ie. the middleware attached though Use() will be
// included in the handlers chain for every single request. Even 404, 405, static files...
// For example, this is the right place for a logger or error management middleware.
func (engine *Engine) Use(middleware ...HandlerFunc) IRoutes {
    engine.RouterGroup.Use(middleware...)
    engine.rebuild404Handlers()
    engine.rebuild405Handlers()
    return engine
}

Adding middleware is actually registering on RouterGroup, so what is RouterGroup?

// RouterGroup is used internally to configure router, a RouterGroup is associated with
// a prefix and an array of handlers (middleware).
type RouterGroup struct {
    Handlers HandlersChain
    basePath string
    engine   *Engine
    root     bool
}

Originally, RouterGroup is used to configure routes. It contains a routing path basePath and a middleware array Handlers
Therefore, adding middleware only adds a new element to Handlers:

// Use adds middleware to the group, see example code in GitHub.
func (group *RouterGroup) Use(middleware ...HandlerFunc) IRoutes {
    group.Handlers = append(group.Handlers, middleware...)
    return group.returnObj()
}

func (group *RouterGroup) returnObj() IRoutes {
    if group.root {
        return group.engine
    }
    return group
}

This is basically the process of initialization

Register handler

Of course, the most important thing for web server is to define routing and processing functions

r.GET("/ping", func(c *gin.Context) {
  c.JSON(200, gin.H{
    "message": "pong",
  })
})

Earlier, we have seen the definition of Engine. Pay attention to the following definitions:

type Engine struct {
  RouterGroup

This shows that RouterGroup is used internally in Engine, so in fact, all HTTP methods are registered on RouterGroup. See GET method for details:

// GET is a shortcut for router.Handle("GET", path, handle).
func (group *RouterGroup) GET(relativePath string, handlers ...HandlerFunc) IRoutes {
    return group.handle("GET", relativePath, handlers)
}

Through comments and code, we can know that GET is just a shortcut. In fact, all HTTP method registration is handled by router.Handle

func (group *RouterGroup) handle(httpMethod, relativePath string, handlers HandlersChain) IRoutes {
    absolutePath := group.calculateAbsolutePath(relativePath)
    handlers = group.combineHandlers(handlers)
    group.engine.addRoute(httpMethod, absolutePath, handlers)
    return group.returnObj()
}

The core statement of handle is group.engine.addRoute(httpMethod, absolutePath, handlers)

First, take a look at the combineHandlers. You can see that the number of handlers is limited to 63
All of a sudden, I feel that slice in Golang is a little bit painful. I have to write three lines

func (group *RouterGroup) combineHandlers(handlers HandlersChain) HandlersChain {
    finalSize := len(group.Handlers) + len(handlers)
    if finalSize >= int(abortIndex) {
        panic("too many handlers")
    }
    mergedHandlers := make(HandlersChain, finalSize)
    copy(mergedHandlers, group.Handlers)
    copy(mergedHandlers[len(group.Handlers):], handlers)
    return mergedHandlers
}

See what operation addRoute has

func (engine *Engine) addRoute(method, path string, handlers HandlersChain) {
    assert1(path[0] == '/', "path must begin with '/'")
    assert1(method != "", "HTTP method can not be empty")
    assert1(len(handlers) > 0, "there must be at least one handler")

    debugPrintRoute(method, path, handlers)
    root := engine.trees.get(method)
    if root == nil {
        root = new(node)
        root.fullPath = "/"
        engine.trees = append(engine.trees, methodTree{method: method, root: root})
    }
    root.addRoute(path, handlers)
}

After skipping the previous judgment, you can see that the core is to operate engine.trees httprouter.
root.addRoute(path, handlers) has a little more content, so it will not be expanded

In a word, the route has been registered here

Function

Finally, take a look at the r.Run() section

// Run attaches the router to a http.Server and starts listening and serving HTTP requests.
// It is a shortcut for http.ListenAndServe(addr, router)
// Note: this method will block the calling goroutine indefinitely unless an error happens.
func (engine *Engine) Run(addr ...string) (err error) {
    defer func() { debugPrintError(err) }()

    address := resolveAddress(addr)
    debugPrint("Listening and serving HTTP on %s\n", address)
    err = http.ListenAndServe(address, engine)
    return
}

This is a blocking method unless an error occurs. The ListenAndServe function of net/http package is used internally

Receiving request

As we have seen above, running is realized through http.ListenAndServe(address, engine),
This is the content of the built-in net/http package. Take a look at the specific definition:

func ListenAndServe(addr string, handler Handler) error

The type of the second parameter is Handler. You can guess the interface type and see what to implement

type Handler interface {
  ServeHTTP(ResponseWriter, *Request)
}

Take a look at how Engine implements the serverhttp method

// ServeHTTP conforms to the http.Handler interface.
func (engine *Engine) ServeHTTP(w http.ResponseWriter, req *http.Request) {
    c := engine.pool.Get().(*Context)
    c.writermem.reset(w)
    c.Request = req
    c.reset()

    engine.handleHTTPRequest(c)

    engine.pool.Put(c)
}

Take a look at engine. Handlehtttprequest (c), which is used to process HTTP requests

func (engine *Engine) handleHTTPRequest(c *Context) {
    httpMethod := c.Request.Method
    rPath := c.Request.URL.Path
    unescape := false
    if engine.UseRawPath && len(c.Request.URL.RawPath) > 0 {
        rPath = c.Request.URL.RawPath
        unescape = engine.UnescapePathValues
    }
    rPath = cleanPath(rPath)

    // Find root of the tree for the given HTTP method
    t := engine.trees
    for i, tl := 0, len(t); i < tl; i++ {
        if t[i].method != httpMethod {
            continue
        }
        root := t[i].root
        // Find route in tree
        value := root.getValue(rPath, c.Params, unescape)
        if value.handlers != nil {
            c.handlers = value.handlers
            c.Params = value.params
            c.fullPath = value.fullPath
            c.Next()
            c.writermem.WriteHeaderNow()
            return
        }
        if httpMethod != "CONNECT" && rPath != "/" {
            if value.tsr && engine.RedirectTrailingSlash {
                redirectTrailingSlash(c)
                return
            }
            if engine.RedirectFixedPath && redirectFixedPath(c, root, engine.RedirectFixedPath) {
                return
            }
        }
        break
    }

    if engine.HandleMethodNotAllowed {
        for _, tree := range engine.trees {
            if tree.method == httpMethod {
                continue
            }
            if value := tree.root.getValue(rPath, nil, unescape); value.handlers != nil {
                c.handlers = engine.allNoMethod
                serveError(c, http.StatusMethodNotAllowed, default405Body)
                return
            }
        }
    }
    c.handlers = engine.allNoRoute
    serveError(c, http.StatusNotFound, default404Body)
}

The code is a bit long, but there are two comments that can help us understand the logic quickly, which are mainly found from engine.trees according to HTTP method and path
When the corresponding handlers can be found:

if value.handlers != nil {
  c.handlers = value.handlers
  c.Params = value.params
  c.fullPath = value.fullPath
  c.Next()
  c.writermem.WriteHeaderNow()
  return
}

Focus on the following c.Next():

// Next should be used only inside middleware.
// It executes the pending handlers in the chain inside the calling handler.
// See example in GitHub.
func (c *Context) Next() {
    c.index++
    for c.index < int8(len(c.handlers)) {
        c.handlers[c.index](c)
        c.index++
    }
}

Therefore, execute the handlers in Context. Note that the handlers include middleware and main processing functions,
Therefore, the route processing is completed

Of course, there are many reasons when the corresponding route cannot be found, such as the HTTP method does not exist, or the path does not exist

Middleware principle

The principle of middleware is involved in the process of calling Next(). Let's talk about it in detail

First, let's see how to define and add middleware from the official document:

func Logger() gin.HandlerFunc {
    return func(c *gin.Context) {
        t := time.Now()

        // Set example variable
        c.Set("example", "12345")

        // before request

        c.Next()

        // after request
        latency := time.Since(t)
        log.Print(latency)

        // access the status we are sending
        status := c.Writer.Status()
        log.Println(status)
    }
}

func main() {
    r := gin.New()
    r.Use(Logger())

    r.GET("/test", func(c *gin.Context) {
        example := c.MustGet("example").(string)

        // it would print: "12345"
        log.Println(example)
    })

    // Listen and serve on 0.0.0.0:8080
    r.Run(":8080")
}

As can be seen from the above example, there is no difference between defining middleware and defining main processing functions. The method definitions are all gin.HandlerFunc:

type HandlerFunc func(*Context)

The middleware also calls c.Next() to divide the request process. c.Next() is before request before running and after request after running

From the above flow chart, it is assumed that two processing functions are registered, the first is log middleware, which is used for logging, and the other is the main processing function of the path
When a request is received, it enters many Next, because the middleware may also call c.Next(). It can also call c.Next()
When running c.Next(), it is actually running the next handler. It is similar to the call stack when recursion

summary

This is the basic process of Gin. Looking at the code roughly, I think it's quite clear. More content needs to be explored

Posted by brandone on Thu, 05 Dec 2019 04:09:38 -0800