introduce

Through a complete example, manage logs reasonably in the Gin framework.

What are the usage scenarios?

Log auto scroll Split into multiple log files Log format modification wait

We will use rk-boot To start the microservices of the Gin framework.

Please visit the following address for a complete tutorial:

• https://rkdocs.netlify.app/cn

install

go get github.com/rookie-ninja/rk-boot

Brief concept

Rk boot uses the following two libraries to manage logs.

• zap Manage log instances
• lumberjack Manage log scrolling

Rk boot defines two log types, which will be described in detail later. Here is a brief introduction.

• ZapLogger: standard log, used to record Error, Info, etc.
• Event logger: JSON or Console format, used to record events, such as RPC requests.

Quick start

In this example, we will try to change the path and format of the zap log.

1. Create boot.yaml

---
zapLogger:
  - name: zap-log                        # Required
    zap:
      encoding: json                     # Optional, options: console, json
      outputPaths: ["logs/zap.log"]      # Optional
gin:
  - name: greeter
    port: 8080
    enabled: true

2. Create main.go

Write a log to the zap log instance.

package main

import (
	"context"
	"github.com/rookie-ninja/rk-boot"
	"github.com/rookie-ninja/rk-entry/entry"
)

func main() {
	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Write zap log
	rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-log").GetLogger().Info("This is zap-log")

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}

4. Verification

Folder structure

├── boot.yaml
├── go.mod
├── go.sum
├── logs
│   └── zap.log
└── main.go

Log output

{"level":"INFO","ts":"2021-10-21T02:10:09.279+0800","msg":"This is zap-log"}

Configure EventLogger

In the above example, we configured the zap log. This time, we modify the EventLogger.

1. Create boot.yaml

---
eventLogger:
  - name: event-log                      # Required
    encoding: json                       # Optional, options: console, json
    outputPaths: ["logs/event.log"]      # Optional
gin:
  - name: greeter
    port: 8080
    enabled: true

2. Create main.go

Write a log to the event log instance.

package main

import (
	"context"
	"github.com/rookie-ninja/rk-boot"
	"github.com/rookie-ninja/rk-entry/entry"
)

func main() {
	// Create a new boot instance.
	boot := rkboot.NewBoot()

	// Bootstrap
	boot.Bootstrap(context.Background())

	// Write event log
	helper := rkentry.GlobalAppCtx.GetEventLoggerEntry("event-log").GetEventHelper()
	event := helper.Start("demo-event")
	event.AddPair("key", "value")
	helper.Finish(event)

	// Wait for shutdown sig
	boot.WaitForShutdownSig(context.Background())
}

3. Start main.go

$ go run main.go

4. Verification

Folder structure

├── boot.yaml
├── go.mod
├── go.sum
├── logs
│   └── event.log
└── main.go

Log content

{
    "endTime":"2021-11-27T01:56:56.001+0800",
    "startTime":"2021-11-27T01:56:56.001+0800",
    "elapsedNano":423,
    "timezone":"CST",
    "ids":{
        "eventId":"70b034b8-27af-43ad-97a5-82c99292297d"
    },
    "app":{
        "appName":"gin-demo",
        "appVersion":"master-f948c90",
        "entryName":"",
        "entryType":""
    },
    "env":{
        "arch":"amd64",
        "az":"*",
        "domain":"*",
        "hostname":"lark.local",
        "localIP":"10.8.0.2",
        "os":"darwin",
        "realm":"*",
        "region":"*"
    },
    "payloads":{},
    "error":{},
    "counters":{},
    "pairs":{
        "key":"value"
    },
    "timing":{},
    "remoteAddr":"localhost",
    "operation":"demo-event",
    "eventStatus":"Ended",
    "resCode":"OK"
}

concept

In the above example, we tried ZapLogger and EventLogger. Next, let's see how rk boot is implemented and used.

framework

ZapLoggerEntry

ZapLoggerEntry is an encapsulation of zap instances.

// ZapLoggerEntry contains bellow fields.
// 1: EntryName: Name of entry.
// 2: EntryType: Type of entry which is ZapLoggerEntryType.
// 3: EntryDescription: Description of ZapLoggerEntry.
// 4: Logger: zap.Logger which was initialized at the beginning.
// 5: LoggerConfig: zap.Logger config which was initialized at the beginning which is not accessible after initialization..
// 6: LumberjackConfig: lumberjack.Logger which was initialized at the beginning.
type ZapLoggerEntry struct {
	EntryName        string             `yaml:"entryName" json:"entryName"`
	EntryType        string             `yaml:"entryType" json:"entryType"`
	EntryDescription string             `yaml:"entryDescription" json:"entryDescription"`
	Logger           *zap.Logger        `yaml:"-" json:"-"`
	LoggerConfig     *zap.Config        `yaml:"zapConfig" json:"zapConfig"`
	LumberjackConfig *lumberjack.Logger `yaml:"lumberjackConfig" json:"lumberjackConfig"`
}

How to configure ZapLoggerEntry in boot.yaml?

ZapLoggerEntry is fully compatible zap and lumberjack YAML structure of.

Users can configure multiple ZapLogger instances according to their needs and access them through name.

Full configuration:

---
zapLogger:
  - name: zap-logger                      # Required
    description: "Description of entry"   # Optional
    zap:
      level: info                         # Optional, default: info, options: [debug, DEBUG, info, INFO, warn, WARN, dpanic, DPANIC, panic, PANIC, fatal, FATAL]
      development: true                   # Optional, default: true
      disableCaller: false                # Optional, default: false
      disableStacktrace: true             # Optional, default: true
      sampling:                           # Optional, default: empty map
        initial: 0
        thereafter: 0
      encoding: console                   # Optional, default: "console", options: [console, json]
      encoderConfig:
        messageKey: "msg"                 # Optional, default: "msg"
        levelKey: "level"                 # Optional, default: "level"
        timeKey: "ts"                     # Optional, default: "ts"
        nameKey: "logger"                 # Optional, default: "logger"
        callerKey: "caller"               # Optional, default: "caller"
        functionKey: ""                   # Optional, default: ""
        stacktraceKey: "stacktrace"       # Optional, default: "stacktrace"
        lineEnding: "\n"                  # Optional, default: "\n"
        levelEncoder: "capitalColor"      # Optional, default: "capitalColor", options: [capital, capitalColor, color, lowercase]
        timeEncoder: "iso8601"            # Optional, default: "iso8601", options: [rfc3339nano, RFC3339Nano, rfc3339, RFC3339, iso8601, ISO8601, millis, nanos]
        durationEncoder: "string"         # Optional, default: "string", options: [string, nanos, ms]
        callerEncoder: ""                 # Optional, default: ""
        nameEncoder: ""                   # Optional, default: ""
        consoleSeparator: ""              # Optional, default: ""
      outputPaths: [ "stdout" ]           # Optional, default: ["stdout"], stdout would be replaced if specified
      errorOutputPaths: [ "stderr" ]      # Optional, default: ["stderr"], stderr would be replaced if specified
      initialFields:                      # Optional, default: empty map
        key: "value"
    lumberjack:                           # Optional
      filename: "rkapp-event.log"         # Optional, default: It uses <processname>-lumberjack.log in os.TempDir() if empty.
      maxsize: 1024                       # Optional, default: 1024 (MB)
      maxage: 7                           # Optional, default: 7 (days)
      maxbackups: 3                       # Optional, default: 3 (days)
      localtime: true                     # Optional, default: true
      compress: true                      # Optional, default: true

How to get ZapLogger in code?

Access via name.

// Access entry
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger")

// Access zap logger
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger").GetLogger()

// Access zap logger config
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger").GetLoggerConfig()

// Access lumberjack config
rkentry.GlobalAppCtx.GetZapLoggerEntry("zap-logger").GetLumberjackConfig()

EventLoggerEntry

Rk boot treats each RPC request as an Event and uses rk-query Event type in to log.

// EventLoggerEntry contains bellow fields.
// 1: EntryName: Name of entry.
// 2: EntryType: Type of entry which is EventLoggerEntryType.
// 3: EntryDescription: Description of EventLoggerEntry.
// 4: EventFactory: rkquery.EventFactory was initialized at the beginning.
// 5: EventHelper: rkquery.EventHelper was initialized at the beginning.
// 6: LoggerConfig: zap.Config which was initialized at the beginning which is not accessible after initialization.
// 7: LumberjackConfig: lumberjack.Logger which was initialized at the beginning.
type EventLoggerEntry struct {
	EntryName        string                `yaml:"entryName" json:"entryName"`
	EntryType        string                `yaml:"entryType" json:"entryType"`
	EntryDescription string                `yaml:"entryDescription" json:"entryDescription"`
	EventFactory     *rkquery.EventFactory `yaml:"-" json:"-"`
	EventHelper      *rkquery.EventHelper  `yaml:"-" json:"-"`
	LoggerConfig     *zap.Config           `yaml:"zapConfig" json:"zapConfig"`
	LumberjackConfig *lumberjack.Logger    `yaml:"lumberjackConfig" json:"lumberjackConfig"`
}

EventLogger field

We can see that the log printed by EventLogger contains fields. Let's introduce these fields.

example

------------------------------------------------------------------------
endTime=2021-11-27T02:30:27.670807+08:00
startTime=2021-11-27T02:30:27.670745+08:00
elapsedNano=62536
timezone=CST
ids={"eventId":"4bd9e16b-2b29-4773-8908-66c860bf6754"}
app={"appName":"gin-demo","appVersion":"master-f948c90","entryName":"greeter","entryType":"GinEntry"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.6","os":"darwin","realm":"*","region":"*"}
payloads={"apiMethod":"GET","apiPath":"/rk/v1/healthy","apiProtocol":"HTTP/1.1","apiQuery":"","userAgent":"curl/7.64.1"}
error={}
counters={}
pairs={}
timing={}
remoteAddr=localhost:61726
operation=/rk/v1/healthy
resCode=200
eventStatus=Ended
EOE

How to configure EventLoggerEntry in boot.yaml?

EventLoggerEntry will inject the Application name into the Event. The launcher extracts the Application name from the go.mod file. If there is no go.mod file, the launcher will use the default name.

You can configure multiple EventLogger instances as required and access them through name.

Full configuration:

---
eventLogger:
  - name: event-logger                 # Required
    description: "This is description" # Optional
    encoding: console                  # Optional, default: console, options: console and json
    outputPaths: ["stdout"]            # Optional
    lumberjack:                        # Optional
      filename: "rkapp-event.log"      # Optional, default: It uses <processname>-lumberjack.log in os.TempDir() if empty.
      maxsize: 1024                    # Optional, default: 1024 (MB)
      maxage: 7                        # Optional, default: 7 (days)
      maxbackups: 3                    # Optional, default: 3 (days)
      localtime: true                  # Optional, default: true
      compress: true                   # Optional, default: true

How to get the EventLogger in the code?

Access via name.

// Access entry
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger")

// Access event factory
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger").GetEventFactory()

// Access event helper
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger").GetEventHelper()

// Access lumberjack config
rkentry.GlobalAppCtx.GetEventLoggerEntry("event-logger").GetLumberjackConfig()

How to use Event?

Event is an interface that contains several methods. Please refer to: Event

Common methods:

// Get EventHelper to create Event instance
helper := rkentry.GlobalAppCtx.GetEventLoggerEntry("event-log").GetEventHelper()

// Start and finish event
event := helper.Start("demo-event")
helper.Finish(event)

// Add K/V
event.AddPair("key", "value")

// Start and end timer
event.StartTimer("my-timer")
event.EndTimer("my-timer")

// Set counter
event.SetCounter("my-counter", 1)