Go Module: how to solve the reference problem of private non compliant Library


Original text: https://mp.weixin.qq.com/s/Q28LwtYfU7KH_Zy0VbHvOA




Hello, I'm polarisxu.

A friend asked a question:

Netizen consultation

In actual projects, when using Go Module, it is inevitable to have some own libraries to reference. These libraries are managed by self built Git services, such as GitLab. In view of this situation, many tutorials let you set GOPRIVATE, that is, do not go to GOPROXY.

However, sometimes, not only is the library private, but the address is likely not "compliant". What is "non-compliance"?

  • Not HTTPS
  • Non 443 or 80 ports
  • Use IP address directly

It's not HTTPS. It's mainly non 443 or 80, and the problem of direct IP address is relatively large. Because Go does not support the following syntax:

import "studygolang.com:8081/polarisxu/util"
import ""

Generally, we should avoid such a situation. After all, it's disgusting and there's no need to ask for trouble.

But what if it happens? This paper discusses this problem.

01 simulation environment

Because I haven't encountered such a problem before, in order to better solve it, this friend directly provided his environment for me to experiment. I registered my account and created a warehouse testgo: http://vitogo.tpddns.cn:9000/polarisxu/testgo . (for convenience, this warehouse is public)

Tip: if you want to experiment, you can register an account yourself. Of course, you can also build one locally through gitlab.

Create a file testgo.go in the warehouse, as follows:

package testgo

func MyName() string {
 return "polarisxu"

The key is how to write the go.mod of this library?

Obviously, we can't use it directly   vitogo.tpddns.cn:9000/polarisxu/testgo   Such a module name.

02 module name

What can we do?

We can use the function of git to   vitogo.tpddns.cn:9000   Replace: (can also be modified directly)  ~/. gitconfig   (file)

git config --global url."http://vitogo.tpddns.cn:9000/".insteadof "https://{{gitlab_url}}/"

Here are two points to note:

1) The content behind the url and the specific value need to be determined according to your situation.

You can view it through your self built warehouse:

View clone

HTTP is selected here (because I have created a public repository). You can also choose to use ssh mode, so that you can Pull normally even if you configure your own SSH KEY in a private warehouse. (you can easily find the tutorial on how to configure SSH KEY on the Internet, which is available on GitHub)

We use it here   http://vitogo.tpddns.cn:9000/ , indicating all contents under this domain name.

2) The content after insteadOf indicates that when accessing this link, it will be replaced by the link after the above url.

What does this value say? Obviously, it must be a compliant domain name. Let's try using any domain name, such as   https://studygolang.com/ .

At this time, we try to execute the following command:

$ git config --global url."http://vitogo.tpddns.cn:9000/".insteadof "https://studygolang.com/"
#  Give Way   studygolang.com   Don't go   GOPROXY
$ go env -w GOPRIVATE=studygolang.com
$ go get -v studygolang.com/polarisxu/testgo
go get: unrecognized import path "studygolang.com/polarisxu/testgo": parsing studygolang.com/polarisxu/testgo: XML syntax error on line 15: unescaped < inside quoted string

It's easy to understand. go get finally needs to download the code. How can I download it? This picture illustrates:

go get process

How do you know that the current warehouse is hosted by VCS? For the domain name studygolang.com, it will try to request and determine the CVS type. Obviously, studygolang.com does not do any processing. It is not a CVS type, so an error is reported.

For those interested in this process, please refer to this article: https://studygolang.com/articles/35235

Netizens want to use vitogo.tpddns.cn, his domain name, but they also have this problem. If you want to make it normal, you need to do special treatment. Refer to the above article for details.

Therefore, we use a very useful off the shelf Git public hosting service, such as gitea.com. (polarisxu/testgo I typed a tag: v0.0.1)

$ git config --global url."http://vitogo.tpddns.cn:9000/".insteadof "https://gitea.com/"
$ go env -w GOPRIVATE=gitea.com
$ go get -v gitea.com/polarisxu/testgo
get "gitea.com/polarisxu/testgo": found meta tag vcs.metaImport{Prefix:"gitea.com/polarisxu/testgo", VCS:"git", RepoRoot:"https://gitea.com/polarisxu/testgo.git"} at //gitea.com/polarisxu/testgo?go-get=1
go: downloading gitea.com/polarisxu/testgo v0.0.1

succeed! You can go there  $ GOPATH/pkg/mod   Let's see if there is a corresponding package.

Note among them  ? go-get=1   This parameter can be accessed by your browser https://gitea.com/polarisxu/testgo?go-get=1, and then check the source code to see what is inside:

<!doctype html>
  <meta name="go-import" content="gitea.com/polarisxu/testgo git https://gitea.com/polarisxu/testgo.git">
  <meta name="go-source" content="gitea.com/polarisxu/testgo _ https://gitea.com/polarisxu/testgo/src/branch/master{/dir} https://gitea.com/polarisxu/testgo/src/branch/master{/dir}/{file}#L{line}">
  go get gitea.com/polarisxu/testgo

Therefore, we can add the go.mod file in polarisxu/testgo:

go mod init gitea.com/polarisxu/testgo

Then type the second tag: v0.0.2 and get the following again:

$ go get -v gitea.com/polarisxu/testgo
get "gitea.com/polarisxu/testgo": found meta tag vcs.metaImport{Prefix:"gitea.com/polarisxu/testgo", VCS:"git", RepoRoot:"https://gitea.com/polarisxu/testgo.git"} at //gitea.com/polarisxu/testgo?go-get=1
go: downloading gitea.com/polarisxu/testgo v0.0.2

03 use this package

Create a project locally and reference the package defined above:

$ mkdir ~/testprivate
$ cd ~/testprivate
$ go mod init testprivate
$ touch main.go

Enter the following in main.go:

package main

import (

func main() {
    fmt.Println("Hello", testgo.MyName())

After executing go mod tidy, run:

$ go run main.go
Hello polarisxu

Normally output the results we expect.

04 summary

By understanding the basic principle of go get, know some processing methods of git and the role of GOPRIVATE. I believe you can solve similar problems yourself.

Note that if you actually use gitea.com, you can choose gitee.com, try.gogs.io, etc.

In addition, Go has several issue s related to this article:

  • https://github.com/golang/go/issues/34436
  • https://github.com/golang/go/issues/38213

Reference articles

  • Use practice of GO module in privatized warehouse: https://studygolang.com/articles/35235
  • go modules uses local libraries, compliance libraries, and private libraries: https://studygolang.com/articles/35234

Posted by kitegirl on Tue, 30 Nov 2021 13:07:49 -0800