go-restful.png
go-restful-code.png
大家好，我叫谢伟，是一名程序员。
下面结合我的经历和见闻，讲述下一名非科班程序员的成长过程：
学习一门编程语言
写尽量多的代码
补尽量多的基础知识
一定阶段后(有开发任务，能按时完成)，开始思考架构：即如何更好的设计一个项目
阅读源代码，看热门的项目的源代码
重点梳理源代码的流程而不是细节
借鉴好的源代码的思路编写程序
掌握更多的软件设计知识
架构师：技术选型、设计
...
一般初学者确定一个方向，比如web 后端、前端等，会选择一门编程语言深入下去，比如后端java、python、go等。通过项目不断练习编程语言和编程思维，知道如何将需求实现出来。一段时间后，有可能算是某一阶段的瓶颈，希望写出更好的代码，除了继续做项目之外，更好的方式是阅读某一个库或者某一项目的源代码，从源代码里学习一些编程的处理方式，之后借鉴到自己的项目中。突破瓶颈，继续精进技能。
一般的软件构建过程是这样的：
设计：方案确定
编写代码
编码风格
技术选型
包
类
子程序
语句
测试
联调
迭代：继续改善代码
本节的主题是：如何阅读源代码？
1. 明确你的问题
开源领域，值得学习的东西太多了，你应该明确知道你需要解决的问题是什么，才能针对性的对某一项目或者某一库进行源代码的阅读。
2. 示例
go-restful是用于构建REST-style web服务的golang包。
在这之前我们需要了解下 HTTP 协议、Web 客户端、服务端。
这些知识和我们访问网址获取到的信息息息相关。
我们在浏览器中输入：URL（www.baidu.com）的整体过程如下：
浏览器(客户端)请求DNS(域名管理系统)，获取IP
IP 能够找到对应的服务器
建立TCP 服务
服务器根据请求处理请求包(HTTP Request)
服务器返回HTTP Response
浏览器(客户端)收到响应后渲染Response 包里的主体(body)
断开连接，浏览器显示网页信息
我们关注里面的：HTTP Request 和 HTTP Response
随意找个网页查看源代码看看：
HTTP Request.png
HTTP 协议：HTTP Request
GET /u/58f0817209aa HTTP/1.1 Host: www.jianshu.com Connection: keep-alive Pragma: no-cache Cache-Control: no-cache Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 Referer: https://www.jianshu.com/ Accept-Encoding: gzip, deflate, br Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
主要包括：
请求行: 请求方法、请求URI、HTTP 协议、协议版本
服务端信息： Host、...
消息体
HTTP 协议 HTTP Response
HTTP/1.1 200 OK Date: Sun, 20 May 2018 03:19:36 GMT Server: Tengine Content-Type: text/html; charset=utf-8Transfer-Encoding: chunked X-Frame-Options: DENY X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' *.jianshu.com *.jianshu.io api.geetest.com static.geetest.com dn-staticdown.qbox.me zz.bdstatic.com *.google-analytics.com hm.baidu.com push.zhanzhang.baidu.com res.wx.qq.com qzonestyle.gtimg.cn as.alipayobjects.com ;style-src 'self' 'unsafe-inline' *.jianshu.com *.jianshu.io api.geetest.com static.geetest.com ; ETag: W/"4d22fb2fcef7cdb3f874a6b4960ff2ae"Cache-Control: max-age=0, private, must-revalidate Set-Cookie: locale=zh-CN; path=/ Set-Cookie: _m7e_session=708ecf714930ebc19da67ae3141bd6c0; path=/; expires=Sun, 20 May 2018 09:19:36 -0000; secure; HttpOnly X-Request-Id: c61a268c-896f-4e03-afbe-2547db04943d X-Runtime: 0.137573Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Content-Encoding: gzip X-Via: 1.1 PSfjfzdx2wn96:6 (Cdn Cache Server V2.0), 1.1 jsyz89:1 (Cdn Cache Server V2.0) Connection: keep-alive X-Dscp-Value: 0
主要包括：
状态行：HTTP 协议、HTTP 协议版本、状态码
服务端信息
消息体
所以关于设计 restful api 的主体部分包括这些：
HTTP 方法：GET、POST、PUT、DELETE
HTTP Request：URI 路径、路径参数、请求参数
HTTP Response：状态码(2XX、3XX、4XX、5XX)、消息体(body)
鉴于上面的知识点，我们如果使用内置的golang 包，处理 http 信息会这么做：
func Downloader(url string) ([]byte, error) {
var (
req *http.Request
err error
) if req, err = http.NewRequest("GET", url, nil); err != nil { return nil, ErrorHttpRequest
}
client := http.DefaultClient
req.Header.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36")
var (
resp *http.Response
) if resp, err = client.Do(req); err != nil { return nil, ErrorHttpResponse
}
defer resp.Body.Close() return ioutil.ReadAll(resp.Body)
}查看下源代码 net/http 库中的 http.Request 和 http.Response 都有些什么？
type Request struct { // Method specifies the HTTP method (GET, POST, PUT, etc.).
// For client requests an empty string means GET.
Method string // URL specifies either the URI being requested (for server
// requests) or the URL to access (for client requests).
//
// For server requests the URL is parsed from the URI
// supplied on the Request-Line as stored in RequestURI. For
// most requests, fields other than Path and RawQuery will be
// empty. (See RFC 2616, Section 5.1.2)
//
// For client requests, the URL's Host specifies the server to
// connect to, while the Request's Host field optionally
// specifies the Host header value to send in the HTTP
// request.
URL *url.URL // The protocol version for incoming server requests.
//
// For client requests these fields are ignored. The HTTP
// client code always uses either HTTP/1.1 or HTTP/2.
// See the docs on Transport for details.
Proto string // "HTTP/1.0"
ProtoMajor int // 1
ProtoMinor int // 0
// Header contains the request header fields either received
// by the server or to be sent by the client.
//
// If a server received a request with header lines,
//
// Host: example.com
// accept-encoding: gzip, deflate
// Accept-Language: en-us
// fOO: Bar
// foo: two
//
// then
//
// Header = map[string][]string{
// "Accept-Encoding": {"gzip, deflate"},
// "Accept-Language": {"en-us"},
// "Foo": {"Bar", "two"},
// }
//
// For incoming requests, the Host header is promoted to the
// Request.Host field and removed from the Header map.
//
// HTTP defines that header names are case-insensitive. The
// request parser implements this by using CanonicalHeaderKey,
// making the first character and any characters following a
// hyphen uppercase and the rest lowercase.
//
// For client requests, certain headers such as Content-Length
// and Connection are automatically written when needed and
// values in Header may be ignored. See the documentation
// for the Request.Write method.
Header Header // Body is the request's body.
//
// For client requests a nil body means the request has no
// body, such as a GET request. The HTTP Client's Transport
// is responsible for calling the Close method.
//
// For server requests the Request Body is always non-nil
// but will return EOF immediately when no body is present.
// The Server will close the request body. The ServeHTTP
// Handler does not need to.
Body io.ReadCloser // GetBody defines an optional func to return a new copy of
// Body. It is used for client requests when a redirect requires
// reading the body more than once. Use of GetBody still
// requires setting Body.
//
// For server requests it is unused.
GetBody func() (io.ReadCloser, error) // ContentLength records the length of the associated content.
// The value -1 indicates that the length is unknown.
// Values >= 0 indicate that the given number of bytes may
// be read from Body.
// For client requests, a value of 0 with a non-nil Body is
// also treated as unknown.
ContentLength int64 // TransferEncoding lists the transfer encodings from outermost to
// innermost. An empty list denotes the "identity" encoding.
// TransferEncoding can usually be ignored; chunked encoding is
// automatically added and removed as necessary when sending and
// receiving requests.
TransferEncoding []string // Close indicates whether to close the connection after
// replying to this request (for servers) or after sending this
// request and reading its response (for clients).
//
// For server requests, the HTTP server handles this automatically
// and this field is not needed by Handlers.
//
// For client requests, setting this field prevents re-use of
// TCP connections between requests to the same hosts, as if
// Transport.DisableKeepAlives were set.
Close bool
// For server requests Host specifies the host on which the
// URL is sought. Per RFC 2616, this is either the value of
// the "Host" header or the host name given in the URL itself.
// It may be of the form "host:port". For international domain
// names, Host may be in Punycode or Unicode form. Use
// golang.org/x/net/idna to convert it to either format if
// needed.
//
// For client requests Host optionally overrides the Host
// header to send. If empty, the Request.Write method uses
// the value of URL.Host. Host may contain an international
// domain name.
Host string // Form contains the parsed form data, including both the URL
// field's query parameters and the POST or PUT form data.
// This field is only available after ParseForm is called.
// The HTTP client ignores Form and uses Body instead.
Form url.Values // PostForm contains the parsed form data from POST, PATCH,
// or PUT body parameters.
//
// This field is only available after ParseForm is called.
// The HTTP client ignores PostForm and uses Body instead.
PostForm url.Values // MultipartForm is the parsed multipart form, including file uploads.
// This field is only available after ParseMultipartForm is called.
// The HTTP client ignores MultipartForm and uses Body instead.
MultipartForm *multipart.Form // Trailer specifies additional headers that are sent after the request
// body.
//
// For server requests the Trailer map initially contains only the
// trailer keys, with nil values. (The client declares which trailers it
// will later send.) While the handler is reading from Body, it must
// not reference Trailer. After reading from Body returns EOF, Trailer
// can be read again and will contain non-nil values, if they were sent
// by the client.
//
// For client requests Trailer must be initialized to a map containing
// the trailer keys to later send. The values may be nil or their final
// values. The ContentLength must be 0 or -1, to send a chunked request.
// After the HTTP request is sent the map values can be updated while
// the request body is read. Once the body returns EOF, the caller must
// not mutate Trailer.
//
// Few HTTP clients, servers, or proxies support HTTP trailers.
Trailer Header // RemoteAddr allows HTTP servers and other software to record
// the network address that sent the request, usually for
// logging. This field is not filled in by ReadRequest and
// has no defined format. The HTTP server in this package
// sets RemoteAddr to an "IP:port" address before invoking a
// handler.
// This field is ignored by the HTTP client.
RemoteAddr string // RequestURI is the unmodified Request-URI of the
// Request-Line (RFC 2616, Section 5.1) as sent by the client
// to a server. Usually the URL field should be used instead.
// It is an error to set this field in an HTTP client request.
RequestURI string // TLS allows HTTP servers and other software to record
// information about the TLS connection on which the request
// was received. This field is not filled in by ReadRequest.
// The HTTP server in this package sets the field for
// TLS-enabled connections before invoking a handler;
// otherwise it leaves the field nil.
// This field is ignored by the HTTP client.
TLS *tls.ConnectionState // Cancel is an optional channel whose closure indicates that the client
// request should be regarded as canceled. Not all implementations of
// RoundTripper may support Cancel.
//
// For server requests, this field is not applicable.
//
// Deprecated: Use the Context and WithContext methods
// instead. If a Request's Cancel field and context are both
// set, it is undefined whether Cancel is respected.
Cancel <-chan struct{} // Response is the redirect response which caused this request
// to be created. This field is only populated during client
// redirects.
Response *Response // ctx is either the client or server context. It should only
// be modified via copying the whole Request using WithContext.
// It is unexported to prevent people from using Context wrong
// and mutating the contexts held by callers of the same request.
ctx context.Context
}type Response struct {
Status string // e.g. "200 OK"
StatusCode int // e.g. 200
Proto string // e.g. "HTTP/1.0"
ProtoMajor int // e.g. 1
ProtoMinor int // e.g. 0
// Header maps header keys to values. If the response had multiple
// headers with the same key, they may be concatenated, with comma
// delimiters. (Section 4.2 of RFC 2616 requires that multiple headers
// be semantically equivalent to a comma-delimited sequence.) Values
// duplicated by other fields in this struct (e.g., ContentLength) are
// omitted from Header.
//
// Keys in the map are canonicalized (see CanonicalHeaderKey).
Header Header // Body represents the response body.
//
// The http Client and Transport guarantee that Body is always
// non-nil, even on responses without a body or responses with
// a zero-length body. It is the caller's responsibility to
// close Body. The default HTTP client's Transport does not
// attempt to reuse HTTP/1.0 or HTTP/1.1 TCP connections
// ("keep-alive") unless the Body is read to completion and is
// closed.
//
// The Body is automatically dechunked if the server replied
// with a "chunked" Transfer-Encoding.
Body io.ReadCloser // ContentLength records the length of the associated content. The
// value -1 indicates that the length is unknown. Unless Request.Method
// is "HEAD", values >= 0 indicate that the given number of bytes may
// be read from Body.
ContentLength int64 // Contains transfer encodings from outer-most to inner-most. Value is
// nil, means that "identity" encoding is used.
TransferEncoding []string
// Close records whether the header directed that the connection be
// closed after reading Body. The value is advice for clients: neither
// ReadResponse nor Response.Write ever closes a connection.
Close bool
// Uncompressed reports whether the response was sent compressed but
// was decompressed by the http package. When true, reading from
// Body yields the uncompressed content instead of the compressed
// content actually set from the server, ContentLength is set to -1,
// and the "Content-Length" and "Content-Encoding" fields are deleted
// from the responseHeader. To get the original response from
// the server, set Transport.DisableCompression to true.
Uncompressed bool
// Trailer maps trailer keys to values in the same
// format as Header.
//
// The Trailer initially contains only nil values, one for
// each key specified in the server's "Trailer" header
// value. Those values are not added to Header.
//
// Trailer must not be accessed concurrently with Read calls
// on the Body.
//
// After Body.Read has returned io.EOF, Trailer will contain
// any trailer values sent by the server.
Trailer Header // Request is the request that was sent to obtain this Response.
// Request's Body is nil (having already been consumed).
// This is only populated for Client requests.
Request *Request // TLS contains information about the TLS connection on which the
// response was received. It is nil for unencrypted responses.
// The pointer is shared between responses and should not be
// modified.
TLS *tls.ConnectionState
}可以看出这两个结构体内存在着我们之前分析的那些点。
如果只使用内置的 net/http 的包如何启动一个web 服务？
package mainimport ( "fmt"
"net/http")func Say(resp http.ResponseWriter, req *http.Request) {
req.ParseForm()
fmt.Println(req.URL.Host, "-", req.URL.Path, "-", req.Form)
fmt.Fprintf(resp, "hello world")
}func main() {
http.HandleFunc("/user/hello", Say)
http.ListenAndServe(":8080", nil)
}访问：localhost:8080/user/hello 返回响应值："hello world"
上文中：URL、和响应值response，我们在代码中进行了处理。同样的我们访问真实的网址， 比如 https://www.baidu.com 则是百度的服务器端代码进行了处理。
3. 抄和使用 example
上文中大概知道了构建 restful api 相关的一些 http 协议的知识， 和内置的库 net/http 的基本使用方法。
但别忘了我们的主题是：阅读 go-restful 的源代码。
首先，我们应该根据官方文档学会基本的使用：
package mainimport ( "fmt"
"log"
"net/http"
"github.com/emicklei/go-restful")type User struct {
Name string
Age string
ID []int}
type UserResource struct {
// normally one would use DAO (data access object)
users map[string]User
}// WebService creates a new service that can handle REST requests for User resources.func (u UserResource) WebService() *restful.WebService {
ws := new(restful.WebService)
ws.
Path("/users").
Consumes(restful.MIME_XML, restful.MIME_JSON).
Produces(restful.MIME_JSON, restful.MIME_XML) // you can specify this per route as well
ws.Route(ws.GET("/").To(u.findAllUsers). // docs
Doc("get all users").
Writes([]User{}).
Returns(200, "OK", []User{}))
ws.Route(ws.GET("/{user-id}").To(u.findUser). // docs
Doc("get a user").
Param(ws.PathParameter("user-id", "identifier of the user").DataType("integer").DefaultValue("1")).
Writes(User{}). // on the response
Returns(200, "OK", User{}).
Returns(404, "Not Found", nil)) return ws
}// GET http://localhost:8080/users//func (u UserResource) findAllUsers(request *restful.Request, response *restful.Response) { list := []User{} for _, each := range u.users { list = append(list, each)
}
response.WriteEntity(list)
}
func (u UserResource) findUser(request *restful.Request, response *restful.Response) {
id := request.PathParameter("user-id")
usr := u.users[id] if len(usr.ID) == 0 {
response.WriteErrorString(http.StatusNotFound, "User could not be found.")
} else {
response.WriteEntity(usr)
}
}
func main() {
type APIServer struct {
Container *restful.Container
}
u := UserResource{map[string]User{}}
u.users["xiewei"] = User{
Name: "xiewei",
Age: "20",
ID: []int{1, 2, 3, 4},
}
apiServer := &APIServer{
Container: restful.DefaultContainer.Add(u.WebService()),
} log.Printf("start listening on localhost:9990") log.Fatal(http.ListenAndServe(":9990", apiServer.Container))
}访问：localhost:9990/users
HTTP/1.1 200 OK
Content-Type: application/jsonDate: Sun, 20 May 2018 04:21:29 GMT
Content-Length: 92[
{ "Name": "xiewei", "Age": "20", "ID": [ 1, 2, 3, 4
]
}
]访问：localhost:9990/users/xiewei
HTTP/1.1 200 OK
Content-Type: application/jsonDate: Sun, 20 May 2018 04:21:29 GMT
Content-Length: 92[
{ "Name": "xiewei", "Age": "20", "ID": [ 1, 2, 3, 4
]
}
]访问：localhost:9990/users/xiewei2
HTTP/1.1 404 Not Found Date: Sun, 20 May 2018 04:22:59 GMT Content-Length: 24Content-Type: text/plain; charset=utf-8User could not be found.
通过这个简单的例子，我们大概能够使用 go-restful 了。
无外乎还是操作：http.Request、http.Response， 上述例子的核心是：findAllUsers 和 findUser 这个两个函数，具体的返回值、状态码什么的都是由这两个函数定义。其他的都是一些路由的定义、定义生产者和消费者格式、启动指定端口的web 服务。
4. 梳理流程
restful-flow.png
1. 启动并监控指定端口的 http 服务
func ListenAndServe(addr string, handler Handler) error { server := &Server{Addr: addr, Handler: handler} return server.ListenAndServe()
}能看出函数的入口是：Handler 接口
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}httpServer 包含 container .
log.Fatal(http.ListenAndServe(":9990", apiServer.Container))一个 Container 包含多个 WebService
type Container struct {
webServicesLock sync.RWMutex
webServices []*WebService
ServeMux *http.ServeMux
isRegisteredOnRoot bool
containerFilters []FilterFunction
doNotRecover bool // default is true
recoverHandleFunc RecoverHandleFunction
serviceErrorHandleFunc ServiceErrorHandleFunction
router RouteSelector // default is a CurlyRouter (RouterJSR311 is a slower alternative)
contentEncodingEnabled bool // default is false}container 实现的了Handler 接口
func (c *Container) ServeHTTP(httpwriter http.ResponseWriter, httpRequest *http.Request) {
c.ServeMux.ServeHTTP(httpwriter, httpRequest)
}一个 webservice 包含多个Route
type WebService struct {
rootPath string
pathExpr *pathExpression // cached compilation of rootPath as RegExp
routes []Route
produces []string
consumes []string
pathParameters []*Parameter
filters []FilterFunction
documentation string
apiVersion string
typeNameHandleFunc TypeNameHandleFunction
dynamicRoutes bool
// protects 'routes' if dynamic routes are enabled
routesLock sync.RWMutex
}一个 Route 包含HTTP 协议协议相关的HTTP Request 、HTTP Reponse 、方法等处理
type Route struct {
Method string
Produces []string
Consumes []string
Path string // webservice root path + described path
Function RouteFunction
Filters []FilterFunction
If []RouteSelectionConditionFunction // cached values for dispatching
relativePath string
pathParts []string
pathExpr *pathExpression // cached compilation of relativePath as RegExp
// documentation
Doc string
Notes string
Operation string
ParameterDocs []*Parameter
ResponseErrors map[int]ResponseError
ReadSample, WriteSample interface{} // structs that model an example request or response payload
// Extra information used to store custom information about the route.
Metadata map[string]interface{} // marks a route as deprecated
Deprecated bool}具体的处理函数是：RouteFunction
type RouteFunction func(*Request, *Response)
再回过来看一下，我们的代码是怎么处理的：
启动http 服务，指定端口并监听：需要传入端口和Handler 接口
log.Fatal(http.ListenAndServe(":9990", apiServer.Container))定义一个 container ，container 类实现了Handler 接口
apiServer := &APIServer{
Container: restful.DefaultContainer.Add(u.WebService()),
}container 内需要定义一个或者多个 webservice, 内含具体的Route 处理函数 RouteFunction
func (u UserResource) WebService() *restful.WebService { ws := new(restful.WebService)
ws. Path("/users"). Consumes(restful.MIME_XML, restful.MIME_JSON). Produces(restful.MIME_JSON, restful.MIME_XML) // you can specify this per route as well
ws.Route(ws.GET("/").To(u.findAllUsers).
// docs Doc("get all users"). Writes([]User{}). Returns(200, "OK", []User{}))
ws.Route(ws.GET("/{user-id}").To(u.findUser).
// docs Doc("get a user"). Param(ws.PathParameter("user-id", "identifier of the user").DataType("integer").DefaultValue("1")). Writes(User{}). // on the response Returns(200, "OK", User{}). Returns(404, "Not Found", nil))
return ws
}好，上面的大致处理流程我们已经梳理清楚。
5. 借鉴使用
如何抽象出的客观实体：比如Route、Webservice、Container
如何对Router、WebService、Container 定义方法
如何对项目进行组织。
方法如何进行的复用
内置库内存在很多的接口，对接口的实现，不断的对内置库的扩展，有可能就重新发明了一个热门的轮子。
go-restful 库便是对内置库 net/http 的扩展。
总结:
阅读源代码首先你需要明确解决的问题是什么，其次你会使用该项目的Demo 或者多个示例，然后你需要根据源代码梳理源代码流程，最后由抄的过程转变为借鉴使用的过程。
再会，希望对你有所启发，我是谢伟。
作者：谢小路
链接：https://www.jianshu.com/p/8cc7ed063e6e