preface
HTTP is our modern application network for exchanging data and media streams. Effective use of HTTP can save bandwidth and load data faster. Square's open source OkHttp network request is an efficient HTTP client. The previous knowledge was limited to the call of framework API. After contacting the actual work, I knew my lack of knowledge, so I dug into the framework source code and tried my best to absorb the design experience of predecessors. There are numerous online tutorials on the source code analysis of this framework. This article is called source code analysis. In fact, it is a work of cooking cold rice. If there are errors and shortcomings, please point out them.
Interceptor
Interceptors are the essence of OkHttp framework design. Interceptors define the responsibility chain through which requests pass, regardless of the specific execution process of requests, and allow developers to customize their own interceptor functions and insert them into the responsibility chain
The user-defined interceptor is located in OkHttpClient.addInterceptor() and added to the interceptors responsibility chain
When RealCall.execute() is executed, call RealCall.getResponseWithInterceptorChain() to add interceptors from OkHttpClient and default interceptors to the RealInterceptorChain responsibility chain and call them. The code does not encapsulate originalRequest, and InterceptorChain and originalRequest flow to the RealInterceptorChain class for processing
CustomInterceptor RetryAndFollowUpInterceptor BridgeInterceptor CacheInterceptor ConnectInterceptor NetworkInterceptors CallServiceInterceptor
1.RealInterceptorChain.proceed()
2.EventListener.callStart() is also embedded in RealCall.execute() into the Request calling procedure, and EventListener.callEnd() is called in StreamAllocation.
3.Request.Builder
url (String/URL/HttpUrl)
header
CacheControl
Tag (Use this API to attach timing, debugging, or other application data to a request so that you may read it in interceptors, event listeners, or callbacks.)
BridgeInterceptor
Bridges from application code to network code. First it builds a network request from a user request. Then it proceeds to call the network. Finally it builds a user response from the network response.
This interceptor is a bridge from application code to network code. It encapsulates the user request into a network request and executes the request. At the same time, it also completes the transformation from network Response to user Response. Finally, the chain. Processed () method starts the interceptor responsibility chain. In RealInterceptorChain, the tasks of network request and Response are allocated to each interceptor through recursive call, and then through ResponseBuilder.build() Method encapsulates the network Response, and then recursively calls the responsibility chain mode, so that the calling and Response processing can be written into BridgeInterceptor together
public final class RealInterceptorChain implements Interceptor.Chain { public Response proceed(Request request, StreamAllocation streamAllocation,
HttpCodec httpCodec, RealConnection connection) throws IOException { if (index >= interceptors.size()) throw new AssertionError();
calls++;
... // Call the next interceptor in the chain.
RealInterceptorChain next = new RealInterceptorChain(interceptors,
streamAllocation, httpCodec,connection, index + 1, request, call,
eventListener, connectTimeout, readTimeout,writeTimeout);
Interceptor interceptor = interceptors.get(index);
Response response = interceptor.intercept(next);
... return response;
}
}CallServiceInterceptor；
The logic of the Interceptor is implemented in the intercept() method. After the request subject is obtained through the Chain entity class, the request is forwarded to the Okio interface through the BufferedSink interface. During the interception process, the Interceptor processing status (mainly RequestBodyStart and RequestBodyEnd) is sent through the EventListener interface
public final class CallServiceInterceptor implements Interceptor { @Override public Response intercept(Chain chain) throws IOException {
Response.Builder responseBuilder = null; if (HttpMethod.permitsRequestBody(request.method()) && request.body() != null) { // If there's a "Expect: 100-continue" header on the request, wait for a "HTTP/1.1 100
// Continue" response before transmitting the request body. If we don't get that, return
// what we did get (such as a 4xx response) without ever transmitting the request body.
if ("100-continue".equalsIgnoreCase(request.header("Expect"))) {
httpCodec.flushRequest();
realChain.eventListener().responseHeadersStart(realChain.call());
responseBuilder = httpCodec.readResponseHeaders(true);
} if (responseBuilder == null) { // Write the request body if the "Expect: 100-continue" expectation was met.
realChain.eventListener().requestBodyStart(realChain.call()); long contentLength = request.body().contentLength();
CountingSink requestBodyOut = new CountingSink(httpCodec.createRequestBody(request, contentLength));
BufferedSink bufferedRequestBody = Okio.buffer(requestBodyOut);
request.body().writeTo(bufferedRequestBody);
bufferedRequestBody.close();
realChain.eventListener()
.requestBodyEnd(realChain.call(), requestBodyOut.successfulCount);
} else if (!connection.isMultiplexed()) { // If the "Expect: 100-continue" expectation wasn't met, prevent the HTTP/1 connection
// from being reused. Otherwise we're still obligated to transmit the request body to
// leave the connection in a consistent state.
streamAllocation.noNewStreams();
}
}
}
}CacheInterceptor；
public final class CacheInterceptor implements Interceptor { @Override public Response intercept(Chain chain) throws IOException {
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse; if (cache != null) { /**
* Track an HTTP response being satisfied with {@code cacheStrategy}.
* It mainly tracks the number of networkrequests and the hitcount of the corresponding Cache
*/
cache.trackResponse(strategy);
} if (cacheCandidate != null && cacheResponse == null) {
closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
} // If we're forbidden from using the network and the cache is insufficient, fail.
if (networkRequest == null && cacheResponse == null) { return new Response.Builder()
.request(chain.request())
.protocol(Protocol.HTTP_1_1)
.code(504)
.message("Unsatisfiable Request (only-if-cached)")
.body(Util.EMPTY_RESPONSE)
.sentRequestAtMillis(-1L)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
} // If we don't need the network, we're done.
if (networkRequest == null) { return cacheResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build();
} //Call the next interceptor in chain. Processed()
Response networkResponse = null; try {
networkResponse = chain.proceed(networkRequest);
} finally { // If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null && cacheCandidate != null) {
closeQuietly(cacheCandidate.body());
}
} //Process the response and return
... return response;
}
}OkHttpClient
OkHttpClient hosts all HTTP calls, and each Client has its own connection pool and thread pool
The method of implementing the abstract class Internal, which is the only implementation of the Internal abstract class. The method is related to the CacheInterceptor controlling the Header.Lenient area of Http and the streamalallocation to obtain connections from the connection pool
private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout, int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException {
... synchronized (connectionPool) {
... if (result == null) { // Attempt to get a connection from the pool.
Internal.instance.get(connectionPool, address, this, null); if (connection != null) {
foundPooledConnection = true;
result = connection;
} else {
selectedRoute = route;
}
}
} return result;RouteDatabase && RouteSeletor
RouteDatabase is a blacklist of connection paths that record connection failures, so that OkHttp can learn from failures and tend to choose other available paths. RouteDatabase.shouldpropone (route) method can be used by RouteDatabase to know whether this path has failed recently. Part of the source code of RouteSelector is as follows:
public final class RouteSelector { /**
* Clients should invoke this method when they encounter a connectivity failure on a connection
* returned by this route selector.
* Added routeSelector.connectFailed() logic in streamalallocation. Streamfailed()
*/
public void connectFailed(Route failedRoute, IOException failure) { if (failedRoute.proxy().type() != Proxy.Type.DIRECT && address.proxySelector() != null) { // Tell the proxy selector when we fail to connect on a fresh connection.
address.proxySelector().connectFailed(
address.url().uri(), failedRoute.proxy().address(), failure);
}
routeDatabase.failed(failedRoute);
}
}synchronized void enqueue(AsyncCall call) { if (runningAsyncCalls.size() < maxRequests && runningCallsForHost(call) < maxRequestsPerHost) {
runningAsyncCalls.add(call);
executorService().execute(call);
} else {
readyAsyncCalls.add(call);
}
}
... /** Used by {@code Call#execute} to signal it is in-flight. */
synchronized void executed(RealCall call) {
runningSyncCalls.add(call);
}ExecutorSevice.execute(AsyncCall) execution code is located in the execute() method replicated inside AsyncCall. The method defines some Callback node operation logic, including the Callback method when the user actively cancels execution (using retryAndFollowUpInterceptor) and when the execution request succeeds or fails
final class AsyncCall extends NamedRunnable {
... @Override protected void execute() { boolean signalledCallback = false; try {
Response response = getResponseWithInterceptorChain(); if (retryAndFollowUpInterceptor.isCanceled()) {
signalledCallback = true;
responseCallback.onFailure(RealCall.this, new IOException("Canceled"));
} else {
signalledCallback = true;
responseCallback.onResponse(RealCall.this, response);
}
} catch (IOException e) { if (signalledCallback) { // Do not signal the callback twice!
Platform.get().log(INFO, "Callback failure for " + toLoggableString(), e);
} else {
eventListener.callFailed(RealCall.this, e);
responseCallback.onFailure(RealCall.this, e);
}
} finally {
client.dispatcher().finished(this);
}
}
}Created lazy members
ExecutorService()
CacheControl
WebSocket
WebSocket asynchronous non blocking web socket interface (implemented by Enqueue method)
OkHttpClient implements factory construction by implementing WebSocket.Factory.newWebSocket interface, which is usually constructed by OkHttpClient
WebSocket lifecycle:
Connecting state: the initial state of each websocket. At this time, the Message may be in the queued state but has not been processed by the Dispatcher
Open status: the WebSocket has been accepted by the server and the Socket is fully open. All messages will be processed immediately after they are queued
Closing status: WebSocket enters an elegant closing state. WebSocket continues to process queued messages, but rejects new messages
Closed status: WebSocket has completed the process of sending and receiving messages and entered the fully closed status
WebSocket is affected by various factors such as the network, and may open circuit and enter the closing process in advance
Cancelled status: the passive WebSocket connection failure is a non elegant process, while the active is an elegant short circuit process
RealWebSocket
RealWebSocket manages the space occupied by the Request queue content and the graceful closing time after closing the Socket. The default is 16M and 60 seconds. In the RealWebSocket.connect() method, RealWebSocket encapsulates OkHttpClient and Request in the form of Call, and then defines the Callback code when the Call succeeds and fails through the Call.enqueue() method
public void connect(OkHttpClient client) {
client = client.newBuilder()
.eventListener(EventListener.NONE)
.protocols(ONLY_HTTP1)
.build(); final Request request = originalRequest.newBuilder()
.header("Upgrade", "websocket")
.header("Connection", "Upgrade")
.header("Sec-WebSocket-Key", key)
.header("Sec-WebSocket-Version", "13")
.build();
call = Internal.instance.newWebSocketCall(client, request);
call.enqueue(new Callback() { @Override public void onResponse(Call call, Response response) { try {
checkResponse(response);
} catch (ProtocolException e) {
failWebSocket(e, response);
closeQuietly(response); return;
} // Promote the HTTP streams into web socket streams.
StreamAllocation streamAllocation = Internal.instance.streamAllocation(call);
streamAllocation.noNewStreams(); // Prevent connection pooling!
Streams streams = streamAllocation.connection().newWebSocketStreams(streamAllocation); // Process all web socket messages.
try {
listener.onOpen(RealWebSocket.this, response);
String name = "OkHttp WebSocket " + request.url().redact();
initReaderAndWriter(name, streams);
streamAllocation.connection().socket().setSoTimeout(0);
loopReader();
} catch (Exception e) {
failWebSocket(e, null);
}
} @Override public void onFailure(Call call, IOException e) {
failWebSocket(e, null);
}
});
}When the Call request is responded by the server, the HTTP stream is imported into the Web Socket stream, and the corresponding status method of WebSocketListener is called. The status of WebSocketListener is as follows:
onOpen()onMessage()onClosing()onClosed()onFailure()
WebSocket -> RealWebSocket
Connection -> RealConnection
Interceptor -> RealInterceptorChain
Call -> RealCall
ResponseBody -> RealResponseBody
Gzip compression mechanism
The code to handle gzip compression is in the BridgeInterceptor. By default, it is gzip compressed, which can be learned from the following source code fragment. If there is no accept encoding in the header, it is automatically added by default, and the tag variable transparentGzip is true
// If we add an "Accept-Encoding: gzip" header field we're responsible for also decompressing
// the transfer stream.
boolean transparentGzip = false; if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) {
transparentGzip = true;
requestBuilder.header("Accept-Encoding", "gzip");
}The BridgeInterceptor decompression process calls the okio.GzipSource() method and Okio.buffer() to cache the decompression process. The source code is as follows
if (transparentGzip
&& "gzip".equalsIgnoreCase(networkResponse.header("Content-Encoding"))
&& HttpHeaders.hasBody(networkResponse)) {
GzipSource responseBody = new GzipSource(networkResponse.body().source());
Headers strippedHeaders = networkResponse.headers().newBuilder()
.removeAll("Content-Encoding")
.removeAll("Content-Length")
.build();
responseBuilder.headers(strippedHeaders); String contentType = networkResponse.header("Content-Type");
responseBuilder.body(new RealResponseBody(contentType, -1L, Okio.buffer(responseBody)));
}RealCall construction method
On the RealCall constructor, the earlier version of RealCall constructor treated EventListener.Factory and EventListenerFactory.Create() separately, resulting in the non thread safe RealCall constructor. The constructor of the current version of RealCall uses OkHttpClient.eventListenerFactory().create()
Previous versions are as follows:
final class RealCall implements Call {
RealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) {
... final EventListener.Factory eventListenerFactory = client.eventListenerFactory(); this.client = client; this.originalRequest = originalRequest; this.forWebSocket = forWebSocket; //Retry and follow up interceptors
this.retryAndFollowUpInterceptor = new RetryAndFollowUpInterceptor(client, forWebSocket); // TODO(jwilson): this is unsafe publication and not threadsafe.
// This is an unsafe release, not thread safe.
this.eventListener = eventListenerFactory.create(this);
}Now the RealCall source code of OkHttp 3.11.0 is as follows
final class RealCall implements Call { private EventListener eventListener;
... private RealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) { this.client = client; this.originalRequest = originalRequest; this.forWebSocket = forWebSocket; this.retryAndFollowUpInterceptor = new RetryAndFollowUpInterceptor(client, forWebSocket);
} static RealCall newRealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) { // Safely publish the Call instance to the EventListener.
RealCall call = new RealCall(client, originalRequest, forWebSocket);
call.eventListener = client.eventListenerFactory().create(call); return call;
}
}ConnetionPool
The connection pool can reuse http connections to reduce network latency when accessing the same target host. This class implements the strategy of managing connection opening and closing, and uses the background thread corresponding to the connection pool to clean up expired connections. ConnectionPool provides methods to operate deque < realconnection >, including put, get, connectionBecameIdle and evictAll. These operations correspond to put connection, get connection, remove connection and remove all connections respectively. Here we give examples of put and get operations.
public final class ConnectionPool {
... private static final Executor executor = new ThreadPoolExecutor(0 /* corePoolSize */,
Integer.MAX_VALUE /* maximumPoolSize */, 60L /* keepAliveTime */, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(), Util.threadFactory("OkHttp ConnectionPool", true)); /** The maximum number of idle connections for each address. */
private final int maxIdleConnections; private final long keepAliveDurationNs; private final Runnable cleanupRunnable = new Runnable() { @Override public void run() { while (true) { long waitNanos = cleanup(System.nanoTime()); if (waitNanos == -1) return; if (waitNanos > 0) { long waitMillis = waitNanos / 1000000L;
waitNanos -= (waitMillis * 1000000L); synchronized (ConnectionPool.this) { try {
ConnectionPool.this.wait(waitMillis, (int) waitNanos);
} catch (InterruptedException ignored) {
}
}
}
}
}
};
...
}In cleanUpRunnable, there is a while(true), and a loop includes:
Call the cleanUp method once to clean up and return a long
If it is - 1, exit. Otherwise, call the wait method to wait for the long value
okhttp automatically recycles connections based on whether the streamalallocation reference count is 0. cleanUpRunnable traverses each RealConnection, determines which are idle and which are in use through the number of references, and finds the RealConnection with the longest idle time. If the idle number exceeds the maximum idle number or the idle time exceeds the maximum idle time, clean up the RealConnection and return 0, indicating that it needs to be cleaned up again immediately
public final class ConnectionPool {
... void put(RealConnection connection) { assert (Thread.holdsLock(this)); if (!cleanupRunning) {
cleanupRunning = true;
executor.execute(cleanupRunnable);
}
connections.add(connection);
}
...
}Before the put operation, we first call executor.execute(cleanupRunnable) to clean up idle threads.
RealConnection
RealConnection is the wrapper of socket physical connection, which maintains the reference of List < reference < streamalallocation > >. The number of streamalallocations in the List is the count of sockets referenced. If the count is 0, it means that the connection is idle and needs to be recycled; If the count is not 0, it means that the upper layer code still references, and there is no need to close the connection.
Related links
[Android tutorial] principle analysis of high availability network framework based on Okhttp