Preface
HTTP is our modern application network for exchanging data and media streams. Effective use of HTTP can save bandwidth and load data faster. Square's open source OkHttp network request is an efficient HTTP client. Previous knowledge is limited to the use of framework API. After touching the actual work, I know my lack of knowledge. So I dig deep into the framework source code and try my best to absorb the design experience of my predecessors. There are numerous tutorials on the source code parsing network of this framework. This article is called source code parsing. It's actually a work of fried rice. If there are any mistakes and shortcomings, I'd like you to point out.
Interceptor
Interceptor is the essence of OkHttp framework design. Interceptor defines the responsibility chain through which Request passes regardless of the specific implementation process of Request, and allows developers to customize their interceptor functions and insert them into the responsibility chain.
User-defined interceptors are located in OkHttpClient.addInterceptor() added to the interceptors responsibility chain
When RealCall.execute() is executed, RealCall.getResponseWithInterceptorChain() is called to add interceptors from OkHttpClient and default interceptors to the RealInterceptorChain responsibility chain and call them. The code does not encapsulate the originalRequest. InterceptorChain and originalRequest flow to the RealInterceptorChain class for processing.
CustomInterceptor RetryAndFollowUpInterceptor BridgeInterceptor CacheInterceptor ConnectInterceptor NetworkInterceptors CallServiceInterceptor
RealInterceptorChain.proceed()
EventListener.callStart() is also embedded in the Request call procedure in RealCall.execute(), and EventListener.callEnd() is called in Stream Allocation.
Request.Builder
url (String/URL/HttpUrl)
header
CacheControl
Tag (Use this API to attach timing, debugging, or other application data to a request so that you may read it in interceptors, event listeners, or callbacks.)
BridgeInterceptor
Bridges from application code to network code. First it builds a network request from a user request. Then it proceeds to call the network. Finally it builds a user response from the network response.
This interceptor is a bridge between application code and network code. It encapsulates the user request into a network request and executes the request. At the same time, it completes the transformation from network response to user response. Finally, Chain.proceed() method starts the responsibility chain of the interceptor. In Real Interceptor Chain, the tasks of network request and response are allocated to each interceptor through recursive calls, and then the network is allocated to each interceptor through ResponseBuilder.build() method. Response encapsulation, and then recursively invoke the chain of responsibility pattern so that both the invocation and the response process can be written into the BridgeInterceptor
public final class RealInterceptorChain implements Interceptor.Chain { public Response proceed(Request request, StreamAllocation streamAllocation,
HttpCodec httpCodec, RealConnection connection) throws IOException { if (index >= interceptors.size()) throw new AssertionError();
calls++;
... // Call the next interceptor in the chain.
RealInterceptorChain next = new RealInterceptorChain(interceptors,
streamAllocation, httpCodec,connection, index + 1, request, call,
eventListener, connectTimeout, readTimeout,writeTimeout);
Interceptor interceptor = interceptors.get(index);
Response response = interceptor.intercept(next);
... return response;
}
}CallServiceInterceptor；
Interceptor's logic is implemented in intercept() method. After obtaining the request topic through Chain entity class, the request is forwarded to Okio interface through BufferedSink interface. During the interception process, the interceptor's processing state (mainly RequestBodyStart and RequestBodyEnd) is sent out through EventListener interface.
public final class CallServiceInterceptor implements Interceptor { @Override public Response intercept(Chain chain) throws IOException {
Response.Builder responseBuilder = null; if (HttpMethod.permitsRequestBody(request.method()) && request.body() != null) { // If there's a "Expect: 100-continue" header on the request, wait for a "HTTP/1.1 100
// Continue" response before transmitting the request body. If we don't get that, return
// what we did get (such as a 4xx response) without ever transmitting the request body.
if ("100-continue".equalsIgnoreCase(request.header("Expect"))) {
httpCodec.flushRequest();
realChain.eventListener().responseHeadersStart(realChain.call());
responseBuilder = httpCodec.readResponseHeaders(true);
} if (responseBuilder == null) { // Write the request body if the "Expect: 100-continue" expectation was met.
realChain.eventListener().requestBodyStart(realChain.call()); long contentLength = request.body().contentLength();
CountingSink requestBodyOut = new CountingSink(httpCodec.createRequestBody(request, contentLength));
BufferedSink bufferedRequestBody = Okio.buffer(requestBodyOut);
request.body().writeTo(bufferedRequestBody);
bufferedRequestBody.close();
realChain.eventListener()
.requestBodyEnd(realChain.call(), requestBodyOut.successfulCount);
} else if (!connection.isMultiplexed()) { // If the "Expect: 100-continue" expectation wasn't met, prevent the HTTP/1 connection
// from being reused. Otherwise we're still obligated to transmit the request body to
// leave the connection in a consistent state.
streamAllocation.noNewStreams();
}
}
}
}CacheInterceptor；
public final class CacheInterceptor implements Interceptor { @Override public Response intercept(Chain chain) throws IOException {
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse; if (cache != null) { /**
* Track an HTTP response being satisfied with {@code cacheStrategy}.
* Mainly track the number of networkRequest and the hitcount of the corresponding Cache
*/
cache.trackResponse(strategy);
} if (cacheCandidate != null && cacheResponse == null) {
closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
} // If we're forbidden from using the network and the cache is insufficient, fail.
if (networkRequest == null && cacheResponse == null) { return new Response.Builder()
.request(chain.request())
.protocol(Protocol.HTTP_1_1)
.code(504)
.message("Unsatisfiable Request (only-if-cached)")
.body(Util.EMPTY_RESPONSE)
.sentRequestAtMillis(-1L)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
} // If we don't need the network, we're done.
if (networkRequest == null) { return cacheResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build();
} //Call the next interceptor in chain.proceed()
Response networkResponse = null; try {
networkResponse = chain.proceed(networkRequest);
} finally { // If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null && cacheCandidate != null) {
closeQuietly(cacheCandidate.body());
}
} //Processing response and returning
... return response;
}
}OkHttpClient
OkHttpClient hosts all HTTP calls, and each Client has its own connection pool and thread pool.
Implementing the method of abstract class Internal, which is the only implementation of the abstract class of Internel, is related to Cache Interceptor controlling the Header.Lenient region of Http and StreamAlloction obtaining connections from the connection pool.
private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout, int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException {
... synchronized (connectionPool) {
... if (result == null) { // Attempt to get a connection from the pool.
Internal.instance.get(connectionPool, address, this, null); if (connection != null) {
foundPooledConnection = true;
result = connection;
} else {
selectedRoute = route;
}
}
} return result;RouteDatabase && RouteSeletor
RouteDatabase is a blacklist of connection paths that record connection failures, so OkHttp can learn from failures and tend to choose other available paths. RouteSeletor can use RouteDatabase.shouldPostpone(route) method to find out whether this path has failed recently. Some of the source codes of RouteSelector are as follows:
public final class RouteSelector { /**
* Clients should invoke this method when they encounter a connectivity failure on a connection
* returned by this route selector.
* RouteSelector. ConneFailed () logic was added to StremAllocation. streamFailed ().
*/
public void connectFailed(Route failedRoute, IOException failure) { if (failedRoute.proxy().type() != Proxy.Type.DIRECT && address.proxySelector() != null) { // Tell the proxy selector when we fail to connect on a fresh connection.
address.proxySelector().connectFailed(
address.url().uri(), failedRoute.proxy().address(), failure);
}
routeDatabase.failed(failedRoute);
}
}synchronized void enqueue(AsyncCall call) { if (runningAsyncCalls.size() < maxRequests && runningCallsForHost(call) < maxRequestsPerHost) {
runningAsyncCalls.add(call);
executorService().execute(call);
} else {
readyAsyncCalls.add(call);
}
}
... /** Used by {@code Call#execute} to signal it is in-flight. */
synchronized void executed(RealCall call) {
runningSyncCalls.add(call);
}ExecutorSevice.execute(AsyncCall) ExecutorSevice. execute() method, in which the execution code is replicated inside AsyncCall. The method defines some Callback backs backadjustment point operation logic, including the user's active cancellation of execution (using retryAndFollowUpInterceptor) and the callback method when execution request succeeds or fails.
final class AsyncCall extends NamedRunnable {
... @Override protected void execute() { boolean signalledCallback = false; try {
Response response = getResponseWithInterceptorChain(); if (retryAndFollowUpInterceptor.isCanceled()) {
signalledCallback = true;
responseCallback.onFailure(RealCall.this, new IOException("Canceled"));
} else {
signalledCallback = true;
responseCallback.onResponse(RealCall.this, response);
}
} catch (IOException e) { if (signalledCallback) { // Do not signal the callback twice!
Platform.get().log(INFO, "Callback failure for " + toLoggableString(), e);
} else {
eventListener.callFailed(RealCall.this, e);
responseCallback.onFailure(RealCall.this, e);
}
} finally {
client.dispatcher().finished(this);
}
}
}Created Lazily members
ExecutorService()
CacheControl
WebSocket
WebSocket asynchronous non-blocking web socket interface (implemented by Enqueue method)
OkHttpClient implements factory construction by implementing WebSocket. Factory. New WebSocket interface, usually constructed by OkHttpClient
WebSocket life cycle:
Connecting state: The initial state of each websocket at which the Message may be in the queue state but has not yet been processed by Dispatcher
Open state: WebSocket has been accepted by the server side and Socket is completely open. All messages will be processed immediately after entering the queue.
Closing status: WebSocket is gracefully closed, and WebSocket continues to process incoming messages but rejects new ones.
Closed state: WebSocket has completed the process of sending and receiving messages and is in a completely closed state
Influenced by various factors such as the network, WebSocket may break the circuit and advance to the shutdown process.
Canceled state: Passive WebSocket failed connection is a non-elegant process, while active is an elegant short-circuit process
RealWebSocket
RealWebSocket manages the size of the space occupied by the content of the Request queue and the graceful closure time after closing the Socket. The default is 16M and 60s. In RealWebSocket.connect() method, RealWebSocket encapsulates OkHttpClient and Request in the form of Call, and then defines the Callback code when the call succeeds or fails through the Call.enqueue() method.
public void connect(OkHttpClient client) {
client = client.newBuilder()
.eventListener(EventListener.NONE)
.protocols(ONLY_HTTP1)
.build(); final Request request = originalRequest.newBuilder()
.header("Upgrade", "websocket")
.header("Connection", "Upgrade")
.header("Sec-WebSocket-Key", key)
.header("Sec-WebSocket-Version", "13")
.build();
call = Internal.instance.newWebSocketCall(client, request);
call.enqueue(new Callback() { @Override public void onResponse(Call call, Response response) { try {
checkResponse(response);
} catch (ProtocolException e) {
failWebSocket(e, response);
closeQuietly(response); return;
} // Promote the HTTP streams into web socket streams.
StreamAllocation streamAllocation = Internal.instance.streamAllocation(call);
streamAllocation.noNewStreams(); // Prevent connection pooling!
Streams streams = streamAllocation.connection().newWebSocketStreams(streamAllocation); // Process all web socket messages.
try {
listener.onOpen(RealWebSocket.this, response);
String name = "OkHttp WebSocket " + request.url().redact();
initReaderAndWriter(name, streams);
streamAllocation.connection().socket().setSoTimeout(0);
loopReader();
} catch (Exception e) {
failWebSocket(e, null);
}
} @Override public void onFailure(Call call, IOException e) {
failWebSocket(e, null);
}
});
}When the Call request is responded by the server, the HTTP stream is imported into the Web Socket stream and the corresponding state method of WebSocketListener is invoked. The state of WebSocketListener is as follows:
onOpen()onMessage()onClosing()onClosed()onFailure()
WebSocket -> RealWebSocket
Connection -> RealConnection
Interceptor -> RealInterceptorChain
Call -> RealCall
ResponseBody -> RealResponseBody
Gzip Compressor Mechanism
The code that handles Gzip compression is in the BridgeInterceptor, which by default is in the gzip compression state, which can be learned from the source code snippet below. If there is no Accept-Encoding in the header, it is automatically added by default, and the marked variable transparentGzip is true
// If we add an "Accept-Encoding: gzip" header field we're responsible for also decompressing
// the transfer stream.
boolean transparentGzip = false; if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) {
transparentGzip = true;
requestBuilder.header("Accept-Encoding", "gzip");
}The decompression process of BridgeInterceptor calls the okio.GzipSource() method and calls the Okio.buffer() cache decompression process. The source code is as follows
if (transparentGzip
&& "gzip".equalsIgnoreCase(networkResponse.header("Content-Encoding"))
&& HttpHeaders.hasBody(networkResponse)) {
GzipSource responseBody = new GzipSource(networkResponse.body().source());
Headers strippedHeaders = networkResponse.headers().newBuilder()
.removeAll("Content-Encoding")
.removeAll("Content-Length")
.build();
responseBuilder.headers(strippedHeaders); String contentType = networkResponse.header("Content-Type");
responseBuilder.body(new RealResponseBody(contentType, -1L, Okio.buffer(responseBody)));
}RealCall Construction Method
On the RealCall constructor, earlier versions of the RealCall constructor separated EventListener.Factory from EventListenerFactory.Create() to make the RealCall constructor non-thread-safe. The current version of the RealCall constructor uses OkHttpClient. EvetListenerFactory (). create ().
The early versions are as follows:
final class RealCall implements Call {
RealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) {
... final EventListener.Factory eventListenerFactory = client.eventListenerFactory(); this.client = client; this.originalRequest = originalRequest; this.forWebSocket = forWebSocket; //Retry and follow-up interceptor
this.retryAndFollowUpInterceptor = new RetryAndFollowUpInterceptor(client, forWebSocket); // TODO(jwilson): this is unsafe publication and not threadsafe.
//This is not a secure release, not a thread-safe release.
this.eventListener = eventListenerFactory.create(this);
}
}Now OkHttp 3.11.0 RealCall source code is as follows
final class RealCall implements Call { private EventListener eventListener;
... private RealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) { this.client = client; this.originalRequest = originalRequest; this.forWebSocket = forWebSocket; this.retryAndFollowUpInterceptor = new RetryAndFollowUpInterceptor(client, forWebSocket);
} static RealCall newRealCall(OkHttpClient client, Request originalRequest, boolean forWebSocket) { // Safely publish the Call instance to the EventListener.
RealCall call = new RealCall(client, originalRequest, forWebSocket);
call.eventListener = client.eventListenerFactory().create(call); return call;
}
}ConnetionPool
Connection pools can reuse http connections to reduce network latency when accessing the same target host. This kind of implementation manages connection opening and closing strategies and uses background threads corresponding to connection pools one by one to clean up outdated connections. ConnectionPool provides several operations for Deque < Real Connection > such as put, get, connection BecameIdle and evictAll. Here we give examples of put and get operations, which correspond to putting connections, getting connections, removing connections, and removing all connection operations.
public final class ConnectionPool {
... private static final Executor executor = new ThreadPoolExecutor(0 /* corePoolSize */,
Integer.MAX_VALUE /* maximumPoolSize */, 60L /* keepAliveTime */, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(), Util.threadFactory("OkHttp ConnectionPool", true)); /** The maximum number of idle connections for each address. */
private final int maxIdleConnections; private final long keepAliveDurationNs; private final Runnable cleanupRunnable = new Runnable() { @Override public void run() { while (true) { long waitNanos = cleanup(System.nanoTime()); if (waitNanos == -1) return; if (waitNanos > 0) { long waitMillis = waitNanos / 1000000L;
waitNanos -= (waitMillis * 1000000L); synchronized (ConnectionPool.this) { try {
ConnectionPool.this.wait(waitMillis, (int) waitNanos);
} catch (InterruptedException ignored) {
}
}
}
}
}
};
...
}In cleanUpRunnable, there is a while(true), and a loop includes:
Call a cleanUp method to clean up and return a long one
If it is - 1, it exits, otherwise the wait method is called to wait for the long value.
okhttp automatically reclaims connections based on whether the Stream Allocation reference count is zero. CleaUpRunnable traverses each Real Connection to determine which is idle and which is in use by referencing the number of references, while finding the Real Connection with the longest idle time. If the number of free time exceeds the maximum number of free time or the free time exceeds the maximum amount of free time, clean up the Real Connection and return to 0, indicating that it needs to be cleaned up again immediately.
public final class ConnectionPool {
... void put(RealConnection connection) { assert (Thread.holdsLock(this)); if (!cleanupRunning) {
cleanupRunning = true;
executor.execute(cleanupRunnable);
}
connections.add(connection);
}
...
}Before put ting, we first call executor.execute(cleanupRunnable) to clean up idle threads.
RealConnection
Real Connection is the packaging of socket physical connections, which maintains the reference of List < Reference < Stream Allocation >. The number of StreamAllocation s in List is the count of sockets referenced. If the count is zero, the connection is idle and needs to be reclaimed. If the count is not zero, it means that the upper code is still referenced and the connection does not need to be closed.