Openresty reverse proxy returns non-200 for retry to custom upstream

Keywords: curl encoding Nginx

By using error_page instruction, the status codes of 404, 302 etc. returned by the reverse proxy are processed twice for sub-requests.

Client - [1] - > nginx proxy < - [2] - > Site A (return 404, 302)
                    |[3]
                    V
             Site B (return 200)
  • For example, as a CDN service, site A returns 302, accesses site B with url of location extracted by sub-request, and returns the result to the client.
  • For example, as a download mirror site, the client requests to download the file, and in Site A can not find the return 404, using sub-requests to request Site B.

testing environment

  • macos
  • openresty/1.13.6.1

Scheme 1 error_page

Here's a simulation example of a 404 status code jumping to a standby site

server {
    listen 8008;
...

 # Testing upstream, proxy_pass errors, secondary requests, such as cdn
 # How to do sub-request jump in 404,302
 location = /test_404 {
  proxy_pass http://127.0.0.1:9996;  # Returns 404 status code
  # uwsgi 
  # uwsgi_intercept_errors on;
  # fastcgi
  # fastcgi_intercept_errors on;

  proxy_intercept_errors on;
  error_page 404 403 = @error_page_404;
 }

 location @error_page_404 {
  content_by_lua_block {
   ngx.say('404 => 200')
   ngx.exit(200)
  }
}

server {
 listen 9996;
 location /test_404 {
  content_by_lua_block {
   ngx.exit(404)
  } 
 }
}

Be careful proxy_intercept_errors This configuration needs to be turned on to transfer the status code >= 300 to the error_page process. The corresponding uwsgi and fastcgi directional agents have similar instructions.

test

$ curl -i http://127.0.0.1:9996/test_404
HTTP/1.1 404 Not Found
Server: openresty/1.13.6.1
Date: Sun, 22 Apr 2018 02:59:12 GMT
Content-Type: text/html
Content-Length: 175
Connection: keep-alive


$ curl -i 127.0.0.1:8008/test_404
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 22 Apr 2018 03:02:45 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive

404 => 200

Scheme 2 proxy_next_upstream

proxy_next_upstream

Main configuration

upstream backend_502_testing {
    server 127.0.0.1:9998; # 502, there is no service on the back end
    server 127.0.0.1:9996;  
}

server {
    listen 8008;
    ...
     # 502 for example, test proxy_next_upstream, failover
   location = /test_502 {
    proxy_next_upstream error timeout http_502;
    proxy_pass http://backend_502_testing;
   }
}

server {
 listen 9996;
 location /test_502 {
  content_by_lua_block {
   ngx.say('502 => 200')
   ngx.exit(200)
  }
 } 
}

test result

$ curl -i 127.0.0.1:9998/test_502
curl: (7) Failed to connect to 127.0.0.1 port 9998: Connection refused

$ curl -i 127.0.0.1:9996/test_502
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 22 Apr 2018 03:06:17 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive

502 => 200

$ curl -i 127.0.0.1:8008/test_502
HTTP/1.1 200 OK
Server: openresty/1.13.6.1
Date: Sun, 22 Apr 2018 03:12:02 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 11
Connection: keep-alive

502 => 200


//Erlog at this point 
2018/04/22 11:12:02 [error] 80096#0: *108 kevent() reported that connect() failed (61: Connection refused) while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET /test_502 HTTP/1.1", upstream: "http://127.0.0.1:9998/test_502", host: "127.0.0.1:8008"

Multiple requests for 127.0.0.1:8008/test_502 reveal that upstream does not access the services of port 9998 first for each request, but when the services of port 9996 are found to be unavailable for 127.0.0.1:9998, requests will automatically be sent to the healthy back end of 127.0.0.1:9996 for a period of time before continuing to try 127.0.0.1:9998.

summary

Comparison of two schemes

  • State code range, error_page mode (scheme 1) can capture more state code
  • Scenario distinction, for each request, you must first try site 1, if you get a non-200 status code, Visit Site 2 (where lua processing can also be added), this logic can only use scheme 1. The second scheme is obviously designed for load balancing, aiming at the healthy state of the back end, and can not achieve the custom logic after switching upstream.

Please understand your needs before making choices.

Posted by guido88 on Tue, 14 May 2019 18:33:26 -0700