摘要：異常與默認(rèn)值為默認(rèn)值為秒。實(shí)驗(yàn)請(qǐng)求里頭的會(huì)發(fā)起一個(gè)，請(qǐng)求請(qǐng)求一次對(duì)逐個(gè)請(qǐng)求，都失敗，則的返回，對(duì)返回的取決于腳本再請(qǐng)求一次該下面的都掛的情況下出現(xiàn)中健康檢查機(jī)制深入分析容錯(cuò)機(jī)制原創(chuàng)胡志廣線上的一次分析

upstream server temporarily disabled while connecting to upstream
no live upstreams while connecting to upstream

max_fails默認(rèn)值為1,fail_timeout默認(rèn)值為10秒。

nginx可以通過(guò)設(shè)置max_fails（最大嘗試失敗次數(shù)）和fail_timeout（失效時(shí)間，在到達(dá)最大嘗試失敗次數(shù)后，在fail_timeout的時(shí)間范圍內(nèi)節(jié)點(diǎn)被置為失效，除非所有節(jié)點(diǎn)都失效，否則該時(shí)間內(nèi)，節(jié)點(diǎn)不進(jìn)行恢復(fù)）對(duì)節(jié)點(diǎn)失敗的嘗試次數(shù)和失效時(shí)間進(jìn)行設(shè)置，當(dāng)超過(guò)最大嘗試次數(shù)或失效時(shí)間未超過(guò)配置失效時(shí)間，則nginx會(huì)對(duì)節(jié)點(diǎn)狀會(huì)置為失效狀態(tài)，nginx不對(duì)該后端進(jìn)行連接，直到超過(guò)失效時(shí)間或者所有節(jié)點(diǎn)都失效后，該節(jié)點(diǎn)重新置為有效，重新探測(cè).

upstream backend {
    server backend1.example.com weight=5;
    server 127.0.0.1:8080       max_fails=3 fail_timeout=30s;
    server unix:/tmp/backend3;

    server backup1.example.com  backup;
}

比如

connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "POST /demo HTTP/1.1", subrequest: "/capture/getstatus", upstream: "http://192.168.99.100:8080/api/demo/

比如

upstream timed out (110: Connection timed out) while reading response header from upstream

Nginx 默認(rèn)判斷失敗節(jié)點(diǎn)狀態(tài)以connect refuse和time out狀態(tài)為準(zhǔn)，不以HTTP錯(cuò)誤狀態(tài)進(jìn)行判斷失敗，因?yàn)镠TTP只要能返回狀態(tài)說(shuō)明該節(jié)點(diǎn)還可以正常連接，所以nginx判斷其還是存活狀態(tài)；除非添加了proxy_next_upstream指令設(shè)置對(duì)404、502、503、504、500和time out等錯(cuò)誤進(jìn)行轉(zhuǎn)到備機(jī)處理，在next_upstream過(guò)程中，會(huì)對(duì)fails進(jìn)行累加，如果備用機(jī)處理還是錯(cuò)誤則直接返回錯(cuò)誤信息（但404不進(jìn)行記錄到錯(cuò)誤數(shù)，如果不配置錯(cuò)誤狀態(tài)也不對(duì)其進(jìn)行錯(cuò)誤狀態(tài)記錄），綜述，nginx記錄錯(cuò)誤數(shù)量只記錄timeout 、connect refuse、502、500、503、504這6種狀態(tài)，timeout和connect refuse是永遠(yuǎn)被記錄錯(cuò)誤狀態(tài)，而502、500、503、504只有在配置proxy_next_upstream后nginx才會(huì)記錄這4種HTTP錯(cuò)誤到fails中，當(dāng)fails大于等于max_fails時(shí)，則該節(jié)點(diǎn)失效.

如果探測(cè)所有節(jié)點(diǎn)均失效，備機(jī)也為失效時(shí)，那么nginx會(huì)對(duì)所有節(jié)點(diǎn)恢復(fù)為有效，重新嘗試探測(cè)有效節(jié)點(diǎn)，如果探測(cè)到有效節(jié)點(diǎn)則返回正確節(jié)點(diǎn)內(nèi)容，如果還是全部錯(cuò)誤，那么繼續(xù)探測(cè)下去，當(dāng)沒(méi)有正確信息時(shí)，節(jié)點(diǎn)失效時(shí)默認(rèn)返回狀態(tài)為502，但是下次訪問(wèn)節(jié)點(diǎn)時(shí)會(huì)繼續(xù)探測(cè)正確節(jié)點(diǎn)，直到找到正確的為止。

upstream test_server{
        server 192.168.99.100:80801;
        server 192.168.99.100:80802;
        server 192.168.99.100:80803;
    }
##for capture
location /api/test/demo{
            proxy_pass http://test_server/api/demo;
}    
location /api/demo{
            default_type application/json;
            content_by_lua_file conf/lua/demo.lua;
}

lua

local cjson = require "cjson.safe"
testres = ngx.location.capture("/api/test/demo",{
    method= ngx.HTTP_POST,
    body = "arg1=xxxx&arg2=xxxxx"
})
ngx.log(ngx.ERR,"status"..testres.status)
local testbody = cjson.decode(testres.body)
ngx.log(ngx.ERR,testbody==nil)

請(qǐng)求192.168.99.100:8080/api/demo,里頭的lua會(huì)發(fā)起一個(gè)capture，請(qǐng)求/api/test/demo

請(qǐng)求一次

2017/02/09 14:48:57 [error] 5#5: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://192.168.99.100:80801/api/demo", host: "192.168.99.100:8080"
2017/02/09 14:48:57 [warn] 5#5: *1 upstream server temporarily disabled while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://192.168.99.100:80801/api/demo", host: "192.168.99.100:8080"
2017/02/09 14:48:57 [error] 5#5: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://192.168.99.100:80802/api/demo", host: "192.168.99.100:8080"
2017/02/09 14:48:57 [warn] 5#5: *1 upstream server temporarily disabled while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://192.168.99.100:80802/api/demo", host: "192.168.99.100:8080"
2017/02/09 14:48:57 [error] 5#5: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://192.168.99.100:80803/api/demo", host: "192.168.99.100:8080"
2017/02/09 14:48:57 [warn] 5#5: *1 upstream server temporarily disabled while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://192.168.99.100:80803/api/demo", host: "192.168.99.100:8080"
2017/02/09 14:48:57 [error] 5#5: *1 [lua] demo.lua:44: status502 while sending to client, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", host: "192.168.99.100:8080"

對(duì)upstream逐個(gè)請(qǐng)求，都失敗，則capture的subrequest返回502，對(duì)client返回的status code取決于lua腳本

再請(qǐng)求一次

2017/02/09 15:09:34 [error] 6#6: *11 no live upstreams while connecting to upstream, client: 192.168.99.1, server: , request: "POST /api/demo HTTP/1.1", subrequest: "/api/test/demo", upstream: "http://test_server/api/demo", host: "192.168.99.100:8080"

該upstream下面的server都掛的情況下出現(xiàn)no live upstreams while connecting to upstream

ngx_http_upstream_module

nginx中健康檢查(health_check)機(jī)制深入分析

nginx upstream 容錯(cuò)機(jī)制 原創(chuàng)-胡志廣

線上nginx的一次“no live upstreams while connecting to upstream ”分析