摘要：前言本文為篤行日常工作記錄，爛筆頭系列。最終通過分析源碼了解到最終的確定是一個協(xié)商的過程，而不是簡單的配置生效。根據(jù)客戶端上報的和服務(wù)端自身的。如果上報的小于則設(shè)置為如果上報的大于則設(shè)置為如果介于兩則之間，則以上報的時間為準(zhǔn)。

本文為篤行日常工作記錄，爛筆頭系列。

源碼前面，了無秘密 — by 侯杰

近期的一個C++項目里使用了Zookeeper做服務(wù)發(fā)現(xiàn)，期間遇到了SessionTimeOut問題的困擾，明明通過zookeeper c client設(shè)置了超時時間，但無效。

請原諒我一開始對zookeeper不熟悉。最終通過分析源碼了解到SessionTimeOut最終的確定是一個協(xié)商的過程，而不是簡單的配置生效。

在這里記錄下Session超時時間的有關(guān)分析,基于zookeeper 3.4.8

項目中使用的是 C client,通過zookeer_init 創(chuàng)建zk session，調(diào)用了zookeeper_init其實就開始了建立鏈接到ZK集群的過程，這里設(shè)置的recv_timeout 為客戶端所期望的session超時時間，單位為毫秒。

ZOOAPI zhandle_t *zookeeper_init(const char *host, watcher_fn fn,
  int recv_timeout, const clientid_t *clientid, void *context, int flags);

連接成功之后客戶端發(fā)起握手協(xié)議,可以看到之前設(shè)置的recv_timeout隨握手協(xié)議一起發(fā)送給服務(wù)端,zookeeper.c#L1485。

static int prime_connection(zhandle_t *zh)
{
    int rc;
    /*this is the size of buffer to serialize req into*/
    char buffer_req[HANDSHAKE_REQ_SIZE];
    int len = sizeof(buffer_req);
    int hlen = 0;
    struct connect_req req;
    req.protocolVersion = 0;
    req.sessionId = zh->seen_rw_server_before ? zh->client_id.client_id : 0;
    req.passwd_len = sizeof(req.passwd);
    memcpy(req.passwd, zh->client_id.passwd, sizeof(zh->client_id.passwd));
    req.timeOut = zh->recv_timeout; <-這里設(shè)置timeOut
    req.lastZxidSeen = zh->last_zxid;
    req.readOnly = zh->allow_read_only;
    hlen = htonl(len);
    /* We are running fast and loose here, but this string should fit in the initial buffer! */
    rc=zookeeper_send(zh->fd, &hlen, sizeof(len));
    serialize_prime_connect(&req, buffer_req);
    rc=rc<0 ? rc : zookeeper_send(zh->fd, buffer_req, len);
    if (rc<0) {
        return handle_socket_error_msg(zh, __LINE__, ZCONNECTIONLOSS,
                "failed to send a handshake packet: %s", strerror(errno));
    }

再來看看處理握手協(xié)議Resp的邏輯 zookeeper.c L1767

static int check_events(zhandle_t *zh, int events)
{
    if (zh->fd == -1)
        return ZINVALIDSTATE;
  ……
  ……
  ……
   deserialize_prime_response(&zh->primer_storage, zh->primer_buffer.buffer);
                /* We are processing the primer_buffer, so we need to finish
                 * the connection handshake */
                oldid = zh->client_id.client_id;
                newid = zh->primer_storage.sessionId;
                if (oldid != 0 && oldid != newid) {
                    zh->state = ZOO_EXPIRED_SESSION_STATE;
                    errno = ESTALE;
                    return handle_socket_error_msg(zh,__LINE__,ZSESSIONEXPIRED,
                            "sessionId=%#llx has expired.",oldid);
                } else {
                    zh->recv_timeout = zh->primer_storage.timeOut; //設(shè)置為Resp的Timeout
                    zh->client_id.client_id = newid;
                 
}

至此可以發(fā)現(xiàn)，最終客戶端的SessionTimeOut時間實際是經(jīng)過服務(wù)端下發(fā)之后的，并不一定是最先設(shè)置的。

來看看服務(wù)端握手的處理邏輯ZooKeeperServer.java#L876。

public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
        BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
        ConnectRequest connReq = new ConnectRequest();
        connReq.deserialize(bia, "connect");
……
……
……
  //根據(jù)客戶端上報的timeout和服務(wù)端自身的minSessionTimeOut。
  //如果上報的timeout小于minSessionTimeOut則 設(shè)置timeout為minSessionTimeOut.
  //如果上報的timeout大于maxSessionTimeOut則 設(shè)置timeout為maxSessionTimeOut.
  //如果介于兩則之間，則以上報的時間為準(zhǔn)。
        int sessionTimeout = connReq.getTimeOut();
        byte passwd[] = connReq.getPasswd();
        int minSessionTimeout = getMinSessionTimeout();
        if (sessionTimeout < minSessionTimeout) {
            sessionTimeout = minSessionTimeout;
        }
        int maxSessionTimeout = getMaxSessionTimeout();
        if (sessionTimeout > maxSessionTimeout) {
            sessionTimeout = maxSessionTimeout;
        }
        cnxn.setSessionTimeout(sessionTimeout);
……
……
……
    }

可以一句話概括，客戶端上報的期望timeout一定要在服務(wù)端設(shè)置的上下界之間，如果越過邊界，則以邊界為準(zhǔn)。

繼續(xù)看ZooKeeperServer.java#L104和ZooKeeperServer.java#L791

    public static final int DEFAULT_TICK_TIME = 3000;
    protected int tickTime = DEFAULT_TICK_TIME;
    /** value of -1 indicates unset, use default */
    protected int minSessionTimeout = -1;
    /** value of -1 indicates unset, use default */
    protected int maxSessionTimeout = -1;
    protected SessionTracker sessionTracker;

tickTime為3000毫秒，minSessionTimeOut和maxSessionTimeOut缺省值為-1

 public int getTickTime() {
        return tickTime;
    }

    public void setTickTime(int tickTime) {
        LOG.info("tickTime set to " + tickTime);
        this.tickTime = tickTime;
    }

    public int getMinSessionTimeout() {
        return minSessionTimeout == -1 ? tickTime * 2 : minSessionTimeout;
      //如果minSessionTimeOut為缺省值這設(shè)置minSessionTimeOut為2倍tickTime
    }

    public void setMinSessionTimeout(int min) {
        LOG.info("minSessionTimeout set to " + min);
        this.minSessionTimeout = min;
    }

    public int getMaxSessionTimeout() {
        return maxSessionTimeout == -1 ? tickTime * 20 : maxSessionTimeout;
      //如果maxSessionTimeout為缺省值則設(shè)置maxSessionTimeout為20倍tickTime
    }

    public void setMaxSessionTimeout(int max) {
        LOG.info("maxSessionTimeout set to " + max);
        this.maxSessionTimeout = max;
    }

可以知道m(xù)inSessionTimeOut和maxSessionTimeOut在缺省的時候則跟tickTime有關(guān)，分別為2和20倍tickTime，繼續(xù)分析。ZooKeeperServer.java#L160

  public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
            int minSessionTimeout, int maxSessionTimeout,
            DataTreeBuilder treeBuilder, ZKDatabase zkDb) {
        serverStats = new ServerStats(this);
        this.txnLogFactory = txnLogFactory;
        this.zkDb = zkDb;
        this.tickTime = tickTime;
        this.minSessionTimeout = minSessionTimeout;
        this.maxSessionTimeout = maxSessionTimeout;
        
        LOG.info("Created server with tickTime " + tickTime
                + " minSessionTimeout " + getMinSessionTimeout()
                + " maxSessionTimeout " + getMaxSessionTimeout()
                + " datadir " + txnLogFactory.getDataDir()
                + " snapdir " + txnLogFactory.getSnapDir());
    }

tickTime、minSessionTimeOut、maxSessionTimeOut實際構(gòu)造函數(shù)傳入，當(dāng)然還有一個無參構(gòu)造函數(shù)以及一些setter和getter可以設(shè)置這幾個參數(shù)。

繼續(xù)分析ZooKeeperServerMain.java#L94

 public void runFromConfig(ServerConfig config) throws IOException {
        LOG.info("Starting server");
        FileTxnSnapLog txnLog = null;
        try {
            // Note that this thread isn"t going to be doing anything else,
            // so rather than spawning another thread, we will just call
            // run() in this thread.
            // create a file logger url from the command line args
            ZooKeeperServer zkServer = new ZooKeeperServer();

            txnLog = new FileTxnSnapLog(new File(config.dataLogDir), new File(
                    config.dataDir));
            zkServer.setTxnLogFactory(txnLog);
            zkServer.setTickTime(config.tickTime);
          //我們可以發(fā)現(xiàn)實際運行的幾個參數(shù)除了默認(rèn)值以外，可以通過配置文件來配置生效。
            zkServer.setMinSessionTimeout(config.minSessionTimeout);
            zkServer.setMaxSessionTimeout(config.maxSessionTimeout);
            cnxnFactory = ServerCnxnFactory.createFactory();
            cnxnFactory.configure(config.getClientPortAddress(),
                    config.getMaxClientCnxns());
            cnxnFactory.startup(zkServer);
            cnxnFactory.join();
            if (zkServer.isRunning()) {
                zkServer.shutdown();
            }
        } catch (InterruptedException e) {
            // warn, but generally this is ok
            LOG.warn("Server interrupted", e);
        } finally {
            if (txnLog != null) {
                txnLog.close();
            }
        }
    }

到此問題就明了了，我們可以通過配置來修改SessionTimeOut，默認(rèn)配置文件只配置了tickTime，如下。

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

經(jīng)過源碼分析，得出SessionTimeOut的協(xié)商如下：

情況1: 配置文件配置了maxSessionTimeOut和minSessionTimeOut

最終SessionTimeOut,必須在minSessionTimeOut和maxSessionTimeOut區(qū)間里，如果跨越上下界，則以跨越的上屆或下界為準(zhǔn)。

情況2:配置文件沒有配置maxSessionTimeOut和minSessionTimeOut

maxSessionTimeout沒配置則 maxSessionTimeOut設(shè)置為 20 * tickTime

minSessionTimeOut沒配置則 minSessionTimeOut設(shè)置為 2 * tickTime

也就是默認(rèn)情況下, SessionTimeOut的合法范圍為 4秒~40秒，默認(rèn)配置中tickTime為2秒。

如果tickTime也沒配置，那么tickTime缺省為3秒。

遇到問題從源碼分析一定是最好的，能使得理解更深入記憶更深刻。

最后 ending...如有不足請指點，亦可留言或聯(lián)系 [email protected].

本文為篤行原創(chuàng)文章首發(fā)于大題小作,永久鏈接:篤行雜記之Zookeeper SessionTimeOut分析

https://www.ifobnn.com/zookeepersessiontimeout.html