摘要：手動(dòng)配置本地卷此節(jié)介紹在不使用情況下如何配置手動(dòng)本地卷。主機(jī)掛載本地卷如本例在主機(jī)配置本地卷，需將目錄手動(dòng)掛載到目錄，如環(huán)境設(shè)置權(quán)限部署可選。

如Hadoop、ES等系統(tǒng)，其DataNode需大量存儲(chǔ)，且其本身提供了冗余功能，那么我們就沒(méi)必要讓其從存儲(chǔ)系統(tǒng)中分配卷，而是像裸機(jī)部署一樣讓其使用本地節(jié)點(diǎn)上的存儲(chǔ)，local volumes出現(xiàn)之前，我們可使用HostPath掛載卷到容器中，但此方案有很多局限性：

The prior mechanism of accessing local storage through hostPath volumes had many challenges. hostPath volumes were difficult to use in production at scale: operators needed to care for local disk management, topology, and scheduling of individual pods when using hostPath volumes, and could not use many Kubernetes features (like StatefulSets). Existing Helm charts that used remote volumes could not be easily ported to use hostPath volumes. The Local Persistent Volumes feature aims to address hostPath volumes’ portability, disk accounting, and scheduling challenges.

注意：本地卷僅適用于少量應(yīng)用，如同HostPath一樣Pod被綁定到特定的主機(jī)上，若主機(jī)異常，則Pod沒(méi)法調(diào)度到其他節(jié)點(diǎn)，其適用場(chǎng)景：

Caching of datasets that can leverage data gravity for fast processing
Distributed storage systems that shard or replicate data across multiplenodes. Examples include distributed datastores like Cassandra, or distributedfile systems like Gluster or Ceph.

localvolume與hostpath類(lèi)似，但其更靈活且統(tǒng)一，如應(yīng)用使用hostpath，則只能在yaml文件中硬編碼，而localvolume如同普通pvc靈活可用，具體見(jiàn)下文。

此節(jié)介紹在不使用local provisioner情況下如何配置手動(dòng)本地卷。

管理員必須為本地卷創(chuàng)建一個(gè)存儲(chǔ)類(lèi)，如下所示，創(chuàng)建了一個(gè)名為local-storage的存儲(chǔ)類(lèi)：

% kubectl create -f - <
注意：

此存儲(chǔ)類(lèi)無(wú)法動(dòng)態(tài)提供存儲(chǔ)功能，所有PV需手動(dòng)創(chuàng)建；

volumeBindingMode: WaitForFirstConsumer：Pod調(diào)度前不先綁定PVC與PV，而是等待Pod被調(diào)度時(shí)，這樣可根據(jù)Pod資源等請(qǐng)求合理調(diào)度，如：selectors, affinity and anti-affinity policies；

宿主機(jī)準(zhǔn)備目錄，即配置本地硬盤(pán)。如實(shí)驗(yàn)環(huán)境okd-c01與okd-c02主機(jī)均配置了/mnt/test本地存儲(chǔ)，而對(duì)于OKD集群需設(shè)置SeLinux權(quán)限：
chcon -R unconfined_u:object_r:svirt_sandbox_file_t:s0 /mnt/test
手動(dòng)創(chuàng)建兩PV：example-local-pv-1、example-local-pv-2分別綁定兩主機(jī)存儲(chǔ)，如綁定okd-c01.zyl.io主機(jī)的本地卷如下：
% kubectl create -f - <
創(chuàng)建PVC：
% kubectl create -f - <
注意：此時(shí)PVC不會(huì)立馬綁定到PV，其等待Pod被調(diào)度時(shí)才會(huì)綁定到PV上，當(dāng)前PVC狀態(tài)為Pending：
% oc describe pvc example-local-claim
...
Events:
  Type    Reason                Age               From                         Message
  ----    ------                ----              ----                         -------
  Normal  WaitForFirstConsumer  5s (x2 over 10s)  persistentvolume-controller  waiting for first consumer to be created before binding
配置應(yīng)用使用存儲(chǔ)：
% kubectl create -f - <<"EOF"
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: local-volume-test
  name: local-volume-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: local-volume-test
  template:
    metadata:
      labels:
        app: local-volume-test
    spec:
      containers:
      - image: busybox
        name: local-volume-test
        imagePullPolicy: IfNotPresent
        command: [ "/bin/sh", "-c", "while true; do sleep 2; echo $(date) $(hostname)>> /mnt/test/test.log; done" ]
        volumeMounts:
        - name: local-data
          mountPath: /mnt/test
      volumes:
      - name: local-data
        persistentVolumeClaim:
          claimName: example-local-claim
EOF
Pod調(diào)度后會(huì)發(fā)現(xiàn)PVC被綁定：
% oc get pvc example-local-claim 
NAME                  STATUS    VOLUME             CAPACITY   ACCESS MODES STORAGECLASS    AGE
example-local-claim   Bound     example-local-pv-2 1Gi        RWO          local-storage   7m
注意：

此時(shí)刪除Pod，可發(fā)現(xiàn)其仍然被調(diào)度到本地存儲(chǔ)example-local-pv-2所在的主機(jī)，即okd-c02.zyl.io；
刪除deployment后，PVC不會(huì)與PV解綁，即第一次Pod調(diào)度后，PVC就與PV關(guān)聯(lián)了；
刪除PVC后，發(fā)現(xiàn)PV一直處于Released狀態(tài)，導(dǎo)致PV無(wú)法被重用，需管理員手動(dòng)刪除PV并重建PV以便被重用：

When local volumes are manually created like this, the only supported persistentVolumeReclaimPolicy is “Retain”. When the PersistentVolume is released from the PersistentVolumeClaim, an administrator must manually clean up and set up the local volume again for reuse.
自動(dòng)配置本地卷
手動(dòng)配置本地券中描述的PV需手動(dòng)創(chuàng)建，而PVC刪除后PV沒(méi)法重用，即需重建PV才行，當(dāng)系統(tǒng)使用大量Local Volume時(shí)會(huì)加重管理負(fù)擔(dān)，鑒于此，可通過(guò)external static provisioner協(xié)助簡(jiǎn)化local volume配置。
當(dāng)前版本（<=K8S 1.12及OKD 1.11）管理員需手動(dòng)在主機(jī)上將volume掛載到特定目錄上，而后external static provisioner會(huì)掃描此目錄，其將自動(dòng)創(chuàng)建PV，而當(dāng)PVC被釋放后又會(huì)清理目錄內(nèi)容并重建PV，其是半自動(dòng)化的，但并非動(dòng)態(tài)提供，后續(xù)版本會(huì)提供動(dòng)態(tài)提供支持，如管理員僅需提供LVM VG，此provisioner將自動(dòng)完成格式化、掛載等步驟：
Dynamic provisioning of local volumes using LVM is under design and an alpha implementation will follow in a future release. This will eliminate the current need for an administrator to pre-partition, format and mount local volumes, as long as the workload’s performance requirements can tolerate sharing disks.
參考：


Local Persistent Volumes for Kubernetes Goes Beta；

[kubernetes-sigs/sig-storage-local-static-provisioner：K8S官方提供的external static provisioner；

Configuring Local Volumes：OKD如何配置local provisioner；

以下描述如何使用OKD提供的local provisioner。
主機(jī)掛載本地卷
如本例在okd-c0[1-3]主機(jī)配置本地卷，需將目錄手動(dòng)掛載到/mnt/local-storage//目錄，如：
cat >> /etc/fstab <
OKD/Openshift環(huán)境設(shè)置SeLinux權(quán)限：
chcon -R unconfined_u:object_r:svirt_sandbox_file_t:s0 /mnt/local-storage/
部署local provisioner

可選。在多帶帶的項(xiàng)目下部署local provisioner，創(chuàng)建項(xiàng)目：
oc new-project local-storage
創(chuàng)建一個(gè)ConfigMap，其被external provisioner使用，用于描述存儲(chǔ)類(lèi)：
% kubectl create -f - <<"EOF"
apiVersion: v1
kind: ConfigMap
metadata:
  name: local-volume-config
data:
    storageClassMap: |
        local-storage-1:
            hostDir: /mnt/local-storage/local-storage-1
            mountDir: /mnt/local-storage/local-storage-1
        local-storage-2:
            hostDir: /mnt/local-storage/local-storage-2
            mountDir: /mnt/local-storage/local-storage-2
EOF
注意：


local-storage-1：為StorageClass名稱(chēng)，與/mnt/local-storage/對(duì)應(yīng)；

hostDir：本機(jī)目錄；

moutDir：external provisioner將hostDir掛載到pod中的目錄，與hostDir保持一致即可。

provisioner以root權(quán)限運(yùn)行，OKD集群需賦權(quán)：
oc create serviceaccount local-storage-admin
oc adm policy add-scc-to-user privileged -z local-storage-admin
OKD集群安裝模板：
oc create -f https://raw.githubusercontent.com/openshift/origin/release-3.11/examples/storage-examples/local-examples/local-storage-provisioner-template.yaml
從以上模板安裝應(yīng)用（PS：其以DS模式運(yùn)行在所有計(jì)算節(jié)點(diǎn)上）：
oc new-app -p CONFIGMAP=local-volume-config 
  -p SERVICE_ACCOUNT=local-storage-admin 
  -p NAMESPACE=local-storage 
  -p PROVISIONER_IMAGE=quay.io/external_storage/local-volume-provisioner:latest 
  local-storage-provisioner
創(chuàng)建所需的StorageClass：
% kubectl create -f - <<"EOF"
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: local-storage-1
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF

% kubectl create -f - <<"EOF"
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
 name: local-storage-2
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
EOF
注意：StorageClass創(chuàng)建完成后，PV才會(huì)被自動(dòng)創(chuàng)建。
接著，我們手動(dòng)創(chuàng)建PVC或者在Statefulset中使用此存儲(chǔ)卷：
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: example-local-claim
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: local-storage-1
  resources:
    requests:
      storage: 5Gi

---
kind: StatefulSet
...
 volumeClaimTemplates:
  - metadata:
      name: example-local-claim
    spec:
      accessModes:
      - ReadWriteOnce
      storageClassName: local-storage-1
      resources:
        requests:
          storage: 5Gi