摘要：多進(jìn)程的方式可以增加腳本的并發(fā)處理能力，支持這種多進(jìn)程的編程方式在類系統(tǒng)中，的模塊內(nèi)置了函數(shù)用以創(chuàng)建子進(jìn)程方式創(chuàng)建子進(jìn)程執(zhí)行結(jié)果從結(jié)果可以看到，從開(kāi)始，下面的部分代碼運(yùn)行了兩次，第一次是父進(jìn)程運(yùn)行，第二次是子進(jìn)程運(yùn)行，且子進(jìn)程的的結(jié)果總是，

多進(jìn)程的方式可以增加腳本的并發(fā)處理能力， python 支持這種多進(jìn)程的編程方式
在類unix系統(tǒng)中， python的os 模塊內(nèi)置了fork 函數(shù)用以創(chuàng)建子進(jìn)程

import os

print "Process %s start ..." %(os.getpid())

pid = os.fork()

if pid == 0:
    print "This is child process and my pid is %d, my father process is %d" %(os.getpid(), os.getppid())
else:
    print "This is Fater process, And Its child pid is %d" %(pid)

執(zhí)行結(jié)果

Process 4276 start ...
This is Fater process, And Its child pid is 4277
This is child process and my pid is 4277, my father process is 4276

從結(jié)果可以看到， 從pid = os.fork() 開(kāi)始， 下面的部分代碼運(yùn)行了兩次， 第一次是父進(jìn)程運(yùn)行， 第二次是子進(jìn)程運(yùn)行， 且子進(jìn)程的fork的結(jié)果總是0， 所以這個(gè)也可以用來(lái)作為區(qū)分父進(jìn)程或是子進(jìn)程標(biāo)志

那么變量在多個(gè)進(jìn)程之間是否相互影響呢
import os

print "Process %s start ..." %(os.getpid())

pid = os.fork()

source = 10

if pid == 0:
    print "This is child process and my pid is %d, my father process is %d" %(os.getpid(), os.getppid())
    source = source - 6
    print "child process source value is "+str(source)
else:
    print "This is Fater process, And Its child pid is %d" %(pid)
    source = source - 1
    print "father process source value is "+str(source)
print "source value is "+str(source)

執(zhí)行的結(jié)果如下：

Process 4662 start ...
This is Fater process, And Its child pid is 4663
This is child process and my pid is 4663, my father process is 4662
father process source value is 9
child process source value is 4
source value is 9
source value is 4

很明顯， 初始值為10的source 在父進(jìn)程中值 減少了 1, 為9， 而子進(jìn)程明顯source的初始值 是10, 也就是說(shuō)多進(jìn)程之間并沒(méi)有什么相互影響

fork 方式是僅在linux 下才有的接口， 在windows下并沒(méi)有， 那么在windows下如何實(shí)現(xiàn)多進(jìn)程呢， 這就用到了multiprocessing

multiprocessing 模塊的Process 對(duì)象表示的是一個(gè)進(jìn)程對(duì)象， 可以創(chuàng)建子進(jìn)程并執(zhí)行制定的函數(shù)

from multiprocessing import Process
import os

def pro_do(name, func):
    print "This is child process %d from parent process %d, and name is  %s which is used for %s" %(os.getpid(), os.getppid(), name, func)

if __name__ == "__main__":
    print "Parent process id %d" %(os.getpid())
    #process 對(duì)象指定子進(jìn)程將要執(zhí)行的操作方法(pro_do), 以及該函數(shù)的對(duì)象列表args(必須是tuple格式， 且元素與pro_do的參數(shù)一一對(duì)應(yīng))
    pro = Process(target=pro_do, args=("test", "dev"))
    print "start child process"
    #啟動(dòng)子進(jìn)程
    pro.start()
    #是否阻塞方式執(zhí)行， 如果有， 則阻塞方式， 否則非阻塞
    pro.join() #if has this, it"s synchronous operation or asynchronous operation
    print "Process end"

執(zhí)行結(jié)果

Parent process id 4878
start child process
This is child process 4879 from parent process 4878, and name is  test which is used for dev
Process end

如果沒(méi)有pro.join()， 則表示非阻塞方式運(yùn)行， 那么最終的Process end的輸出位置就有可能出現(xiàn)在pro_do 方法執(zhí)行之前了

Parent process id 4903
start child process
Process end
This is child process 4904 from parent process 4903, and name is  test which is used for dev

通過(guò)multiprocessing 的process對(duì)象創(chuàng)建多進(jìn)程， 還可以從主進(jìn)程中向子進(jìn)程傳遞參數(shù)， 例如上面例子中的pro_do的參數(shù)

from multiprocessing import Pool
import os, time

def pro_do(process_num):
    print "child process id is %d" %(os.getpid())
    time.sleep(6 - process_num)
    print "this is process %d" %(process_num)

if __name__ == "__main__":
    print "Current process is %d" %(os.getpid())
    p = Pool()
    for i in range(5):
        p.apply_async(pro_do, (i,))  #增加新的進(jìn)程
    p.close() # 禁止在增加新的進(jìn)程
    p.join()
    print "pool process done"

輸出：

Current process is 19138
child process id is 19139
child process id is 19140
this is process 1
child process id is 19140
this is process 0
child process id is 19139
this is process 2
child process id is 19140
this is process 3
this is process 4
pool process done

其中

   child process id is 19139
   child process id is 19140

是立即輸出的， 后面的依次在等待了sleep的時(shí)間后輸出 ， 之所以立即輸出了上面兩個(gè)是因?yàn)檎OPool 進(jìn)程池默認(rèn)是按照cpu的數(shù)量開(kāi)啟子進(jìn)程的， 我是在虛擬機(jī)中運(yùn)行， 只分配了兩核， 所以先立即啟動(dòng)兩個(gè)子進(jìn)程， 剩下的進(jìn)程要等到前面的進(jìn)程執(zhí)行完成后才能啟動(dòng)。
不過(guò)也可以在p=Poo() 中使用Pool(5)來(lái)指定啟動(dòng)的子進(jìn)程數(shù)量， 這樣輸出就是下面的了：

Current process is 19184
child process id is 19185
child process id is 19186
child process id is 19188
child process id is 19189
child process id is 19187
this is process 4
this is process 3
this is process 2
this is process 1
this is process 0
pool process done

且

Current process is 19184
child process id is 19185
child process id is 19186
child process id is 19188
child process id is 19189
child process id is 19187

都是立即輸出的

父進(jìn)程可以指定子進(jìn)程執(zhí)行的方法及其參數(shù)， 達(dá)到父進(jìn)程向子進(jìn)程傳遞消息的單向通信的目的， 那子進(jìn)程之間或子進(jìn)程怎么向父進(jìn)程通信呢

Queue 是一種方式

from multiprocessing import Process, Queue
import os, time

def write_queue(q):
    for name in ["Yi_Zhi_Yu", "Tony" ,"San"]:
        print "put name %s to queue" %(name)
        q.put(name)
        time.sleep(2)
    print "write data finished"

def read_queue(q):
    print "begin to read data"
    while True:
        name = q.get()
        print "get name %s from queue" %(name)

if __name__ == "__main__":
    q = Queue()
    pw = Process(target=write_queue, args=(q,))
    pr = Process(target=read_queue,args=(q,))

    pw.start()
    pr.start()
    pw.join() #這個(gè)表示是否阻塞方式啟動(dòng)進(jìn)程， 如果要立即讀取的話， 兩個(gè)進(jìn)程的啟動(dòng)就應(yīng)該是非阻塞式的， 所以pw在start后不能立即使用pw.join(), 要等pr start后方可
    pr.terminate() #服務(wù)進(jìn)程，強(qiáng)制停止

結(jié)果

put name Yi_Zhi_Yu to queue
begin to read data
get name Yi_Zhi_Yu from queue
put name Tony to queue
get name Tony from queue
put name San to queue
get name San from queue
write data finished

另外還有Pipe
其原理參見(jiàn)http://ju.outofmemory.cn/entry/106041， 其只能作為兩個(gè)進(jìn)程之間的通信

#!/usr/bin/env python
#encoding=utf-8

from multiprocessing import Process,Pipe
import os,time,sys

def send_pipe(p):
    names = ["Yi_Zhi_Yu", "Tony", "San"]
    for name in names:
        print "put name %s to Pipe" %(name)
        p.send(name)
        time.sleep(1)
def recv_pipe(p):
    print "Try to read data in pipe"
    while True:
            name = p.recv()
            print "get name %s from pipe" %(name)

if __name__ == "__main__":
   #pipe, one for send, one for read
   ps_pipe, pr_pipe = Pipe()
   #process
   ps = Process(target=send_pipe, args=(ps_pipe,))
   pr = Process(target=recv_pipe, args=(pr_pipe,))
   pr.start()
   ps.start()
   ps.join()
   pr.terminate()

在實(shí)例化Pipe的時(shí)候， 會(huì)產(chǎn)生兩個(gè)ps_pipe(read-write Connection, handle 5)， pr_pipe(read-write Connection, handle 5) , 都可以作為發(fā)送或者接受方， 一旦一個(gè)確認(rèn)為攻， 另一個(gè)自然就是受了(之所以Pipe只能作為兩個(gè)進(jìn)程之間的通信方式， 原因也許就是這個(gè))，產(chǎn)生的結(jié)果如下

Try to read data in pipe
put name Yi_Zhi_Yu to Pipe
get name Yi_Zhi_Yu from pipe
put name Tony to Pipe
get name Tony from pipe
put name San to Pipe
get name San from pipe

還有一種Array, Value 的形式， 暫且不表， 有時(shí)間在折騰

以上均為python 學(xué)習(xí)筆記和練習(xí)， 如有錯(cuò)誤， 歡迎指出