tornado的mongo驱动选择,pymongo,motor,asyncmongo还是其他?

背景
我用apache的abtest在公司的两台虚拟机上面测试,发现用pymongo的速度最快,asyncmongo其次,最后才是motor库.囧
機器配置
server端
ubuntu12.04
Intel(R)Core(TM)2DuoCPUE7500@2.93GHz
內存500M
基本的server配置全部打開˜(比如maxsoconn设到很大)
client端
freebsd(机子在公司,暂时空缺=.=)
測試工具
abtest
测试用例
我这里只贴了asyncmongo的用例,其他两个的代码结构类似,但跟其他业务结合得比较紧,所以就不贴了.大概的用例描述:客户端发起一个json格式的post请求,tornado这边根据player_id跟mongo要数据,只是读请求,不存在锁的问题.
删了一些敏感的信息.
#!/usr/bin/envpython
#encoding:utf-8
importasyncmongo
importtornado.web
fromtornadoimportweb
importtornado.ioloop
fromtornado.ioloopimportIOLoop
fromtornado.httpserverimportHTTPServer
classRankHandler(tornado.web.RequestHandler):
def__init__(self,application,request,**kwargs):
super(RankHandler,self).__init__(application,request,**kwargs)
self.set_header('Content-Type','application/json')
@property
defdb(self):
returnself.application.db
@tornado.web.asynchronous
defpost(self):
r={}
##decodemsgbody
try:
d=tornado.escape.json_decode(self.request.body)
exceptValueError,e:
self.log.error('decodetrackdataerror.e=%s'%e)
r['status_code']=500
r['status_txt']='decodejsonerror'
self.write(tornado.escape.json_encode(r))
self.finish()
return
event=d.get('event')
ifnotevent:
self.log.error('trackargsmissingargevent.')
r['status_code']=500
r['status_txt']='missing_arg_event'
self.write(tornado.escape.json_encode(r))
self.finish()
return
event_data=d.get('data')
ifevent_dataandnotisinstance(event_data,dict):
self.log.error('trackargsbadargdata.')
r['status_code']=500
r['status_txt']='bad_arg_data'
self.write(tornado.escape.json_encode(r))
self.finish()
return
if(event=="u_add"):
pass
elif(event=="u_group"):
pass
elif(event=="u_update"):
pass
elif(event=="u_get_point"):
self.db.ranking_list.find_one({"_id":event_data["player_id"]},callback=self._on_response)
def_on_response(self,response,error):
r={}
iferror:
raisetornado.web.HTTPError(500)
result={"data":{"_id":response['_id'],"rank_point":response["rank_point"]}}
r.update(result)
ifnotr.get('status_code',None):
r['status_code']=200
r['status_txt']='OK'
self.write(tornado.escape.json_encode(r))
self.finish()
return
classApplication(web.Application):
def__init__(self):
"""
"""
handlers=[
(r"/api/xxx",RankHandler),
]
settings=dict(
debug=True,
autoescape=None,
)
super(Application,self).__init__(handlers,**settings)
self.db=asyncmongo.Client(pool_id='mydb',host='0.0.0.0',port=27017,maxcached=10,maxconnections=20000,dbname='xxx')
defmain():
http_server=HTTPServer(Application(),xheaders=True)
http_server.bind(8880,'127.0.0.1')
http_server.start()
IOLoop.instance().start()
if__name__=="__main__":
main()
测试结果
同步mongo庫
➜testgit:(master)✗ab-n10000-c3000-pdata-get-user-rank_point.xml-T'application/json''http://192.168.0.201:8880/api/xxx'
ThisisApacheBench,Version2.3<$Revision:1430300$>
Copyright1996AdamTwiss,ZeusTechnologyLtd,http://www.zeustech.net/
LicensedtoTheApacheSoftwareFoundation,http://www.apache.org/
Benchmarking192.168.0.201(bepatient)
Completed1000requests
Completed2000requests
Completed3000requests
Completed4000requests
Completed5000requests
Completed6000requests
Completed7000requests
Completed8000requests
Completed9000requests
Completed10000requests
Finished10000requests
ServerSoftware:TornadoServer/3.1.1
ServerHostname:192.168.0.201
ServerPort:8880
DocumentPath:/api/xxx
DocumentLength:80bytes
ConcurrencyLevel:3000
Timetakenfortests:23.551seconds
Completerequests:10000
Failedrequests:0
Writeerrors:0
Totaltransferred:2170000bytes
Totalbodysent:1990000
HTMLtransferred:800000bytes
Requestspersecond:424.61[#/sec](mean)
Timeperrequest:7065.317[ms](mean)
Timeperrequest:2.355[ms](mean,acrossallconcurrentrequests)
Transferrate:89.98[Kbytes/sec]received
82.52kb/ssent
172.50kb/stotal
ConnectionTimes(ms)
minmean[+/-sd]medianmax
Connect:118062222.9106110825
Processing:26511302042.653920975
Waiting:25510402018.951520972
Total:28229362824.0293020976
Percentageoftherequestsservedwithinacertaintime(ms)
50%2930
66%3402
75%3526
80%3592
90%6670
95%6823
98%9961
99%15001
100%20976(longestrequest)
異步mongo庫
(q2_rank)➜testgit:(master)✗ab-n10000-c3000-pdata-get-user-rank_point.xml-T'application/json''http://192.168.0.201:8880/api/xxx'
ThisisApacheBench,Version2.3<$Revision:1430300$>
Copyright1996AdamTwiss,ZeusTechnologyLtd,http://www.zeustech.net/
LicensedtoTheApacheSoftwareFoundation,http://www.apache.org/
Benchmarking192.168.0.201(bepatient)
Completed1000requests
Completed2000requests
Completed3000requests
Completed4000requests
Completed5000requests
Completed6000requests
Completed7000requests
Completed8000requests
Completed9000requests
Completed10000requests
Finished10000requests
ServerSoftware:TornadoServer/3.1.1
ServerHostname:192.168.0.201
ServerPort:8880
DocumentPath:/api/xxx
DocumentLength:80bytes
ConcurrencyLevel:3000
Timetakenfortests:24.629seconds
Completerequests:10000
Failedrequests:0
Writeerrors:0
Totaltransferred:2170000bytes
Totalbodysent:1990000
HTMLtransferred:800000bytes
Requestspersecond:406.02[#/sec](mean)
Timeperrequest:7388.749[ms](mean)
Timeperrequest:2.463[ms](mean,acrossallconcurrentrequests)
Transferrate:86.04[Kbytes/sec]received
78.90kb/ssent
164.95kb/stotal
ConnectionTimes(ms)
minmean[+/-sd]medianmax
Connect:0412794.5176205
Processing:102462861793.2708810475
Waiting:83662561843.9708310468
Total:103266981894.1719914014
Percentageoftherequestsservedwithinacertaintime(ms)
50%7199
66%7700
75%7825
80%7875
90%8244
95%9161
98%10366
99%10763
100%14014(longestrequest)
98%6677
結果分析
理論上來說,異步的mongo應該可以處理更多的併發連接,但實際測試來看,併發連接數相差不大.我想到的原因:
-测试机子太挫
-測試的數據量不夠大,如果增大返回的數據大小,或者延長查詢的時間,這樣理論上motor性能會更好看點
-我查看了mongod的日志,发现异步的驱动,每次都会跟mongo新生成一个连接,频繁的连接建立与删除,造成了性能上的损耗?而pymongo则从头到尾都是一个连接.
最后
有没童鞋做过这方面的性能测试呢,可否把测试结果分享下=.=?
然后,大家能否推荐下一些比较好的结合方式?比如我在知乎上面看到过有人问类似的问题.
qq_遁去的一_1
浏览 363回答 2
2回答
打开App,查看更多内容
随时随地看视频慕课网APP

相关分类

JavaScript