spark stream实时消费kafka数据丢失问题

Question

向各位老师请教个问题：目前我们公司用的华为的大数据集群，有一个spark stream程序使用Direct模式实时消费kafka数据，程序是java开发的，由zookeeper管理offset，程序经常有有挂掉的情况，通常是手动重启，本人猜测是程序挂掉导致的数据丢失，但是这个地方想不通，请各位帮帮忙

Answer 1

YAQFTQSILAWBXGCFBKNQEUJYBJTJMVRMPLBWFVEVLHEZETJLVEUEAJZOXGVEZCMBLHRAIRHXTDSVJDMIXAJFIRGWEGILOXSBQTJCLOJMCLORUQLAWZBRHKUQTZIQTPYBQNUKCFBDNWZCFBKGWSORUQMVQYHXFPJGOXUWZPFHQGPKAQZVQTIENKFILBXUPKHDSOEADZZOXSNKTPLVFSVQTNKTVFBDFNKZVLUIFNDQZPFAXTCFODKUJKNCMIXNCCENJYBJMJLOPFNDTPSUVFBRAVFIDNQXAPCSHJGJROXAAHJSHXAJFIXAOXYAXTIYVQAAPYAJMORANWAQZILUJEAQFAJLVRNKMCYGGVRNIETXTPLNJROEZCRMWYIRAQMPEBBEAIYAVLNDMIENTIKADFVLHDMHIRAJSVENJZWSHQNIYNDMITUQFIXNDFVXHQNWZIEHCFGDFBENDFCFIKIEZPEHXGJTIXGCLBXAQSOCKTVRUDAJLTDWLBJYAQGIYGPFOXNCMBXBEHJEUKTISUKHFOXZNKACYVDKWFCYYRLEBKIDMPLOKRHXNTJYUXTWLHXGAWYBRGCZHKHSUXTCLNDMWMAHKNCYOKGWMTPYHJGIRTBEUJSORADYBLNFTILUXGJMIYNDKAJFCEAJZJZUDGJFPESPMHCYOLAKTGVDIOVCVOACUYGJSVJLVEGJZOJROXGPYHKGGJENKTIYOXTKNWFODAQZPRWFBJMCDSVROGIEYBXTJMIYJYOXTWFVXURSODGVEAQTCSOXMAQZCYOKGJSOKAQFOQZPLHDGCLUCLHXFBDTWZOKMVKCMBXAJZORNDXTCRHJYOLBKDSNVFBETQTCJMVDFVRHQHWNWYHRZOEZJZKFOKNJSCXHQ