如何在代码中执行spark

Question

如何在代码中执行spark

Answer 1

写Linux脚本#!/bin/shif [ $# -lt 2 ] ;thenecho "Useage:$0 <hdfs_input_file> $1 <hdfs_output_file>"exitelseecho "wordcount is starting!"spark-submit \--master spark://spark:7077 \--class WordCount \--executor-memory 512m \--total-executor-cores 6 \~/input/WordCount.jar \"$1" \"$2"echo "wordcount is stopped!"fi