导入错误 Python Hive UDF

Question

我想将一些常量放在一个 Python 文件中，然后将其导入到另一个文件中。我创建了两个文件，一个带有常量，一个导入它，一切都在本地运行良好：

常量.py：

CONST = "hi guy"

test_constants.py：

from constants import CONST

import sys

for line in sys.stdin:

    print(CONST)

本地测试：

$ echo "dummy" | python test_constants.py

hi guy

使用 Hive (beeline) 进行测试：

hive> add file hdfs://path/.../test_constants.py;

No rows affected (0.191 seconds)

hive> add file hdfs://path/.../constants.py;

No rows affected (0.049 seconds)

hive> list files;

resource

/tmp/bb09f878-7e36-4aa2-8566-a30950072bcb_resources/test_constants.py

/tmp/bb09f878-7e36-4aa2-8566-a30950072bcb_resources/constants.py

2 rows selected (0.179 seconds)

hive> with t as (select 1 as dummy) 

  select transform (dummy) 

  using 'python test_constants.py' 

  as dummy_out 

  from t;

Error: org.apache.hive.service.cli.HiveSQLException: 

Error while processing statement: FAILED: 

Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. 

Vertex failed, vertexName=Map 1, vertexId=vertex_1535407036047_170618_1_00, diagnostics=[Task failed, taskId=task_1535407036047_170618_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1535407036047_170618_1_00_000000_0:

java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators