猿问

导入错误 Python Hive UDF

我想将一些常量放在一个 Python 文件中,然后将其导入到另一个文件中。我创建了两个文件,一个带有常量,一个导入它,一切都在本地运行良好:


常量.py:


CONST = "hi guy"

test_constants.py:


from constants import CONST

import sys


for line in sys.stdin:

    print(CONST)

本地测试:


$ echo "dummy" | python test_constants.py

hi guy

使用 Hive (beeline) 进行测试:


hive> add file hdfs://path/.../test_constants.py;

No rows affected (0.191 seconds)

hive> add file hdfs://path/.../constants.py;

No rows affected (0.049 seconds)

hive> list files;

resource

/tmp/bb09f878-7e36-4aa2-8566-a30950072bcb_resources/test_constants.py

/tmp/bb09f878-7e36-4aa2-8566-a30950072bcb_resources/constants.py

2 rows selected (0.179 seconds)

hive> with t as (select 1 as dummy) 

  select transform (dummy) 

  using 'python test_constants.py' 

  as dummy_out 

  from t;

Error: org.apache.hive.service.cli.HiveSQLException: 

Error while processing statement: FAILED: 

Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. 

Vertex failed, vertexName=Map 1, vertexId=vertex_1535407036047_170618_1_00, diagnostics=[Task failed, taskId=task_1535407036047_170618_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1535407036047_170618_1_00_000000_0:

java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators


江户川乱折腾
浏览 288回答 1
1回答
随时随地看视频慕课网APP

相关分类

Python
我要回答