JSON Path 2.3.0 与 hadoop 2.7 Environment JSON-smar

Question

1. 在 HDFS 位置输入 JSON

2. 需要解析 JSON 并聚合结果

3. 我正在使用使用 JSON 路径库的 PIG UDF

4. hadoop2.7环境jar上：json-smart1.2,json-path1.2 hardbinded

5. 每当我执行 PIG Mapreduce 时，它都会抛出异常

java.lang.NoSuchFieldError: defaultReader

at com.jayway.jsonpath.spi.json.JsonSmartJsonProvider.<init>(JsonSmartJsonProvider.java:39)

at com.jayway.jsonpath.internal.DefaultsImpl.jsonProvider(DefaultsImpl.java:21)

at com.jayway.jsonpath.Configuration.defaultConfiguration(Configuration.java:174)

at com.jayway.jsonpath.internal.JsonContext.<init>(JsonContext.java:52)

at com.jayway.jsonpath.JsonPath.parse(JsonPath.java:596)

为了解决问题，请尝试以下选项

选项 1：尝试设置注册 json-smart2.3.jar 和 json-path2.3.0.jar 但没有令人满意的结果（因为它引用的 Jar 是 json-path1.2.jar）

选项 2：将我的模块依赖项降级为 json-path1.2.jar 无结果

选项 3：使用自定义类加载器尝试加载 JSON-path2.3.0 jar 的 jar 它加载的类进入了 Org.slf4j 绑定问题

识别出多个绑定路径，但 sun.misc 类加载器出现问题

有人可以建议我解决这个问题吗，可以找到一个 stackoverflow 链接，它讲述了 weblogic 而不是可以应用于 Hadoop2.7 的通用解决方案（JSON Parser -java.lang.NoSuchFieldError: defaultReader）

Answer 1

提供自定义实现来加载/解析JSON，这样我们就可以避免使用Hadoop/lib的Json-smart-2.x/1.x&nbsp; public static void changeJsonPathConfig() {&nbsp; &nbsp; if (!configChanged) {&nbsp; &nbsp; &nbsp; &nbsp; Configuration.setDefaults(new Configuration.Defaults() {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; private final JsonProvider jsonProvider = new GsonJsonProvider(&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; new GsonBuilder().serializeNulls().create());&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; private final MappingProvider mappingProvider = new GsonMappingProvider();&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; @Override&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; public JsonProvider jsonProvider() {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return jsonProvider;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; @Override&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; public MappingProvider mappingProvider() {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return mappingProvider;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; @Override&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; public Set<Option> options() {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return EnumSet.noneOf(Option.class);&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; });&nbsp; &nbsp; &nbsp; &nbsp; configChanged = true;&nbsp; &nbsp; }}

Answer 2

由于 hadoop 环境（Pig、hdfs、Hive 等）正在使用 json-path-2.3.0，其更好的用户 Mapper 逻辑应该使用另一个版本“jsonpath-1.0.jar”将解决问题