工作中遇到的hive常见报错及问题

于 2021-11-25 09:23:35 发布
阅读量1.6w 收藏 2
点赞数 1
文章标签： hive big data hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_42223833/article/details/120767512
版权
本文详细讲述了Hive CLI中遇到的各种问题，包括元数据服务未启动、JDBC驱动版本不匹配导致的SQL错误、以及运行时的系统变量问题。提供了启动MetastoreServer、更换JDBC驱动和配置临时目录的解决方法。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

1，执行#hive命令进入Hive CLI时报如下错误：

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

原因：因为没有正常启动Hive 的 Metastore Server服务进程。 
解决方法：启动Hive 的 Metastore Server服务进程，执行如下命令：

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OPTION SQL_SELECT_LIMIT=DEFAULT' at line 1
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
        at org.datanucleus.api.jdo.JDOPersistenceManager.getDataStoreConnection(JDOPersistenceManager.java:2259)
        at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.trySetAnsiQuotesForMysql(MetaStoreDirectSql.java:166)

注：碰到上述问题执行：--service metastore后报版本不匹配问题

解决方式：

把mysql-connector-java-5.0.3-bin.jar换成mysql-connector-java-5.1.34-bin.jar，问题解决。

或者直接进入mysql直接修改VERSION表即可。

2，执行#hive命令进入Hive CLI时报如下错误：

  1. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OPTION SQL_SELECT_LIMIT=DEFAULT' at line 1

  2. at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)

  3. at org.datanucleus.api.jdo.JDOPersistenceManager.getDataStoreConnection(JDOPersistenceManager.java:2259)

  4. at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.trySetAnsiQuotesForMysql(MetaStoreDirectSql.java:166)

原因：可能是因为Hive连接Mysql的JDBC驱动版本有问题。Mysql版本是5.6.23，用的JDBC驱动是mysql-connector-java-5.0.3-bin.jar。 
解决方法：把mysql-connector-java-5.0.3-bin.jar换成mysql-connector-java-5.1.34-bin.jar，问题解决。

3，执行#hive命令进入Hive CLI时报如下错误： 
在hive1.1版本会有如下错：

  1. # hive

  3. ......

  4. Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

  5. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:472)

  6. at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)

  7. at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)

  8. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  9. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  10. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  11. at java.lang.reflect.Method.invoke(Method.java:606)

  12. at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

  13. at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

  14. Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

  15. at org.apache.hadoop.fs.Path.initialize(Path.java:206)

  16. at org.apache.hadoop.fs.Path.<init>(Path.java:172)

  17. at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:515)

  18. at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:458)

  19. ... 8 more

  20. Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

  21. at java.net.URI.checkPath(URI.java:1804)

  22. at java.net.URI.<init>(URI.java:752)

  23. at org.apache.hadoop.fs.Path.initialize(Path.java:203)

  24. ... 11 more

原因：system:java.io.tmpdir 和system:user.name提示需要使用绝对路径

Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

解决方法： 
将conf/hive-site.xml中所有的system:java.io.tmpdir 和system:user.name都换成绝对路径。 
Hive-1.1.0版本中有以下四处：

  1. <property>

  2. <name>hive.exec.local.scratchdir</name>

  3. <value>${system:java.io.tmpdir}/${system:user.name}</value>

  4. <description>Local scratch space for Hive jobs</description>

  5. </property>

  6. <property>

  7. <name>hive.downloaded.resources.dir</name>

  8. <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value>

  9. <description>Temporary local directory for added resources in the remote file system.</description>

  10. </property>

  11. <property>

  12. <name>hive.querylog.location</name>

  13. <value>${system:java.io.tmpdir}/${system:user.name}</value>

  14. <description>Location of Hive run time structured log file</description>

  15. </property>

  16. <property>

  17. <name>hive.server2.logging.operation.log.location</name>

  18. <value>${system:java.io.tmpdir}/${system:user.name}/operation_logs</value>

  19. <description>Top level directory where operation logs are stored if logging functionality is enabled</description>

  20. </property>

配置之后如下：

  1. <property>

  2. <name>hive.exec.local.scratchdir</name>

  3. <value>/root/hadoop/apache-hive-1.1.0-bin/iotmp/hive</value>

  4. <description>Local scratch space for Hive jobs</description>

  5. </property>

  6. <property>

  7. <name>hive.downloaded.resources.dir</name>

  8. <value>/root/hadoop/apache-hive-1.1.0-bin/iotmp/${hive.session.id}_resources</value>

  9. <description>Temporary local directory for added resources in the remote file system.</description>

  10. </property>

  11. <property>

  12. <name>hive.querylog.location</name>

  13. <value>/root/hadoop/apache-hive-1.1.0-bin/iotmp/hive</value>

  14. <description>Location of Hive run time structured log file</description>

  15. </property>

  16. <property>

  17. <name>hive.server2.logging.operation.log.location</name>

  18. <value>/root/hadoop/apache-hive-1.1.0-bin/iotmp/hive/operation_logs</value>

  19. <description>Top level directory where operation logs are stored if logging functionality is enabled</description>

  20. </property>

4，执行#hive命令进入Hive CLI时报如下错误： 
在hive1.1版本，对应的Hadoop版本是2.6。启动Hive时有如下错：

  1. # hive

  3. ......

  4. [ERROR] Terminal initialization failed; falling back to unsupported

  5. java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected

  6. at jline.TerminalFactory.create(TerminalFactory.java:101)

  7. at jline.TerminalFactory.get(TerminalFactory.java:158)

  8. at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)

  9. at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)

  10. at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)

  11. at org.apache.hadoop.hive.cli.CliDriver.getConsoleReader(CliDriver.java:773)

  12. at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:715)

  13. at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)

  14. at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)

  15. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  16. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  17. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  18. at java.lang.reflect.Method.invoke(Method.java:606)

  19. at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

  20. at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

原因：因为在hadoop目录下存在老版本jline: 
/root/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar

解决方法：将hive下的新版本jline的JAR包拷贝到hadoop下：

#cp /hive/apache-hive-1.1.0-bin/lib/jline-2.12.jar /root/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/

  1. /root/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib

  2. -rw-r--r--. 1 20000 20000 87325 Nov 14 05:20 jline-0.9.94.jar.bak

  3. -rw-r--r--. 1 root root 213854 Apr 2 08:55 jline-2.12.jar

Hive CLI可以成功启动。

5，在Hive中执行SQL时，报如下错：

  1. Diagnostic Messages for this Task:

  2. Container launch failed for container_1431310046688_0001_02_000013 : org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

  3. This token is expired. current time is 1431311178863 found 1431311136701

  4. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

原因：namenode和datanode的时间不一致，需要保证namenode和datanode的时间一致。

评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

一二三四！

来来来！交个朋友，一起学习

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值