22

hive on spark报timed out waiting for connection from the Remote Spark Driver解决...

 2 years ago
source link: https://blog.51cto.com/wang/5088823
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

hive on spark报timed out waiting for connection from the Remote Spark Driver解决方案

原创

江南独孤客 2022-03-10 08:24:46 博主文章分类：大数据篇 ©著作权

文章标签 spark hive 解决方案 文章分类 Spark 大数据 阅读数269

[INFO] 2022-03-09 00:55:34.227 - [taskAppId=TASK-122-55596-532256]:[138] - -> FAILED: Execution Error, return code 30041 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Failed to create Spark client for Spark session 1c5c0ae6-d56d-47b3-ae5a-d85fd941b14e_0: java.util.concurrent.TimeoutException: Client '1c5c0ae6-d56d-47b3-ae5a-d85fd941b14e_0' timed out waiting for connection from the Remote Spark Driver

​Client '1c5c0ae6-d56d-47b3-ae5a-d85fd941b14e_0' timed out waiting for connection from the Remote Spark Driver

​报错的原因是连接5分钟超时​

解决方案：

hive on spark报timed out waiting for connection from the Remote Spark Driver解决方案_spark

hive on spark报timed out waiting for connection from the Remote Spark Driver解决方案_解决方案_02

调整的参数：

hive.spark.client.server.connect.timeout=300000

hive.spark.client.connect.timeout=30000

hive.spark.client.future.timeout=1200

调整后调整中的任务无报错。

  • 打赏
  • 赞
  • 收藏
  • 评论
  • 分享
  • 举报

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK