Spark源码之Executor线程池

docsz

于 2020-03-16 00:13:23 发布

阅读量1k

点赞数

分类专栏： Spark 文章标签： spark

本文链接：https://blog.csdn.net/docsz/article/details/104889468

版权

Spark 专栏收录该内容

7 篇文章

订阅专栏

1：Executor线程池

  // Start worker thread pool
  private val threadPool = {
    val threadFactory = new ThreadFactoryBuilder()
      .setDaemon(true)
      .setNameFormat("Executor task launch worker-%d")
      .setThreadFactory(new ThreadFactory {
        override def newThread(r: Runnable): Thread =
          // Use UninterruptibleThread to run tasks so that we can allow running codes without being
          // interrupted by `Thread.interrupt()`. Some issues, such as KAFKA-1894, HADOOP-10622,
          // will hang forever if some methods are interrupted.
          new UninterruptibleThread(r, "unused") // thread name will be set by ThreadFactoryBuilder
      })
      .build()
    Executors.newCachedThreadPool(threadFactory).asInstanceOf[ThreadPoolExecutor]
  }