文章目录
Kylin 启动异常
错误日志如下:
2021-10-18 09:34:23,210 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:101 : 71583@gapuat01 trying to lock /job_engine/global_job_engine_lock
2021-10-18 09:34:23,236 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:167 : 71583@gapuat01 see /job_engine/global_job_engine_lock is already locked
2021-10-18 09:34:23,243 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:117 : 71583@gapuat01 failed to acquire lock at /job_engine/global_job_engine_lock, which is held by 13249@gapuat02
2021-10-18 09:34:23,244 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:182 : 71583@gapuat01 will wait for lock path /job_engine/global_job_engine_lock
2021-10-18 09:34:32,091 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:101 : 71583@gapuat01 trying to lock /job_engine/global_job_engine_lock
2021-10-18 09:34:32,096 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:167 : 71583@gapuat01 see /job_engine/global_job_engine_lock is already locked
2021-10-18 09:34:32,098 DEBUG [localhost-startStop-1] zookeeper.ZookeeperDistributedLock:117 : 71583@gapuat01 failed to acquire lock at /job_engine/global_job_engine_lock, which is held by 13249@gapuat02
2021-10-18 09:34:32,100 WARN [localhost-startStop-1] support.XmlWebApplicationContext:550 : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'migrationService': Unsatisfied dependency expressed through field 'cubeService'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cubeMgmtService': Unsatisfied dependency expressed through field 'jobService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jobService' defined in URL [jar:file:/data/software/apache-kylin-3.1.1-bin/tomcat/webapps/kylin/WEB-INF/lib/kylin-server-base-3.1.1.jar!/org/apache/kylin/rest/service/JobService.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: Cannot start job scheduler due to lack of job lock
2021-10-18 09:34:32,102 WARN [metrics-blocking-reservoir-scheduler-0] impl.BlockingReservoir:204 : Interrupted during running
2021-10-18 09:34:32,102 INFO [metrics-blocking-reservoir-scheduler-0] impl.BlockingReservoir:108 : Will report 0 metrics records
2021-10-18 09:34:32,102 INFO [metrics-blocking-reservoir-scheduler-0] impl.BlockingReservoir:197 : Reporter finishes reporting metrics.
2021-10-18 09:34:32,110 ERROR [localhost-startStop-1] context.ContextLoader:350 : Context initialization failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'migrationService': Unsatisfied dependency expressed through field 'cubeService'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cubeMgmtService': Unsatisfied dependency expressed through field 'jobService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jobService' defined in URL [jar:file:/data/software/apache-kylin-3.1.1-bin/tomcat/webapps/kylin/WEB-INF/lib/kylin-server-base-3.1.1.jar!/org/apache/kylin/rest/service/JobService.class]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: Cannot start job scheduler due to lack of job lock
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1257)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:757)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5197)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5720)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1016)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:992)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2020)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
问题原因
kylin只能有一个节点配置为kylin.server.mode=all,配置为all的节点调用zookeeper分布式锁异常如下:
failed to acquire lock at /job_engine/global_job_engine_lock, which is held by 13249@gapuat02
说明gapuat2这一台已经启动了一个kylin.server.mode=all的节点,导致当前节点启动报错
解决
保证集群里只有一台kylin server配置为:kylin.server.mode=all