hdfs datanode启动失败 org.apache.hadoop.util.DiskChecker$DiskErrorException: Directory is not readable

但行益事莫问前程

已于 2022-06-24 14:51:33 修改

阅读量7.3w

点赞数 2

分类专栏： hdfs 异常处理文章标签： hdfs

于 2022-06-24 14:51:20 首次发布

本文链接：https://blog.csdn.net/javahelpyou/article/details/125444988

版权

异常处理同时被 2 个专栏收录

24 篇文章

订阅专栏

hdfs

5 篇文章

订阅专栏

博客讲述了在运行Hadoop集群时遇到DataNode启动失败的问题，日志显示磁盘不可读。通过检查发现挂载的磁盘不存在，且配置的容错磁盘数量为0。解决方案包括增加容错磁盘数量或调整配置文件中不正确的磁盘路径。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

问题：

datanode启动失败,查看datanode日志

org.apache.hadoop.util.DiskChecker$DiskErrorException: Directory is not readable: /var/lib/docker/overlay2/2f1ec1e560a2d5d6d51fa8bb23bc3d0fe7dbcd07e6460273da8f70b7bc472af0/merged/hadoop/hdfs/data
	at org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskChecker.java:119)
	at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:99)
	at org.apache.hadoop.hdfs.server.datanode.StorageLocation.check(StorageLocation.java:128)
	at org.apache.hadoop.hdfs.server.datanode.StorageLocation.check(StorageLocation.java:44)
	at org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker$1.call(ThrottledAsyncChecker.java:127)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2022-06-24 14:23:47,782 WARN  checker.StorageLocationChecker (StorageLocationChecker.java:check(208)) - Exception checking StorageLocation [DISK]file:/var/lib/docker/overlay2/de2f6b02326e3ebe8f209474868b6c0d55bb0c5fe3c798f9aa5bd5048ed38dbc/merged/hadoop/hdfs/data/
org.apache.hadoop.util.DiskChecker$DiskErrorException: Directory is not readable: /var/lib/docker/overlay2/de2f6b02326e3ebe8f209474868b6c0d55bb0c5fe3c798f9aa5bd5048ed38dbc/merged/hadoop/hdfs/data
	at org.apache.hadoop.util.DiskChecker.checkAccessByFileMethods(DiskChecker.java:119)
	at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:99)
	at org.apache.hadoop.hdfs.server.datanode.StorageLocation.check(StorageLocation.java:128)
	at org.apache.hadoop.hdfs.server.datanode.StorageLocation.check(StorageLocation.java:44)
	at org.apache.hadoop.hdfs.server.datanode.checker.ThrottledAsyncChecker$1.call(ThrottledAsyncChecker.java:127)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2022-06-24 14:23:47,782 ERROR datanode.DataNode (DataNode.java:secureMain(2691)) - Exception in secureMain
org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 4, volumes configured: 6, volumes failed: 2, volume failures tolerated: 0
	at org.apache.hadoop.hdfs.server.datanode.checker.StorageLocationChecker.check(StorageLocationChecker.java:216)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2583)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2492)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2539)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2684)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2708)

原因：
查看服务器文件系统和挂载发现挂载的磁盘不存在

df -h

查看hdfs数据目录 hdfs-site.xml的配置dfs.datanode.data.dir数据目录和dfs.datanode.failed.volumes.tolerated磁盘容错

<property>
    <name>dfs.datanode.data.dir</name>
    <value> file://目录</value>
</property>
<property>
        <name>dfs.datanode.failed.volumes.tolerated</name>
        <value>0</value>
</property>

解决：

因为没磁盘换，可将容错数量提高

<property>
        <name>dfs.datanode.failed.volumes.tolerated</name>
        <value>2</value>
</property>

或者将磁盘目录从配置项中删除