Delayed block allocation failed for inode
On a CentOS server, nginx web server stopped serving files. On checking logs, I found the following error message
|
1 |
2021/10/25 19:08:35 [crit] 42762#0: *78977627 pread() "/var/www/html/995.mp4" failed (5: Input/output error) while sending response to client, client: 59.92.71.53, server: 158.58.173.92, request: "GET /995.mp4 HTTP/1.1", host: "domain.tld" |
I tried to copy the file, copy filed with error message “Input/output error”
|
1 2 3 |
[root@mta1 html]# cp /var/www/html/995.mp4 ~/ cp: error reading '/var/www/html/995.mp4': Input/output error [root@mta1 html]# |
On /var/log/messages, I had the following error message
|
1 2 3 |
Oct 25 19:19:03 mta1 kernel: EXT4-fs error (device dm-0): ext4_wait_block_bitmap:520: comm kworker/u32:2: Cannot read block bitmap - block_group = 7705, block_bitmap = 252182537 Oct 25 19:19:03 mta1 kernel: EXT4-fs (dm-0): Delayed block allocation failed for inode 39976964 at logical offset 33810 with max blocks 14 with error 5 Oct 25 19:19:03 mta1 kernel: EXT4-fs (dm-0): This should not happen!! Data will be lost |
The problem is fixed by rebooting the server into rescue mode, then running the fsck command.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
[root@sysresccd ~]# fsck -y /dev/vg/root fsck from util-linux 2.35.2 e2fsck 1.45.6 (20-Mar-2020) /dev/mapper/vg-root contains a file system with errors, check forced. Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Unconnected directory inode 61210647 (/var/cache/dnf/epel-modular-95d9a0c53e492cbd/???) Connect to /lost+found? yes Unconnected directory inode 61210656 (/var/cache/dnf/extras-2770d521ba03e231/???) Connect to /lost+found? yes Unconnected directory inode 61210668 (/var/cache/dnf/epel-6519ee669354a484/???) Connect to /lost+found? yes Pass 4: Checking reference counts Inode 39845944 ref count is 1, should be 2. Fix? yes ... Unattached inode 61210637 Connect to /lost+found? yes Inode 61210637 ref count is 2, should be 1. Fix? yes Inode 61210641 ref count is 3, should be 2. Fix? yes Directories count wrong for group #7474 (5, counted=0). Fix? yes Free inodes count wrong (120894192, counted=120894225). Fix? yes Padding at end of inode bitmap is not set. Fix? yes /dev/mapper/vg-root: ***** FILE SYSTEM WAS MODIFIED ***** /dev/mapper/vg-root: 44271/120938496 files (2.2% non-contiguous), 235802671/483741696 blocks [root@sysresccd ~]# |