Tag: hdd

When I try to mount a partition, I get the following error

[root@sysresccd ~]# mount /dev/nvme0n1p3 /mnt
mount: /mnt: unknown filesystem type 'LVM2_member'.
[root@sysresccd ~]# 

This is because the disk is LVM. First of all identify the name of the volume group and logical volume.

vgs
lvs

Example

[root@sysresccd ~]# vgs
  VG #PV #LV #SN Attr   VSize  VFree
  vg   2   3   0 wz--n- <1.82t    0 
[root@sysresccd ~]# lvs
  LV   VG Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root vg -wi-a-----   1.80t                                                    
  swap vg -wi-a----- <15.69g                                                    
  tmp  vg -wi-a-----   1.00g                                                    
[root@sysresccd ~]#

From the above result, we found the server has 3 logical volumes. The biggest volume is root. Other volumes swap and tmp we can ignore. So the device name is

/dev/vg/root

Next run

modprobe dm-mod
lvscan
vgchange -ay

Example

[root@sysresccd ~]# lvscan
  ACTIVE            '/dev/vg/swap' [<15.69 GiB] inherit
  ACTIVE            '/dev/vg/tmp' [1.00 GiB] inherit
  ACTIVE            '/dev/vg/root' [1.80 TiB] inherit
[root@sysresccd ~]# vgchange -ay
  3 logical volume(s) in volume group "vg" now active
[root@sysresccd ~]#

Now you should be able to mount with disk using command

mount /dev/vg/root /mnt/

On a CentOS server, nginx web server stopped serving files. On checking logs, I found the following error message

2021/10/25 19:08:35 [crit] 42762#0: *78977627 pread() "/var/www/html/995.mp4" failed (5: Input/output error) while sending response to client, client: 59.92.71.53, server: 158.58.173.92, request: "GET /995.mp4 HTTP/1.1", host: "domain.tld"

I tried to copy the file, copy filed with error message “Input/output error”

[root@mta1 html]# cp /var/www/html/995.mp4 ~/
cp: error reading '/var/www/html/995.mp4': Input/output error
[root@mta1 html]# 

On /var/log/messages, I had the following error message

Oct 25 19:19:03 mta1 kernel: EXT4-fs error (device dm-0): ext4_wait_block_bitmap:520: comm kworker/u32:2: Cannot read block bitmap - block_group = 7705, block_bitmap = 252182537
Oct 25 19:19:03 mta1 kernel: EXT4-fs (dm-0): Delayed block allocation failed for inode 39976964 at logical offset 33810 with max blocks 14 with error 5
Oct 25 19:19:03 mta1 kernel: EXT4-fs (dm-0): This should not happen!! Data will be lost

The problem is fixed by rebooting the server into rescue mode, then running the fsck command.

[root@sysresccd ~]# fsck -y /dev/vg/root
fsck from util-linux 2.35.2
e2fsck 1.45.6 (20-Mar-2020)
/dev/mapper/vg-root contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Unconnected directory inode 61210647 (/var/cache/dnf/epel-modular-95d9a0c53e492cbd/???)
Connect to /lost+found? yes

Unconnected directory inode 61210656 (/var/cache/dnf/extras-2770d521ba03e231/???)
Connect to /lost+found? yes

Unconnected directory inode 61210668 (/var/cache/dnf/epel-6519ee669354a484/???)
Connect to /lost+found? yes

Pass 4: Checking reference counts
Inode 39845944 ref count is 1, should be 2.  Fix? yes

...

Unattached inode 61210637
Connect to /lost+found? yes

Inode 61210637 ref count is 2, should be 1.  Fix? yes

Inode 61210641 ref count is 3, should be 2.  Fix? yes

Directories count wrong for group #7474 (5, counted=0).
Fix? yes

Free inodes count wrong (120894192, counted=120894225).
Fix? yes

Padding at end of inode bitmap is not set. Fix? yes


/dev/mapper/vg-root: ***** FILE SYSTEM WAS MODIFIED *****
/dev/mapper/vg-root: 44271/120938496 files (2.2% non-contiguous), 235802671/483741696 blocks
[root@sysresccd ~]# 

See fsck