Category: Linux

  • Delayed block allocation failed for inode

    Delayed block allocation failed for inode

    On a CentOS server, nginx web server stopped serving files. On checking logs, I found the following error message

    2021/10/25 19:08:35 [crit] 42762#0: *78977627 pread() "/var/www/html/995.mp4" failed (5: Input/output error) while sending response to client, client: 59.92.71.53, server: 158.58.173.92, request: "GET /995.mp4 HTTP/1.1", host: "domain.tld"

    I tried to copy the file, copy filed with error message “Input/output error”

    [root@mta1 html]# cp /var/www/html/995.mp4 ~/
    cp: error reading '/var/www/html/995.mp4': Input/output error
    [root@mta1 html]# 

    On /var/log/messages, I had the following error message

    Oct 25 19:19:03 mta1 kernel: EXT4-fs error (device dm-0): ext4_wait_block_bitmap:520: comm kworker/u32:2: Cannot read block bitmap - block_group = 7705, block_bitmap = 252182537
    Oct 25 19:19:03 mta1 kernel: EXT4-fs (dm-0): Delayed block allocation failed for inode 39976964 at logical offset 33810 with max blocks 14 with error 5
    Oct 25 19:19:03 mta1 kernel: EXT4-fs (dm-0): This should not happen!! Data will be lost

    The problem is fixed by rebooting the server into rescue mode, then running the fsck command.

    [root@sysresccd ~]# fsck -y /dev/vg/root
    fsck from util-linux 2.35.2
    e2fsck 1.45.6 (20-Mar-2020)
    /dev/mapper/vg-root contains a file system with errors, check forced.
    Pass 1: Checking inodes, blocks, and sizes
    Pass 2: Checking directory structure
    Pass 3: Checking directory connectivity
    Unconnected directory inode 61210647 (/var/cache/dnf/epel-modular-95d9a0c53e492cbd/???)
    Connect to /lost+found? yes
    
    Unconnected directory inode 61210656 (/var/cache/dnf/extras-2770d521ba03e231/???)
    Connect to /lost+found? yes
    
    Unconnected directory inode 61210668 (/var/cache/dnf/epel-6519ee669354a484/???)
    Connect to /lost+found? yes
    
    Pass 4: Checking reference counts
    Inode 39845944 ref count is 1, should be 2.  Fix? yes
    
    ...
    
    Unattached inode 61210637
    Connect to /lost+found? yes
    
    Inode 61210637 ref count is 2, should be 1.  Fix? yes
    
    Inode 61210641 ref count is 3, should be 2.  Fix? yes
    
    Directories count wrong for group #7474 (5, counted=0).
    Fix? yes
    
    Free inodes count wrong (120894192, counted=120894225).
    Fix? yes
    
    Padding at end of inode bitmap is not set. Fix? yes
    
    
    /dev/mapper/vg-root: ***** FILE SYSTEM WAS MODIFIED *****
    /dev/mapper/vg-root: 44271/120938496 files (2.2% non-contiguous), 235802671/483741696 blocks
    [root@sysresccd ~]# 

    See fsck

  • qdbus: not found

    When running a program Ubuntu server with XFCE, I get the error

    ubuntu@ubuntu:~$ ~/Downloads/grab-song-master/check-media-players.sh
    /home/ubuntu/Downloads/grab-song-master/check-media-players.sh: 3: qdbus: not found
    ubuntu@ubuntu:~$ 
    

    The error is fixed by installing qbus

    apt-get install qdbus
    

    On CentOS, you need to install qdbus-qt5 package

    yum install qdbus-qt5
    
  • Set Static IP in RHEL 8 OVH VPS

    Set Static IP in RHEL 8 OVH VPS

    On OVH VPS running AlmaLinux 8, IPv4 IP address gets dropped. I have to connect to the server using KVM feature available in OVH control panel.

    OVH Cloud VPS KVM

    I checked IP of the server with the command

    ip a
    

    It did hot show any IPv4 IP address. I checked the network configuration file

    vi /etc/sysconfig/network-scripts/ifcfg-eth0
    

    That had the following content

    BOOTPROTO=dhcp
    DEFROUTE=yes
    DEVICE=eth0
    DHCLIENT_SET_DEFAULT_ROUTE=yes
    HWADDR=fa:16:3e:6b:a6:1a
    IPV6ADDR=2607:5300:201:3100::952/56
    IPV6INIT=yes
    IPV6_AUTOCONF=no
    IPV6_DEFAULTGW=2607:5300:201:3100::1
    IPV6_FORCE_ACCEPT_RA=no
    MTU=1500
    ONBOOT=yes
    TYPE=Ethernet
    USERCTL=no
    

    To make the IP static, I made the following changes.

    Find

    BOOTPROTO=dhcp
    

    Replace with

    BOOTPROTO=static
    

    Also added following 2 entry

    IPADDR=144.217.13.76
    GATEWAY=144.217.12.1
    

    144.217.13.76 is the IP address of the VPS.

    144.217.12.1 is the gateway used on the VPS, found using “ip route” or “route -n” command.

    [root@vps-eb960963 network-scripts]# route -n
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    0.0.0.0         144.217.12.1    0.0.0.0         UG    0      0        0 eth0
    144.217.12.1    0.0.0.0         255.255.255.255 UH    0      0        0 eth0
    [root@vps-eb960963 network-scripts]#
    

    Create a static route file

    vi /etc/sysconfig/network-scripts/route-eth0
    

    Add

    144.217.12.1 dev eth0
    default via 144.217.12.1 dev eth0
    

    Here 144.217.12.1 is the gateway IP address found in the above step.

    See IP

  • How to Convert CentOS 8 to Rocky Linux

    How to Convert CentOS 8 to Rocky Linux

    You can convert your remote VPS or Dedicated Server running CentOS 8 to Rocky Linux easily by running the following commands.

    Install wget and tmux packages.

    dnf install wget tmux
    

    Next, start a tmux session, so in case you get disconnected from the remote server, you can connect back

    tmux
    

    Now download migration script

    wget https://raw.githubusercontent.com/rocky-linux/rocky-tools/main/migrate2rocky/migrate2rocky.sh
    

    Now verify your system can be converted with command

    bash migrate2rocky.sh -V
    

    If all is good, you can convert CentOS 8 to Rocky Linux with the command

    bash migrate2rocky.sh -r
    
  • List contents of jks keystore file

    List contents of jks keystore file

    To list the content of jks keystore file used by tomcat web server, run command

    keytool -list -keystore FILE.jks 
    

    It will ask Keystore password. Once you enter the password, it will list the contents of the file.

    keytool list certificates

    In the above keystore, there are 5 certificates with names inter, root1, root2, ssl_tomcat2, and tomcat.

    To get detailed information on the certificates, use the command

    keytool -list -v -keystore FILE.jks 
    

    See keytool

  • keytool

    The keytool command is a key and certificate management utility. It enables users to administer their own public/private key pairs and associated certificates for use in self-authentication (where a user authenticates themselves to other users and services) or data integrity and authentication services, by using digital signatures. The keytool command also enables users to cache the public keys (in the form of certificates) of their communicating peers.

    A certificate is a digitally signed statement from one entity (person, company, and so on), which says that the public key (and some other information) of some other entity has a particular value. When data is digitally signed, the signature can be verified to check the data integrity and authenticity. Integrity means that the data hasn’t been modified or tampered with, and authenticity means that the data comes from the individual who claims to have created and signed it.

    The keytool command also enables users to administer secret keys and passphrases used in symmetric encryption and decryption (Data Encryption Standard). It can also display other security-related information.

    The keytool command stores the keys and certificates in a keystone.

    List contents of jks keystore file
    Extracting SSL certificate from the Java Keystore (JKS)

  • How to view the contents of tar.gz file

    How to view the contents of tar.gz file

    To view the contents of tar.gz file, run

    tar tf FILE.tar.gz
    

    tar view files

    -f, --file=ARCHIVE         use archive file or device ARCHIVE
    -t, --list                 list the contents of an archive
    
  • How to install zabbix agent on Ubuntu

    How to install zabbix agent on Ubuntu

    To install the Zabbix agent on Ubuntu, run the command

    apt install zabbix-agent
    

    To autostart, Zabbix agent on boot, enable it with

    systemctl enable zabbix-agent
    

    To manage Zabbix, use commands

    systemctl stop zabbix-agent
    systemctl start zabbix-agent
    systemctl restart zabbix-agent
    

    To see Zabbix listening port

    netstat -lntp | grep zabbix
    

    Zabbix agent port

    Edit file

    vi /etc/zabbix/zabbix_agentd.conf 
    

    Find

    Server=127.0.0.1
    

    Replace with

    Server=127.0.0.1,IP_OF_ZABBIX_SERVER_HERE
    

    zabbix agent Server config

    Restart zabbix agent

    systemctl restart zabbix-agent
    

    After the Zabbix agent is installed, you need to add the sevrer in your Zabbix installation.

  • How to Migrate CentOS 8 to AlmaLinux

    How to Migrate CentOS 8 to AlmaLinux

    CentOS 8 is reaching its End of Life in December 2021. If you are using CentOS 8, you may need to consider migrating to other operating systems.

    Other alternatives for CentOS 8 are AlmaLinux, Rocky Linux, and Oracle Linux. All of these are free and are based on RHEL 8.

    AlmaLinux is released a migration script

    https://github.com/AlmaLinux/almalinux-deploy

    This script allows you to convert CentOS 8 to AlmaLinux.

    WARNING: Make sure you take a full system backup before migrating.

    Before migrating, make sure your system is up to date.

    dnf update
    

    If you get error, Error: Failed to download metadata for repo ‘appstream’, run

    sed -i -e "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-*
    sed -i -e "s|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g" /etc/yum.repos.d/CentOS-*
    dnf update
    

    Reboot the server

    reboot
    

    Once the server is back online, we can run the AlmaLinux migration script to start the migration.

    curl -O https://raw.githubusercontent.com/AlmaLinux/almalinux-deploy/master/almalinux-deploy.sh
    bash almalinux-deploy.sh
    

    After the migration is complete, run

    dnf distro-sync -y 
    

    Now you need to reboot the server

    reboot