Category: Linux

  • Install Gitlab on Debian/Ubuntu

    Install requirements

    echo "postfix postfix/main_mailer_type string 'Internet Site'" | debconf-set-selections
echo "postfix postfix/mailname string localhost" | debconf-set-selections
curl -Ls https://packages.gitlab.com/gpg.key | apt-key add -
apt-get update
apt-get --force-yes -y upgrade
apt-get clean

    Setup repo/dependency

    wget https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
bash script.deb.sh

    Install gitlab

    apt-get install -y gitlab-ce

  • Install SpeedTest cli on Ubuntu

    speedtest.net is a popular web site for testing braodband speed.

    To install speedtest cli on Ubuntu, run

    apt install -y speedtest-cli

    This will be very helpful for testing speed of servers and remote systems using SSH.

    To test speed, run

    speedtest

    Example

    speedtest-cli

    To get list of available speedtest servers, run

    speedtest --list

    To test with a specific server, run

    speedtest --server SERVER_ID

  • Install vsftpd on CentOS

    To install vsftpd FTP server in CentOS, run

    yum install -y vsftpd

    Enable vsftpd to start on boot

    systemctl enable vsftpd

    Edit configuration file

    vi /etc/vsftpd/vsftpd.conf

    You need to update/add following configuration options

    chroot_local_user=YES
allow_writeable_chroot=YES
local_umask=002
file_open_mode=0755
force_dot_files=YES

    Restart vsftpd

    systemctl restart vsftpd

    Allow System Accounts to login

    local_enable=YES
write_enable=YES

    Find

    anonymous_enable=YES

    Replace With

    anonymous_enable=NO

    Find

    #chroot_local_user=YES

    Replace with

    chroot_local_user=YES

    Add

    allow_writeable_chroot=YES
local_umask=002
file_open_mode=0755

    See vsftpd

  • Test network performance with iperf

    iperf can be used to test network performance between two servers. You can install iperf on Ubuntu/Debian with command

    apt install -y iperf

    To run test, you need to run iperf in server mode in one of the server using command

    iperf -s

    On other server, run

    iperf -c IP_OF_REMOTE_SERVER

    Install iperf3 on ArchLinux

  • Benchmark

    Load Testing with Siege
    geekbench

  • Load Testing with Siege

    siege is an open source program to load test and bechmark web site.

    To install siege on Ubuntu/debian, run

    apt -y install siege

    To generate 250 concurrent users, run

    siege -c 250 http://mydomain.com/

    To simiulate 100 users with 10 second delay betwen requests

    siege -d 10 -c 100 http://mydomain.com/

    To simulate real work load, you may need to send traffic to multiple URLs on your web site. You can do this by creating a file with all urls of your web site. Now run

    siege -d 10 -c 100 -i -f all-urls.txt

    See load

  • Convert Video info MP4 using ffmpeg

    Convert Video For Streaming

    ffmpeg -y -i FILE_NAME.wmv -c:v libx264 -c:a aac -coder 1 -b:v 500k -threads 0 -b:a 96k FILE_NAME.mp4

    if you want to scale the video to a specific size, use -vf scale option, for example

    ffmpeg -y -i FILE_NAME.wmv -vf scale=720:480 -c:v libx264 -c:a aac -coder 1 -b:v 500k -threads 0 -b:a 96k FILE_NAME.mp4

    Inject metadata

    qt-faststart FILE_NAME.mp4 FILE_NAME-meta.mp4

    Convert Video to MP4

    This is verified method, and works good, i converted many videos, no quality lose

    ffmpeg  -i FILE_NAME.INPUT_EXTN -crf 25.0 -c:v libx264 -c:a libfaac -ar 48000 -b:a 160k -coder 1 -threads 0 FILE_NAME_nm.mp4

    If you get error related to libfaac, try changing it with aac.

    To cut a video during transcoding

    time ffmpeg -ss 00:34:00 -t 00:02:00 -i FILE_NAME.INPUT_EXTN -crf 25.0 -c:v libx264 -c:a libfaac -ar 48000 -b:a 160k -coder 1 -threads 0 FILE_NAME_nm.mp4

    Other Methods, Need to experiment

    To convert video to H.264 (mp4) format

    ffmpeg -i 'input.mpg' -acodec libfaac -ab 96k -vcodec libx264 -vpre hq -crf 22 -threads 0 output.mp4
    ffmpeg -i 'input.mpg' -vcodec libx264 -vpre hq -b 1M -bt 1M -threads 0 outfile.mp4
    ffmpeg -i test.avi -acodec libfaac -ab 128k -ar 44100 -s 704x400 -r 20 -vcodec libx264 -b 256000 -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -me_method umh -subq 5 -trellis 1 -refs 2 -bf 1 -coder 1 -me_range 16 -g 300 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 256000 -maxrate 4M -bufsize 4M -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 21 test.mp4
    ffmpeg -y -i source.avi -vcodec libx264 -b BITRATE -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -flags2 +mixed_refs -me umh -subq 5 -trellis 1 -refs 5 -bf 3 -b_strategy 1 -coder 1 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt "$vbitrate" -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 41 -acodec libfaac -ac 2 -ar 48000 -ab "$abitrate" -f mov out.mov

    Using ffmpeg and x264

    Example two-pass encoding script:

    #!/bin/bash
# Two-Pass x264 Encoding Script
# Usage: ./264encode.sh input output.mp4

# Pass 1
ffmpeg -y -i $1 -pass 1 -b 512k -bt 512k -vcodec libx264 -an -threads 0 -coder 1 -flags +loop -cmp +chroma -partitions -parti8x8-parti4x4-partp8x8-partp4x4-partb8x8 -me_method dia -subq 1 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -bf 16 -refs 1 -directpred 1 -bidir_refine 0 -trellis 0 -flags2 -bpyramid-wpred-mixed_refs-dct8x8+fastpskip -f mp4 /dev/null

# Pass 2
ffmpeg -y -i $1 -pass 2 -b 512k -bt 512k -vcodec libx264 -acodec libfaac -ab 128k -ac 2 -threads 0 -coder 1 -flags +loop -cmp +chroma -partitions +parti8x8+parti4x4+partp8x8+partb8x8 -me_method umh -subq 8 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -b_strategy 1 -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -bf 16 -refs 4 -directpred 3 -trellis 1 -flags2 +bpyramid+wpred+mixed_refs+dct8x8+fastpskip -f mp4 $2

    To use

    sh 264encode.sh input.avi output.mp4

    See ffmpeg

  • Create video thumbnail using ffmpeg/mplayer

    Create thumbnail with ffmpeg

    To create thumbnail from video using ffmpeg, use

    ffmpeg -y -ss HH:MM:SS -i VIDEO.AVI -f image2 -vframes 1 1.jpg

    Here

    HH:MM:SS = replace with time where you want to take screen shot.

    1.jpg = is the tumbnail image generated.

    Create thumbnail with mplayer

    METHOD 1

    mkdir out
mplayer VIDEO.AVI -ss 0:8:0 -nosound -vo jpeg:outdir=out -frames 2

    -ss = you can specify time in seconds or in hh:mm:ss format.

    -frames 2 = create 2 thumbnails with name 00000001.jpg and 00000002.jpg

    -ss 0:8:0 = thumbnail from 8th minutes

    METHOD 2

    mencoder -ss 00:25:00 -endpos 0.001 -ovc copy -nosound VIDEO.AVI -o tmpfile1
mplayer -nosound -vo jpeg tmpfile1

    Create thumbnail with avconv

    avconv -ss 00:01:00 -i VIDEO_FILE.mp4 -f image2 -frames:v 1 1.jpeg

    This will create thumbnail 1.jpeg from video.

    -ss 00:01:00 – Time from where image is taken.

    See ffmpeg

  • Scale a video using ffmpeg

    To Scape a Video using ffmpeg, run

    ffmpeg -i FILE_NAME.avi -vf scale=1280:720 FILE_NAME_OUT.avi

    See ffmpeg

  • Cut a Video using ffmpeg

    To cut a video using ffmpeg, use

    ffmpeg -ss 00:17:00 -i IN_FILE.avi -t DURATION_IN_SECONDS -acodec copy -vcodec copy -async 1 OUT_FILE.avi

    To cut a video and encoding it

    ffmpeg -ss 00:17:00 -i IN_FILE.avi -t DURATION_IN_SECONDS -crf 25.0 -c:v libx264 -c:a aac -ar 48000 -b:a 160k -coder 1 -threads 0 OUT_FILE.mp4

    DURATION_IN_SECONDS specify duration of new clip. If you specify 60, it will cut 60 seconds of video start from time specified in -ss. In this cause, from 00:17:00 to 00:18:00.

    For more info, see “Cutting small sections” part of ffmpeg documentation.

    https://trac.ffmpeg.org/wiki/Seeking

    ffmpeg -ss 00:34:00 -t 00:02:00 -i IN_FILE.wmv -crf 25.0 -c:v libx264 -c:a libfaac -ar 48000 -b:a 160k -coder 1 -threads 0 OUT_FILE.mp4

    In this example

    -ss 0:34:00 -t 0:02:00

    Means cut video from 00:34:00 to 00:36:00

    If you want to cut a video from time 01:00:00 to 01:20:00, use

    -ss 01:00:00 -t 00:20:00

    10-Apr-2013 This command works great, i tested on many videos.

    Cut video with avconv

    avconv -y -ss 00:04:00 -i BootstrapWebDesign-03.mov -t 00:00:30 -crf 25.0 -c:v libx264 -c:a libvo_aacenc -ar 48000 -b:a 160k -coder 1 -threads 0 1.mp4

    Cut video with mencoder

    mencoder -ss 01:00:00 -endpos 00:00:30 -ovc copy -oac copy 1.avi -o 2.avi

    -ss = start time in hh:mm:ss format

    -endpos = Duration in hh:mm:ss format

    Example

    Following with create a video “out.avi” from MSFT.avi from 00:24:00 to 00:25:00

    mencoder -ss 00:24:00 -endpos 00:01:00 -ovc copy -oac copy MSFT.avi -o out.avi

    21 April 2011 verified and working

    See ffmpeg

  • lftp Fatal error: Host key verification failed

    When i connect to a remote SFTP server using lftp, i get following error

    root@ok:~# lftp sftp://okuser@ws.nii.net:22
Password:
lftp okuser@ws.nii.net:~> set ftp:ssl-allow false
lftp okuser@ws.nii.net:~> mirror --verbose --use-pget-n=8 -c /home/okuser/www.site.com/docs /root/site-backup/
mirror: Fatal error: Host key verification failed.   
mkdir: Access failed: /root/site-backup/docs: File exists
lftp: MirrorJob.cc:242: void MirrorJob::JobFinished(Job*): Assertion `transfer_count>0' failed.
Aborted
root@ok:~#

    The error is “Fatal error: Host key verification failed.”

    To fix this, just SSH into this server with command

    ssh -p 22 okuser@ws.nii.net

    See lftp

  • SELinux

    To disable SELinux, edit

    sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

    Now you need to reboot. If you don’t want to reboot, disable it for current session with

    setenforce 0

    If you use SELinux, you may need to configure it for each applications.

    For web server

    setsebool httpd_can_network_connect true
setsebool -P httpd_can_network_connect_db 1
setsebool -P memcached_connect_any 1

    CURL ERROR 7 could not establish a secure connection to WordPress.org
    SELinux allow non default SSH port