Category: Cloud

  • Amazon Elastic Container Registry

    Amazon Elastic Container Registry

    Amazon Elastic Container Registry is used to store docker images in Amazon AWS cloud.

    To create a repository using awscli command line tool, run

    aws ecr create-repository --repository-name sok-repository --region ap-southeast-1
    

    In Amazon AWS console, you can see the newly created repository by going to “Elastic Container Registry” page in the region where you created the repository.

    https://ap-southeast-1.console.aws.amazon.com/ecr/repositories?region=ap-southeast-1

    amazon docker registry (ECR)

    To see the details from command line, run

    aws ecr describe-repositories --region ap-southeast-1
    

    Amazon AWS ECR awscli

    From the command, you will see repositoryUri, this is used to push your docker images.

    I have following docker images

    [root@instance-20210426-0136 ~]# docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    sevrerok/okapache   1.2                 c3832b03b548        2 hours ago         214MB
    sevrerok/okapache   1.1                 d1a86f0eb69a        2 hours ago         214MB
    ubuntu              20.04               7e0aa2d69a15        2 days ago          72.7MB
    sevrerok/okapache   1.0                 7e0aa2d69a15        2 days ago          72.7MB
    [root@instance-20210426-0136 ~]# 
    

    I need to push the image sevrerok/okapache:1.2 to Amazon ECR, for this first tag the docker image with repository name.

    docker tag sevrerok/okapache:1.2 497940214440.dkr.ecr.ap-southeast-1.amazonaws.com/sok-repository
    

    Now docker images will show

    [root@instance-20210426-0136 ~]# docker images
    REPOSITORY                                                         TAG                 IMAGE ID            CREATED             SIZE
    sevrerok/okapache                                                  1.2                 c3832b03b548        2 hours ago         214MB
    497940214440.dkr.ecr.ap-southeast-1.amazonaws.com/sok-repository   latest              c3832b03b548        2 hours ago         214MB
    sevrerok/okapache                                                  1.1                 d1a86f0eb69a        2 hours ago         214MB
    ubuntu                                                             20.04               7e0aa2d69a15        2 days ago          72.7MB
    sevrerok/okapache                                                  1.0                 7e0aa2d69a15        2 days ago          72.7MB
    [root@instance-20210426-0136 ~]# 
    

    Login to ECR

    aws ecr get-login
    

    It will display command you need to login to ECR using docker. Run the command to login to ECR.

    To push the docker image to ECR, run

    docker push 497940214440.dkr.ecr.ap-southeast-1.amazonaws.com/sok-repository
    

    ECR push

    Now the image is pushed to ECR, you will be able to see it using AWS console or awscli

    [root@instance-20210426-0136 ~]# aws ecr list-images --repository-name sok-repository
    {
        "imageIds": [
            {
                "imageTag": "latest", 
                "imageDigest": "sha256:3cb5b8ef33bf913018f28dc3adf93b96c66667b517fe800a99bd0defd9dc6130"
            }
        ]
    }
    [root@instance-20210426-0136 ~]# 
    

    To delete the ECR repo, use following command

    aws ecr delete-repository --repository-name sok-repository --region ap-southeast-1 --force
    

    See AWS

  • AWS CLI list all IAM users

    To list all IAM users in Amazon AWS from awscli, run

    aws iam list-users
    

    Example

    aws iam list-users

    See aws cli

  • How to Migrate Amazon AWS RDS Database to another AWS account

    This video shows how to transfer an Amazon RDS database (MySQL) from one AWS account to another.

  • Amazon EC2 disk resize No space left on device

    On an Amazon EC2 instamce disk usage was full.

    root@ip-172-31-46-249:/# df -h
    Filesystem      Size  Used Avail Use% Mounted on
    udev            3.9G     0  3.9G   0% /dev
    tmpfs           791M  8.9M  782M   2% /run
    /dev/nvme0n1p1  9.7G  9.6G   65M 100% /
    tmpfs           3.9G     0  3.9G   0% /dev/shm
    tmpfs           5.0M     0  5.0M   0% /run/lock
    tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
    /dev/loop0       97M   97M     0 100% /snap/core/9665
    /dev/loop1       97M   97M     0 100% /snap/core/9436
    /dev/loop2       18M   18M     0 100% /snap/amazon-ssm-agent/1566
    /dev/loop3       29M   29M     0 100% /snap/amazon-ssm-agent/2012
    tmpfs           791M     0  791M   0% /run/user/998
    tmpfs           791M     0  791M   0% /run/user/1000
    root@ip-172-31-46-249:/#
    

    I increased disk size on Amazone AWS console. But disk did not get changed in EC2 instance.

    root@ip-172-31-46-249:~# parted -l
    Model: NVMe Device (nvme)
    Disk /dev/nvme0n1: 21.5GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Disk Flags: 
    
    Number  Start   End     Size    Type     File system  Flags
     1      1049kB  10.7GB  10.7GB  primary  ext4         boot
    
    
    root@ip-172-31-46-249:~# 
    

    The partition still shows 10 GB. When i try growpart, i get error

    root@ip-172-31-46-249:/# growpart /dev/nvme0n1 1
    mkdir: cannot create directory ‘/tmp/growpart.1889’: No space left on device
    FAILED: failed to make temp dir
    root@ip-172-31-46-249:/# 
    

    This is because disk is full. I try delete some unwanted files. But was not able to free up much disk space. To fix the error, i mounted /tmp in memory with commands.

    mkdir /dev/shm/tmp
    chmod 1777 /dev/shm/tmp
    mount --bind /dev/shm/tmp /tmp
    

    This ec2 instance had lot of free RAM, so it should handle /tmp folder with out any issue. Now growpart worked.

    root@ip-172-31-46-249:/# growpart /dev/nvme0n1 1
    CHANGED: partition=1 start=2048 old: size=20969439 end=20971487 new: size=41940959,end=41943007
    root@ip-172-31-46-249:/# 
    

    parted -l shows the partition using all available disk space

    root@ip-172-31-46-249:/# parted -l
    Model: NVMe Device (nvme)
    Disk /dev/nvme0n1: 21.5GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Disk Flags: 
    
    Number  Start   End     Size    Type     File system  Flags
     1      1049kB  21.5GB  21.5GB  primary  ext4         boot
    
    
    root@ip-172-31-46-249:/# 
    

    still df -h won’t show increased disk space, this is because you need to increase filesystem size.

    resize2fs /dev/nvme0n1p1
    

    See Amazon EC2

  • Mount Amazon s3 on Linux

    To mount Amazon S3 bucket as a filesystem on linux, you can use s3fs.

    https://github.com/s3fs-fuse/s3fs-fuse

    To install s3fs on Ubuntu/Debian, run

    apt install s3fs
    

    To mount an s3 buclet, create a password file with your AWS secret key and

    echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs
    chmod 600 ${HOME}/.passwd-s3fs
    

    Then run

    s3fs sokmount /home/boby/ -o passwd_file=${HOME}/.passwd-s3fs -o dbglevel=info -f -o curldbg -o endpoint=ap-southeast-1 -o url="https://s3-ap-southeast-1.amazonaws.com"
    

    In above command replace

    sokmount = bucket name
    /home/boby/ = your mount point
    ap-southeast-1 = AWS region where bucket is created.

    See Amazon S3

  • Amazon ECS

    Amazon ECS is a container service from Amazon AWS Cloud. This allow you to run Docker containers in AWS cloud. It is like Kubernetes.

    Here are some good resources to get started with Amazon ECS

    https://github.com/nathanpeck/awesome-ecs

    https://ecsworkshop.com

    See AWS

  • AlibabaCloud DirectMail

    AlibabaCloud DirectMail

    Alibaba Cloud DirectMail

    AlibabaCloud DirectMail is transactional email service from AlibabaCloud. It is like Amazon SES, MailGun, SendGrid etc…

    They have 3 differnt mail server hostname depending on region. Unlike other SMTP providers, they allow email using port 80.

    SMTP service address (China East 1): smtpdm.aliyun.com
    
    SMTP service address (Singapore): smtpdm-ap-southeast-1.aliyun.com
    
    SMTP service address (Sydney): smtpdm-ap-southeast-2.aliyun.com
    
    SMTP port: 25, 80, 465 (SSL encrypted).
    
    

    Note: Port 25 is currently disabled to maintain ECS security. If your sending application is deployed on Alibaba Cloud ECS, we recommend that you use port 80 when SSL is not checked, and use port 465 when SSL is checked. Besides, we support explict ssl/tls functionality on port 25 or 80 with “STARTTLS” smtp cmd.

  • Whitelist an IP in Amazon AWS security group from command line

    To white list an IP in Amazon AWS security group using aws command line tool, run

    aws ec2 authorize-security-group-ingress --group-id SECURITY_GROUP_ID_HERE --protocol tcp --port PORT_HERE --cidr IP_IN_CIDR_HERE
    

    Example

    aws ec2 authorize-security-group-ingress --group-id sg-a777eee6  --protocol tcp --port 443 --cidr 94.31.33.128/27
    
  • AWS Elastic Beanstalk

    Here are some useful command working with AWS Elastic Beanstalk

    eb init = initialize environment
    eb list
    eb logs
    eb console = open aws console
    eb open = open application in web browser
    eb appversion = show application versions
    eb health = show health of application
    eb codesource = select local or codecommit
    eb deploy = deploy code
    eb events = Gets recent events.
    eb create = Creates a new environment.
    eb labs download = download application to local computer
    

    Application get stored in folder

    /var/app/ondeck