Category: Python

  • pyenv

    pyenv

    pyenv is a popular command-line tool that helps Python developers manage multiple Python versions efficiently. If you work on various Python projects, each requiring a different version of Python, pyenv keeps your workflow smooth by allowing you to install, switch, and manage Python versions effortlessly.

    https://github.com/pyenv/pyenv

    To install, run

    curl -fsSL https://pyenv.run | bash

    Edit ~/.profile

    vi ~/.profile

    add at the end of the file

    export PYENV_ROOT="$HOME/.pyenv"
    [[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
    eval "$(pyenv init - bash)"

    Install build dependency

    on Ubunutu, run

    apt update; apt install make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev curl git libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

    For other operating systems, see

    https://github.com/pyenv/pyenv/wiki#suggested-build-environment

    Install a specific Python version

    pyenv install 3.10.7

    Set the global Python version:

    pyenv global 3.10.7

    Set a local version for a project:

    pyenv local 3.9.13

    To list all installed python versions, run:

    pyenv versions

    Back to Python

  • Serve a Django application using Gunicorn

    Serve a Django application using Gunicorn

    Create systemd service file

    vi /usr/lib/systemd/system/backend.service
    

    Add

    [Unit]
    Description=Gunicorn instance to serve Django Application
    After=network.target
    
    [Service]
    User=www-data
    Group=www-data
    Environment="APP=app"
    WorkingDirectory=/path/to/app/backend
    Environment="PATH=/path/to/app/backend/venv/bin"
    ExecStart=/path/to/app/backend/venv/bin/gunicorn -w 3 --timeout 120 --bind 0.0.0.0:8080 backend.wsgi
    
    [Install]
    WantedBy=multi-user.target
    

    On the above, you need to replace “/path/to/app/backend” with the path to your application. In backend.wsgi, backend is the application name here, change it to whatever your Django application name is.

    User=www-data
    Group=www-data
    

    You can change this to whatever user you use to upload your Django application files.

    Enable and start the service

    systemctl enable backend.service
    systemctl start backend.service
    

    Now your Django application will be available on port 8080. You can configure nginx or apache to proxy to this port.

    Here is a sample nginx config

    server {
        listen 443 ssl;
        server_name app.serverok.in;
        ssl_certificate /etc/ssl/app.serverok.in.crt;
        ssl_certificate_key /etc/ssl/app.serverok.in.key;
        root /var/www/app.serverok.in/html/;
        index index.html;
        location /media/ {
            alias /var/www/app.serverok.in/html/media/;
        }
    
        location /api/ {
            include proxy_params;
            proxy_pass http://localhost:8080/api/;
        }
    }
    

    Here Django application is served as /api/ folder, you can change this as required.

    Back to Gunicorn

  • ModuleNotFoundError: No module named ‘MySQLdb’

    ModuleNotFoundError: No module named ‘MySQLdb’

    When running migration for a Django application with “python manage.py migrate”, I got the following error message

    ModuleNotFoundError: No module named 'MySQLdb'
    

    To fix install

    pip install mysqlclient
    

    This will require MySQL development package. On the Ubuntu server running MariaDB, I installed packaged libmariadb-dev

    sudo apt install libmariadb-dev
    

    Back to Errors

  • error: invalid command ‘bdist_wheel’

    error: invalid command ‘bdist_wheel’

    When installing a python module on Ubuntu, I got the following error message

    error: invalid command 'bdist_wheel'
    

    The error is fixed by installing

    apt install -y python3-dev libpq-dev python3-wheel gcc
    pip3 install wheel
    

    Back to Errors

  • How to install Python 3.10 on CentOS 7

    How to install Python 3.10 on CentOS 7

    First, install the dependency

    yum install -y wget zlib-devel
    yum groupinstall -y "Development tools"
    

    Install OpenSSL as the openssl-devel package provided by CentOS is old version. Install OpenSSL from source with

    cd /usr/local/src
    wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
    tar xvf openssl-1.1.1g.tar.gz
    cd openssl-1.1.1g
    ./config --prefix=/usr/serverok/openssl --openssldir=/usr/serverok/openssl no-ssl2
    make
    make install
    

    Run following commands

    export PATH=/usr/serverok/openssl/bin:$PATH
    export LD_LIBRARY_PATH=/usr/serverok/openssl/lib
    export LC_ALL="en_US.UTF-8"
    export LDFLAGS="-L/usr/serverok/openssl/lib -Wl,-rpath,/usr/serverok/openssl/lib"
    

    Download the latest version of Python from

    https://www.python.org/downloads/

    To install, run

    cd /usr/local/src
    wget https://www.python.org/ftp/python/3.10.6/Python-3.10.6.tar.xz
    tar xvf Python-3.10.6.tar.xz
    cd /usr/local/src/Python-3.10.6
    make clean && make distclean
    ./configure --enable-optimizations  --with-openssl=/usr/serverok/openssl/
    make altinstall
    

    Python 10 will be installed in folder

    /usr/local/bin/python3.10
    

    To install pip modules, you can use

    python3.10 -m pip install -r requirements.txt 
    

    Back to Python

  • ModuleNotFoundError: No module named ‘PIL’

    ModuleNotFoundError: No module named ‘PIL’

    When running a python application, I got the error message ModuleNotFoundError: No module named ‘PIL’

    (venv) boby@sok-01:~/work/powder (master)$ python main.py 
    Traceback (most recent call last):
      File "main.py", line 6, in 
        from extras import *
      File "/home/boby/work/powder/extras.py", line 2, in 
        from PIL import Image, ImageDraw, ImageFont
    ModuleNotFoundError: No module named 'PIL'
    (venv) boby@sok-01:~/work/powder (master)$
    

    To fix the error install Pillow with

    pip install Pillow
    
  • This version of ChromeDriver only supports Chrome version

    This version of ChromeDriver only supports Chrome version

    When running a python application, I get the error message

    (venv) boby@sok-01:~/work/powder (master)$ python main.py 
    59 seconds remaining
    Process Process-1:
    Traceback (most recent call last):
      File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
      File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
        self._target(*self._args, **self._kwargs)
      File "main.py", line 51, in main
        driver = webdriver.Chrome(
      File "/home/boby/work/powder/venv/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
        super().__init__(DesiredCapabilities.CHROME['browserName'], "goog",
      File "/home/boby/work/powder/venv/lib/python3.8/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__
        super().__init__(
      File "/home/boby/work/powder/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 277, in __init__
        self.start_session(capabilities, browser_profile)
      File "/home/boby/work/powder/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
        response = self.execute(Command.NEW_SESSION, parameters)
      File "/home/boby/work/powder/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
        self.error_handler.check_response(response)
      File "/home/boby/work/powder/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
        raise exception_class(message, screen, stacktrace)
    selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 98
    Current browser version is 103.0.5060.53 with binary path /usr/bin/google-chrome
    

    To fix the error, download the chrome driver for your google chrome browser version from

    https://chromedriver.chromium.org/downloads

    Extract the file. You will get a file with the name “chromedriver”. Replace your chromedriver file with this one.

    sudo mv ~/Downloads/chromedriver_linux64/chromedriver /usr/local/bin/chromedriver
    
  • OSError: mysql_config not found

    OSError: mysql_config not found

    When installing the python mysqlclient module with pip, I got the error “OSError: mysql_config not found”.

    (venv) root@sok:~# pip install mysqlclient
    Collecting mysqlclient
    Downloading mysqlclient-2.1.0.tar.gz (87 kB)
    |████████████████████████████████| 87 kB 1.2 MB/s
    ERROR: Command errored out with exit status 1:
    command: /root/venv/bin/python3 -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘”‘”‘/tmp/pip-install-r5qp0a97/mysqlclient/setup.py’”‘”‘; __file__=’”‘”‘/tmp/pip-install-r5qp0a97/mysqlclient/setup.py’”‘”‘;f=getattr(tokenize, ‘”‘”‘open’”‘”‘, open)(__file__);code=f.read().replace(‘”‘”‘\r\n’”‘”‘, ‘”‘”‘\n’”‘”‘);f.close();exec(compile(code, __file__, ‘”‘”‘exec’”‘”‘))’ egg_info –egg-base /tmp/pip-install-r5qp0a97/mysqlclient/pip-egg-info
    cwd: /tmp/pip-install-r5qp0a97/mysqlclient/
    Complete output (15 lines):
    /bin/sh: 1: mysql_config: not found
    /bin/sh: 1: mariadb_config: not found
    /bin/sh: 1: mysql_config: not found
    Traceback (most recent call last):
    File ““, line 1, in
    File “/tmp/pip-install-r5qp0a97/mysqlclient/setup.py”, line 15, in
    metadata, options = get_config()
    File “/tmp/pip-install-r5qp0a97/mysqlclient/setup_posix.py”, line 70, in get_config
    libs = mysql_config(“libs”)
    File “/tmp/pip-install-r5qp0a97/mysqlclient/setup_posix.py”, line 31, in mysql_config
    raise OSError(“{} not found”.format(_mysql_config_path))
    OSError: mysql_config not found
    mysql_config –version
    mariadb_config –version
    mysql_config –libs
    —————————————-
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    (venv) root@sok:~#

    The error is fixed by installing MySQL development library.

    On Ubuntu server with MariaDB, i installed

    apt install libmariadb-dev
    

    For MySQL, you can use

    apt-get install libmysqlclient-dev
    
  • Uvicorn

    Uvicorn

    Uvicorn is a lightning-fast ASGI server used to serve Python ASGI frameworks like FastAPI, Django Channels, Quart, BlackSheep, etc…

    https://www.uvicorn.org

    To install Uvicorn, run

    pip install uvicorn

    To run, you can use

    uvicorn main:app --host 0.0.0.0 --port 8000 --reload

    See Python

  • Flask Python framework

    Flask is a Python framework. You can find more info at the official website.

    https://flask.palletsprojects.com/

    How to start a Flask application?

    To start flask application, run

    python app.py
    

    How to run flask application on port 80

    To run flask application on port 80, use command

    flask run --host 0.0.0.0 --port 80
    

    Or edit app.py, use the following code

    if __name__ == '__main__':
        app.run(host='0.0.0.0', port=80)
    

    Or you can use the following environment variables

    FLASK_RUN_HOST=0.0.0.0
    FLASK_RUN_PORT=80
    

    Port 80 can only be used by the root user, so you need to run the above command as root. Only do this for testing as it is insecure to run an application as the root user.

    Default flask app name

    When you run

    flash run
    

    It looks for app.py file. If you need to change the file name, you can use the environment variable

    FLASK_APP=application.py
    

    See python

  • Python mechanize HTTP Error 403 request disallowed by robots.txt

    Python mechanize HTTP Error 403 request disallowed by robots.txt

    When running following python script i get error “HTTP Error 403 request disallowed by robots.txt”

    from mechanize import Browser
    
    a = ['https://google.com', 'https://serverok.in', 'https://msn.com']
    
    br = Browser() 
    
    for x in range(len(a)):     
        br.open(a[x]) 
        print("Website title: ")
        print(br.title())
        print("\n")
    

    To fix this, find

    br = Browser() 
    

    Add below

    br.set_handle_robots(False)
    br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36')]
    

    set_handle_robots is used to disable robots.txt checking. Second line will set a User-agent, so remote won’t won’t block you as a robot.

    See Python

  • error: Python.h: No such file or directory

    error: Python.h: No such file or directory

    When install dlib python module, i get error

    ERROR: Failed building wheel for dlib fatal 
    error: Python.h: No such file or directory
    

    This is because missing python3-dev package. To fix, install

    apt install python3-dev
    

    If you are using python virtualenv, you will need to recreate the virtualenv after installing dev package.

    virtualenv -p /usr/bin/python3 venv