255 lines
6.2 KiB
Markdown
255 lines
6.2 KiB
Markdown
# Python Script for Downloading the Latest Amateur Radio Callbook of Austria
|
|
(c) KW4NZ since 2024
|
|
- Guide is based on Debian 12.5, and Arch Linux
|
|
- Last modified 2024-06-12
|
|
|
|
## Prerequisites
|
|
|
|
1. A running Linux system, e.g. Debian, Arch, etc.
|
|
0. User has administrative access via sudo
|
|
0. Python Version >3
|
|
0. Chromium Web browser (headless)
|
|
0. Selenium for Python via Webdriver Manager
|
|
0. Optional: User has an ssh key pair
|
|
0. Optional: git, ? wget, ? curl
|
|
|
|
## Installation of Python, Selenium, Chromium
|
|
|
|
### Debian Linux
|
|
|
|
#### Check Version
|
|
|
|
|
|
$ cat /etc/debian_version
|
|
12.5
|
|
|
|
$ cat /etc/os-release
|
|
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
|
|
NAME="Debian GNU/Linux"
|
|
VERSION_ID="12"
|
|
VERSION="12 (bookworm)"
|
|
VERSION_CODENAME=bookworm
|
|
ID=debian
|
|
HOME_URL="https://www.debian.org/"
|
|
SUPPORT_URL="https://www.debian.org/support"
|
|
BUG_REPORT_URL="https://bugs.debian.org/"
|
|
|
|
Upgrade your Debian installation if the version is lower than the above value.
|
|
|
|
#### Update/Upgrade Debian
|
|
|
|
$ sudo apt update
|
|
$ sudo apt upgrade
|
|
|
|
If this does not work, then the user \<username\> must be added to the sudo group with :
|
|
|
|
$ su -
|
|
Password:
|
|
root@debian:~# usermod -aG sudo <username>
|
|
For the group membership to take effect, the user must log out and log in again or the system must be restarted.
|
|
|
|
#### Python3
|
|
Check if Python is already installed in Debian:
|
|
|
|
$ python3 --version
|
|
Python 3.11.2
|
|
|
|
If it is not already installed, then:
|
|
|
|
$ sudo apt install python3
|
|
|
|
#### Chromium Browser
|
|
|
|
$ sudo apt install chromium-driver
|
|
|
|
#### Manager (selenium)
|
|
Afterwards, selenium is installed:
|
|
|
|
$ sudo apt install python3-selenium
|
|
|
|
#### SSH key generation
|
|
|
|
The program `openssh` is preinstalled on Debian systems. But there is no private/public key installed yet. Best practices generating the key pair:
|
|
|
|
$ ssh-keygen -t ed25519 -C "your_email@example.com"
|
|
|
|
Personally, I don't use a different name for the keys, I just click through the process.
|
|
The same applies to the passphrase.
|
|
Now go to the website https://git.kuschel.at and copy the content of the public key from ~/.ssh/id__ed25519.pub to the "Manage SSH keys" -- via "Add Key".
|
|
|
|
$ cat .ssh/id_ed25519.pub
|
|
|
|
#### Git installation and clone the scripts from git.kuschel.at
|
|
|
|
$ sudo apt install git
|
|
|
|
You have to globally configure your `git` with:
|
|
|
|
$ git config --global user.name "John Doe"
|
|
$ git config --global user.email your_email@example.com
|
|
Checking the git configuration with:
|
|
|
|
$ git config -l
|
|
user.name=<your name>
|
|
user.email=<your e-mail>
|
|
etc.
|
|
|
|
Now create a folder and cd into it
|
|
|
|
$ mkdir gitea
|
|
$ cd gitea
|
|
|
|
Clone the repository `script` to your site with:
|
|
|
|
~/gitea$ git clone ssh://git@kuschel.at:21861/public/scripts.git
|
|
|
|
### Check and Test a Website
|
|
|
|
Either go to the directory `~/gitea/scripts/afu/` and run the file `test.py` or create a similar one:
|
|
|
|
#!/usr/bin/python3
|
|
from selenium import webdriver
|
|
from selenium.webdriver.chrome.service import Service as ChromiumService
|
|
print('We try to connect to https://kuschel.at and get an answer "Family Kuschel and friends"')
|
|
options = webdriver.ChromeOptions()
|
|
options.add_argument('--headless')
|
|
options.add_argument('--no-sandbox')
|
|
options.add_argument('--disable-dev-shm-usage')
|
|
driver = webdriver.Chrome(options=options)
|
|
driver.get("https://kuschel.at")
|
|
print(driver.title)
|
|
driver.close()
|
|
|
|
--
|
|
|
|
|
|
|
|
### Arch Linux (@todo)
|
|
|
|
#### Check Version
|
|
|
|
$ cat /etc/lsb-release
|
|
DISTRIB_ID="Arch"
|
|
DISTRIB_RELEASE="rolling"
|
|
DISTRIB_DESCRIPTION="Arch Linux"
|
|
|
|
$ cat /etc/os-release
|
|
NAME="Arch Linux"
|
|
PRETTY_NAME="Arch Linux"
|
|
ID=arch
|
|
BUILD_ID=rolling
|
|
ANSI_COLOR="38;2;23;147;209"
|
|
HOME_URL="https://archlinux.org/"
|
|
DOCUMENTATION_URL="https://wiki.archlinux.org/"
|
|
SUPPORT_URL="https://bbs.archlinux.org/"
|
|
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
|
|
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
|
|
LOGO=archlinux-logo
|
|
|
|
#### Update Arch Linux
|
|
Because Arch linux is a rolling distribution, simply update the version to the latest version with:
|
|
|
|
$ sudo pacman -Syuv
|
|
|
|
#### Installing of yay (package manager yet another yogurt)
|
|
|
|
$ sudo pacman -Sy --needed git base-devel
|
|
$ cd
|
|
$ git clone https://aur.archlinux.org/yay.git
|
|
$ cd yay
|
|
$ makepkg -si
|
|
|
|
#### Python3
|
|
Check if Python is already installed in Arch Linux:
|
|
|
|
$ python --version
|
|
Python 3.12.3
|
|
|
|
If it is not already installed, then:
|
|
|
|
$ yay -S python
|
|
|
|
#### Chromium Browser
|
|
|
|
$ yay -S chromium
|
|
|
|
#### Manager (selenium)
|
|
Afterwards, selenium is installed:
|
|
|
|
$ yay -S python-selenium
|
|
|
|
$ yay -S selenium-manager
|
|
|
|
#### SSH key generation
|
|
|
|
The program `openssh` is not preinstalled on Arch Linux. Also, there is no private/public key installed yet.
|
|
|
|
$ yay -S openssh
|
|
|
|
Best practices generating the key pair:
|
|
|
|
$ ssh-keygen -t ed25519 -C "your_email@example.com"
|
|
|
|
Personally, I don't use a different name for the keys, I just click through the process.
|
|
The same applies to the passphrase.
|
|
Now go to the website https://git.kuschel.at and copy the content of the public key from ~/.ssh/id__ed25519.pub to the "Manage SSH keys" -- via "Add Key".
|
|
|
|
$ cat .ssh/id_ed25519.pub
|
|
|
|
#### Git installation and clone the scripts from git.kuschel.at
|
|
|
|
$ yay -S git less
|
|
|
|
You have to globally configure your `git` with:
|
|
|
|
$ git config --global user.name "John Doe"
|
|
$ git config --global user.email your_email@example.com
|
|
Checking the git configuration with:
|
|
|
|
$ git config -l
|
|
user.name=<your name>
|
|
user.email=<your e-mail>
|
|
etc.
|
|
|
|
Now create a folder and cd into it
|
|
|
|
$ mkdir gitea
|
|
$ cd gitea
|
|
|
|
Clone the repository `script` to your site with:
|
|
|
|
~/gitea$ git clone ssh://git@kuschel.at:21861/public/scripts.git
|
|
|
|
|
|
## Connecting to MariaDB database
|
|
|
|
### Install mariadb database
|
|
|
|
$ yay -S mariadb
|
|
|
|
Configure the database with e.g.
|
|
|
|
$ sudo mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
|
|
|
|
Enable and start the database service with:
|
|
|
|
$ sudo systemctrl enable mariadb.service
|
|
$ sudo systemctrl start mariadb.service
|
|
|
|
#### Create a user "om" who has only access to the database table
|
|
|
|
# mariadb
|
|
|
|
CREATE DATABASE callbook;
|
|
|
|
GRANT ALL PRIVILEGES ON callbook.* TO 'om'@'localhost' IDENTIFIED BY 'oe3tkt';
|
|
|
|
FLUSH PRIVILEGES;
|
|
QUIT;
|
|
|
|
## Install python-mariadb
|
|
|
|
$ yay -S python-mysql-connector
|
|
|
|
Hint: At the moment the compilation fails. Will be updated soon.
|
|
See https://jira.mariadb.org/projects/CONPY/issues/CONPY-284 (2024-06-16) |