En

Installation Process for Sherpa Orchestrator (Local Version)#

Server Preparation#

Adding User to the Sudo Group#

# Log in as the root user
su

# Add the user to the sudo group
/sbin/usermod -aG sudo <userName>

# Reboot the system to apply changes
exit
💡 Comments on Adding User to Sudo

/sbin/usermod -aG sudo - adds the user to the sudo group

  • -a - append (adds to existing groups)
  • -G sudo - adds to the sudo group
  • <userName> - replace with your username

Important: After executing the command, you need to reboot the system to apply the changes.

Setting the Time Zone#

# Set the time zone to UTC
sudo timedatectl set-timezone UTC

# Check the settings
timedatectl
💡 Comments on Setting the Time Zone

sudo timedatectl set-timezone UTC - sets the time zone to UTC timedatectl - shows the current time and date settings

It is recommended to use UTC for server applications.

Updating the System#

# Update the package list
sudo apt -y update

# Install tools for working with repositories
sudo apt -y install software-properties-common gnupg2

# Upgrade the system
sudo apt -y upgrade
💡 Comments on System Update

sudo apt -y update - updates the list of available packages from the repositories sudo apt -y install software-properties-common gnupg2 - installs tools for working with repositories

  • software-properties-common - utilities for managing repositories
  • gnupg2 - tool for working with GPG keys

sudo apt -y upgrade - upgrades all installed packages to the latest versions

  • -y - automatic confirmation of installation

Extracting the Update Archive#

At this stage, you will extract the archive with Sherpa Orchestrator files and prepare the system for installation.

# Change to the directory with the files
cd /opt

# Find and extract the update archive (the latest version is automatically selected)
tar -xvzf "$(ls orchestrator_local_update_*.tgz | sort -V | tail -n 1)"

# Change to the directory with the extracted files
cd SherpaOrchestrator
💡 Comments on Extracting the Archive

cd /opt - changes to the directory with the installation files tar -xvzf "$(ls orchestrator_local_update_*.tgz | sort -V | tail -n 1)" - extracts the update archive

  • tar -xvzf - extracts the archive with detailed output
  • ls orchestrator_local_update_*.tgz - finds all update archive files
  • sort -V - sorts versions naturally (1.0 < 1.1 < 1.10)
  • tail -n 1 - selects the latest file

cd SherpaOrchestrator - changes to the directory with the extracted files

Expected Result: The necessary files and directories for installing Sherpa Orchestrator will be extracted.

Preparing Scripts for Execution#

# Change to the sh_scripts directory
cd sh_scripts/

# Make all scripts executable
chmod +x *.sh

# Return to the project's root directory
cd ..
💡 Comments on Preparing Scripts

cd sh_scripts/ - changes to the directory with the installation scripts

  • chmod +x *.sh - sets execution rights for all shell scripts

  • chmod +x - adds execution rights

  • *.sh - all files with the .sh extension

cd .. - returns to the project's root directory

Initializing Configuration#

# Execute the initialization of settings
sudo ./sh_scripts/create_config.sh
💡 Comments on Initializing Configuration

sudo ./sh_scripts/create_config.sh - runs the script to initialize the basic configuration

  • Creates necessary directories
  • Configures basic system parameters
  • Prepares the structure for further configuration

Installing MariaDB Database Management System#

# Add the MariaDB repository
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0xF1656F24C74CD1D8
sudo add-apt-repository "deb [arch=amd64] http://mariadb.mirror.liquidtelecom.com/repo/10.5/debian $(lsb_release -cs) main"

# Update the package list
sudo apt update

# Install MariaDB
sudo apt install mariadb-server mariadb-client
💡 Comments on Installing MariaDB

Adding the repository:

  • sudo apt-key adv --recv-keys - adds the GPG key of the repository
  • sudo add-apt-repository - adds the MariaDB repository
  • $(lsb_release -cs) - automatically determines the codename of the distribution

Installing packages:

  • mariadb-server - database server
  • mariadb-client - client for connecting to the database

Basic Security Configuration for MySQL/MariaDB#

# Run the security setup script
sudo mysql_secure_installation

Security setup questions and options:

Switch to unix_socket authentication [Y/n] y
Change the root password? [Y/n] y
Remove anonymous users? [Y/n] y
Disallow root login remotely? [Y/n] n
Remove test database and access to it? [Y/n] y
Reload privilege tables now? [Y/n] y
💡 Comments on Security Configuration

mysql_secure_installation - interactive script for basic security configuration of MySQL/MariaDB

Recommended answers:

  • Switch to unix_socket authentication - yes (y) - use authentication via unix socket
  • Change the root password - yes (y) - set a password for the root user
  • Remove anonymous users - yes (y) - remove anonymous users
  • Disallow root login remotely - no (n) - allow root to connect remotely
  • Remove test database - yes (y) - remove the test database
  • Reload privilege tables - yes (y) - reload the privilege tables

Configuring Database and User#

# Connect to MySQL as root
mysql -u root -p

# Execute the following commands in the MySQL shell:

-- Adding a database user (replace the password with a strong one)
GRANT ALL ON orchestrator.* TO 'orchestrator'@'localhost' IDENTIFIED BY 'mD2vjt(HqZKW' WITH GRANT OPTION;

-- Reloading privileges
FLUSH PRIVILEGES;

-- Selecting the database
USE orchestrator;

-- Updating the account record
UPDATE `accounts` SET `parent_account_id` = '1' WHERE `accounts`.`id` = 1;

-- Exiting MySQL
exit;
💡 Comments on Database Configuration

GRANT ALL ON orchestrator.* TO 'orchestrator'@'localhost'... - creates the user orchestrator with full rights on the orchestrator database

  • 'orchestrator'@'localhost' - the user can connect only from localhost
  • WITH GRANT OPTION - the user can grant rights to other users

FLUSH PRIVILEGES - reloads the privilege tables

UPDATE accounts... - sets the parent_account_id for the system account

Important: Replace the password 'mD2vjt(HqZKW' with a strong unique password!

Installing Web Server and PHP#

# Install necessary packages
sudo apt -y install lsb-release apt-transport-https ca-certificates curl

# Add the PHP repository
sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list

# Update the package list
sudo apt-get update

# Install Nginx and PHP 8.5
sudo apt-get install -y nginx php8.5 php8.5-cli php8.5-fpm php8.5-opcache php8.5-curl php8.5-mbstring php8.5-zip php8.5-xml php8.5-mysql php8.5-pdo-mysql php8.5-pgsql
💡 Comments on Installing Web Server

Adding the PHP repository:

  • Adds the GPG key of the PHP repository from sury.org
  • Adds the repository for PHP 8.5

Installing packages:

  • nginx - web server
  • php8.5 - PHP interpreter
  • php8.5-fpm - FastCGI Process Manager for PHP
  • php8.5-* - PHP extensions for various functions

Configuring PHP#

# Open the PHP-FPM configuration file
sudo nano /etc/php/8.5/fpm/php.ini

# Find and change the following parameters:
upload_max_filesize = 100M
post_max_size = 100M

# Save the file and restart PHP-FPM
sudo service php8.5-fpm restart
💡 Comments on Configuring PHP

upload_max_filesize = 100M - maximum size of the uploaded file post_max_size = 100M - maximum size of POST data

sudo service php8.5-fpm restart - restarts PHP-FPM to apply changes

Configuring Nginx#

# Open the Nginx configuration file
sudo nano /etc/nginx/nginx.conf

# Find the http section and add:
client_max_body_size 100m;

# Save the file
💡 Comments on Configuring Nginx

client_max_body_size 100m - sets the maximum size of the client request body

  • Should be in the http { ... } section
  • The value corresponds to PHP settings

Configuring Domain and Sherpa Configuration#

# Open the domain configuration file
sudo nano /opt/SherpaOrchestrator/backend/config/domain.conf

# Replace "orchestrator.sherparpa.ru" with your domain or IP address (4 replacements in total)

# Define the path to the PHP-FPM socket depending on the OS:
# For Ubuntu 18.04:
# fastcgi_pass unix:/var/run/php/php8.5-fpm.sock;

# For Debian:
fastcgi_pass unix:/run/php/php8.5-fpm.sock;

# For CentOS 8:
# fastcgi_pass unix:/run/php-fpm/www.sock;

# For CentOS 7:
# fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
💡 Comments on Domain Configuration

Domain Replacement: You need to replace all occurrences of "orchestrator.sherparpa.ru" with your actual domain or IP address

Configuring PHP-FPM socket: The path depends on the Linux distribution:

  • Debian/Ubuntu typically: /run/php/php8.5-fpm.sock
  • CentOS: /run/php-fpm/www.sock

Important: If the path is incorrect, check the Nginx logs: /var/log/nginx/error.log

Configuring SSL Certificates#

# Copy the SSL certificates to the configuration directory
# Replace the paths with the actual paths to your certificates
sudo cp /path/to/your/certificate.crt /opt/SherpaOrchestrator/backend/config/certs/orchestrator.crt
sudo cp /path/to/your/private.key /opt/SherpaOrchestrator/backend/config/certs/orchestrator.key
💡 Comments on SSL Certificates

Certificate Requirements:

  • The certificate must be renamed to orchestrator.crt
  • The private key must be renamed to orchestrator.key
  • Formats: .crt/.pem for the certificate, .key for the key

Obtaining Certificates:

  • Contact the system administrator for corporate certificates
  • Use Let's Encrypt for free certificates
  • For testing, self-signed certificates can be created

Configuring Application Settings#

# Open the application configuration file
sudo nano /opt/SherpaOrchestrator/backend/config/config.ini

# Configure the database connection parameters:
database_host=127.0.0.1
database_port=3306
database_user=orchestrator
database_password="mD2vjt(HqZKW"
database_dbname=orchestrator
💡 Comments on Application Configuration

Database Parameters:

  • database_host - address of the database server (usually 127.0.0.1 for local installation)
  • database_port - MySQL/MariaDB port (default is 3306)
  • database_user - database user (orchestrator)
  • database_password - user password (must match the one created earlier)
  • database_dbname - database name (orchestrator)

Important: The password must exactly match the password set when creating the database user

Configuring phinx.php (DB Migrations)#

The file backend/config/phinx.php is used by Phinx to perform migrations. The password in it must match the DB password from config.ini:

sudo nano /opt/SherpaOrchestrator/backend/config/phinx.php

In the environments section under the used DB block (for example, orchestrator or mysql), the parameter 'pass' must match database_password from config.ini. Otherwise, migrations during installation or update will fail.

Activating Nginx Configuration#

# Copy the configuration to sites-available
sudo cp /opt/SherpaOrchestrator/backend/config/domain.conf /etc/nginx/sites-available/default

# Restart Nginx
sudo systemctl restart nginx
💡 Comments on Activating Nginx

sudo cp ... /etc/nginx/sites-available/default - copies the site configuration to active Nginx configurations

sudo systemctl restart nginx - restarts Nginx to apply the new configuration

Setting Permissions#

# Set correct permissions and owner
sudo chown -R www-data:www-data /opt/SherpaOrchestrator
sudo chmod -R 775 /opt/SherpaOrchestrator
💡 Comments on Setting Permissions

sudo chown -R www-data:www-data /opt/SherpaOrchestrator - sets www-data as the owner of the files

  • www-data - user of the Nginx/PHP-FPM web server
  • -R - recursively for all files and directories

sudo chmod -R 775 /opt/SherpaOrchestrator - sets permissions on files

  • 775 - owner and group can read/write/execute, others can only read/execute

Creating an Archive Database#

-- Connect to MySQL
mysql -u root -p

-- Create the archive database
CREATE DATABASE IF NOT EXISTS orchestrator_archive CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- Exit
exit;
💡 Comments on Archive Database

CREATE DATABASE IF NOT EXISTS orchestrator_archive - creates the archive database

  • IF NOT EXISTS - creates only if it does not exist
  • CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci - sets UTF-8 encoding with Unicode support

Updating Database Structure#

# Change to the application directory
cd /opt/SherpaOrchestrator

# Check the database connection settings in the phinx.php file
sudo nano backend/config/phinx.php

# Execute the database update
sudo ./sh_scripts/migrate.sh

Expected result after a successful update:

Phinx by CakePHP - https://phinx.org.

using config file ./backend/config/phinx.php
using config parser php
using migration paths
using seed paths
warning no environment specified, defaulting to: orchestrator
using adapter mysql
using database orchestrator

== 20241201120000 CreateInitialSchema: migrated 0.0123s ==
== 20241201120000 CreateInitialSchema: migrated 0.0123s ==

All Done. Took 0.0345s
💡 Comments on Database Update

Checking phinx.php:

'environments' => [
    'orchestrator' => [
        'adapter' => 'mysql',
        'host' => 'localhost',
        'name' => 'orchestrator',
        'user' => 'orchestrator',
        'pass' => 'mD2vjt(HqZKW',
        // ...
    ],
],

sudo ./migrate.sh - runs database migrations via Phinx

  • Updates table structures
  • Adds necessary indexes
  • Creates triggers and procedures

Configuring CRON Jobs#

# Open the CRON editor
sudo crontab -e

# Add a line to run the task scheduler:
* * * * * php /opt/SherpaOrchestrator/backend/app/scheduleCronRunner.php

# Save and exit the editor
💡 Comments on Configuring CRON

sudo crontab -e - opens the CRON editor for the root user

  • * * * * * - runs every minute
  • php /opt/SherpaOrchestrator/backend/app/scheduleCronRunner.php - runs the Sherpa task scheduler

Task Scheduler:

  • Manages the schedule for task execution
  • Processes delayed tasks
  • Performs automatic cleanup

Installing Node.js and PM2#

# Install Node.js 22.x
sudo curl -sL https://deb.nodesource.com/setup_22.x | bash -
sudo apt-get install -y nodejs

# Update npm
sudo npm install -g npm@latest

# Install PM2
sudo npm install -g pm2
💡 Comments on Installing Node.js and PM2

curl -sL https://deb.nodesource.com/setup_22.x | bash - - adds the Node.js repository

  • -sL - silent and follow redirects
  • setup_22.x - script for Node.js version 22.x

sudo apt-get install -y nodejs - installs Node.js

sudo npm install -g pm2 - installs PM2 globally

  • PM2 - process manager for Node.js applications
  • Automatic restart of applications
  • Log management and monitoring

Configuring WebSocket Service#

# Change to the websocket service directory
cd /opt/SherpaOrchestrator/backend/app/websocket/

# Install dependencies
sudo npm install

# Start the service via PM2
sudo pm2 start index.js --watch --ignore-watch="node_modules" --name "Websockets"

# Configure PM2 to start on boot
sudo pm2 startup

# Save PM2 configuration
sudo pm2 save
💡 Comments on Configuring WebSocket

sudo npm install - installs Node.js dependencies from package.json

sudo pm2 start index.js --watch --ignore-watch="node_modules" --name "Websockets"

  • --watch - restarts on file changes
  • --ignore-watch="node_modules" - ignore changes in node_modules
  • --name "Websockets" - name of the process in PM2

sudo pm2 startup - configures PM2 to start on system boot sudo pm2 save - saves the current process configuration

Configuring Domain Name (Optional)#

If you are satisfied with access to the Orchestrator interface via the server IP, skip this step.

# Open the hosts file
sudo nano /etc/hosts

# Add an entry (replace with your IP and domain):
192.168.1.100    orchestrator.mycompany.com

Initializing the Orchestrator#

  1. Access the URL: https://<your_domain_or_IP>/setup.php
  2. If the IP address displays correctly, click the "Submit" button
  3. The script response will include the GUID of the orchestrator - write it down along with the registration details

This completes the installation!

The Sherpa Orchestrator system is available in the browser at the domain name or IP address.

Next, you need to add and activate the license for the orchestrator and the license for the robots in the web interface of the orchestrator.