Jordan | Geek Tank

MSP360 (Cloudberry) Backup SSL Certificate Expired when using Backblaze B2 Storage

byJordan
January 5, 2022
1 minute read

If you’ve been using the MSP360 backup client on older CentOS hosts, you might have run into this error.

SSL certificate problem: certificate has expired

Upon further investigation, the agent error log /opt/local/Online Backup/<INSERTUUIDHERE>/logs/log_worker_<ANOTHERUUID>}.log show the following errors.

2022-01-05 07:41:41,509729 [WARN ]: [ CBC ] [ 8 ] Exception (curlpp::LibcurlRuntimeError): SSL certificate problem: certificate has expired
2022-01-05 07:41:41,509802 [WARN ]: [ CBC ] [ 8 ] Trying 104.153.233.180...
TCP_NODELAY set
Connected to api001.backblazeb2.com (104.153.233.180) port 443 (#0)
ALPN, offering http/1.1
Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
successfully set certificate verify locations:
CAfile: /opt/local/Online Backup/share/.cacerts.pem
  CApath: none
TLSv1.2 (OUT), TLS header, Certificate Status (22):
TLSv1.2 (OUT), TLS handshake, Client hello (1):
TLSv1.2 (IN), TLS handshake, Server hello (2):
TLSv1.2 (IN), TLS handshake, Certificate (11):
TLSv1.2 (OUT), TLS alert, certificate expired (557):
SSL certificate problem: certificate has expired
Closing connection 0

2022-01-05 07:41:41,509857 [WARN ]: [ CBC ] [ 8 ] Exception (curlpp::LibcurlRuntimeError): SSL certificate problem: certificate has expired code: 60
2022-01-05 07:41:41,510057 [WARN ]: [ CBC ] [ 8 ] Ssl problem: SSL certificate problem: certificate has expired

The issue was being on the client version Linux (rpm) v. 3.2.0.114 of which this MSP360 community support thread speaks about upgrading to the latest version of the client.

https://forum.msp360.com/discussion/2429/certificate-renewal-errors-for-backblaze-b2

To do so, you need to click on “Downloads” from your MSP360 admin dashboard and request the update of the default build. The new build ended up being 3.2.3.156

Once done, you can then update your clients under RMM->Remote Management, then click the settings icon to the far right and then click “Force Update”.

UUncategorized

MikroTik Scripts

byJordan
October 8, 2021
1 minute read

Created File and Place SSH Key Contents

/file print file=pubkey
/file set pubkey.txt contents="paste your key here"

Set SSH Key for Users

/user ssh-keys import public-key-file=jordan.pub.txt

Setup Email Sending on Mikrotik via Postmark

/tool e-mail
set address=smtp.postmarkapp.com from="backup@geektank.net" password=nv283818j945vb3 port=587 start-tls=\
    yes user=nv283818j945vb3

Send a Test Email from Mikrotik

/tool e-mail send to=test@test.com subject="email test" body="email test" start-tls=yes

Backup Mikrotik Config and Email – Example 1

#
# Created Jotne 2020 v1.0
#
# Takes two different backup and send then to email
#
# backup.rsc readable backup
# Certificates, the Dude and Usermanager are also NOT exported or backed up fully and should be backed up separately.  Can be used to restore config to different routers
#
# backup.bin binary backup
# Binary backup that can only be used to fully restored the same router.
#

:local email "xyz@gmail.com"

/export file=backup
:delay 2s
/system backup save name=bin
:delay 2s
:local date [/system clock get date]
:local time [/system clock get time]
:local info [/system identity get name]
:local serial [/system routerboard get serial-number]

:local files "backup.rsc,bin.backup"
/tool e-mail send to="$email" subject="Mikrotik: Backup $info $serial" file=$files body="Automatic Backup of $info $serial at $date $time."
:delay 20s
/file remove backup.rsc
/file remove backup.bin
:log info message="Backup router=$info serial=$serial ok"

Backup Mikrotik Config and Email – Example 2

/export file=([/system identity get name] . "-" . \
[:pick [/system clock get date] 7 11] . [:pick [/system clock get date] 0 3] . [:pick [/system clock get date] 4 6]); \
/tool e-mail send to="test@test.com" subject=([/system identity get name] . " Backup " . \
[/system clock get date]) file=([/system identity get name] . "-" . [:pick [/system clock get date] 7 11] . \
[:pick [/system clock get date] 0 3] . [:pick [/system clock get date] 4 6] . ".rsc"); :delay 10; \
/file rem [/file find name=([/system identity get name] . "-" . [:pick [/system clock get date] 7 11] . \
[:pick [/system clock get date] 0 3] . [:pick [/system clock get date] 4 6] . ".rsc")]; \
:log info ("System Backup emailed at " . [/sys cl get time] . " " . [/sys cl get date])

CCode Snippets

Dell OSMA on Proxmox/Debian Buster

byJordan
October 5, 2021
1 minute read

I found a Reddit post that I wanted to make sure I kept in case it gets deleted and lost forever.

My Issue

I have some Dell servers that I wanted to enable iDRAC on remotely via a secondary network interface. Unfortunately, I didn’t have physical access. So I needed to use OSMA to enable iDRAC and configure the necessary network details.

Proxmox and Dell OSMA

If you have a Dell server and you want to be able to use the OSMA (Dell EMC OpenManage Server Administrator) on your Proxmox server. Here’s a quick guide that someone posted on Reddit and on the Proxmox forums.

WARNING

The code below on the Proxmox forums works, but the Reddit code doesn’t.

Since the code is half-baked on both. I’ve maintained my own version.

https://forum.proxmox.com/threads/dell-openmanage-on-proxmox-6-x.57932/

DELL OMSA on Proxmox 6 (Debian 10) from homelab

UUncategorized

Ultimate Linux Shell Guide for Windows 10 – WSL + Terminal + ZSH + oh-my-zsh + Powerlevel10k

byJordan
May 25, 2021
3 minute read

Draft

This is a draft article, so it’s incomplete!

Introduction

The following guide will help you install Windows Subsystem Linux, Windows Terminal and ZSH.

Step 1 – Enable WSL

It’s pretty easy to install Windows Subsystem Linux, open Powershell as Administrator, and run the following. The following command will require a restart.

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Step 2 – Install Ubuntu

Navigate to the Windows Store and search for Ubuntu. You’ll have three options.

You can choose to install Ubuntu 20.04 LTS or Ubuntu 18.04 LTS. However, if you simply choose Ubuntu, you will get Ubuntu 20.04 LTS.

Totally up to you what you pick. Either works.

Once installed, you will have to launch Ubuntu so that it can continue the installation process. You’ll choose a username and password.

Step 3 – Install Windows Terminal

Windows Terminal provides a tabbed manager for all of your various terminals, such as Powershell, Command Prompt and WSL!

You can download Windows Terminal via the Windows Store

Download Windows Terminal from Windows Store

Step 4 – Configure Windows Terminal

Windows Terminal has many configuration options that you will find useful. Here are the ones I suggest you configure.

Start up -> Default Profile -> Ubuntu
Interaction -> Automatically copy selection to clipboard

Step 5 – Fixing the WSL Home Directory Issue

When you launch WSL you’re placed into /mnt/c/Users/<username> versus the typical Linux home directory of /home/<username>

So you’ll have to make one small change.

Go into Windows Terminal Settings ( Ctrl + , )
Profiles -> Ubuntu -> General
Set Starting Directory to: //wsl$/Ubuntu/home/<username>

Step 6 – Install ZSH

You can install ZSH like you would on any Ubuntu system!

sudo apt-get update;sudo apt-get install zsh

Enter in your password and Ubuntu will update it’s cache of packages and install ZSH.

You can then type “zsh” to launch the ZSH shell. You can also set your default shell to zsh by typing “chsh”. However, you will need to know the full path to zsh, and we can find this out by typing “which ZSH”

user@comp:/mnt/c/Users/user$ which zsh
/usr/bin/zsh

Now you can type “chsh” enter your password, and the path to zsh like so!

user@comp:/mnt/c/Users/user$ chsh
Password:
Changing the login shell for user
Enter the new value, or press ENTER for the default
        Login Shell [/bin/bash]: /usr/bin/zsh

Step 7 – Install oh-my-zsh

You can install oh-my-zsh using a one-liner from their website at https://ohmyz.sh or use the command below taken from their website.

sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

Step 8 – Install Powerlevel10k Font

What’s Powerlevel10k? it’s a zsh theme and successor to Powerlevel9k. I love using it. You can see the full installation instructions on Github at https://github.com/romkatv/powerlevel10k

The installation involves installing a supported font and the theme itself. I highly suggest installing the font as you’ll not be able to see all the fancy icons.

If you visit the installation page mentioned above, you’ll find a link to download the font. Here’s a direct link.

https://github.com/romkatv/powerlevel10k-media/raw/master/MesloLGS%20NF%20Regular.ttf

Download the font, double click on the font and press install!

You’ll then need to set up Windows Terminal with the font.

Open Settings (Ctrl+,), under “Profiles” click “Ubuntu” then “Appearance” and set “Font Face” to “MesloLGS NF”. Save, exit and relaunch.

Step 9 – Installing Powerlevel10k Theme

Again, you can refer to the Github page for installation instructions or use the following one-liner for oh-my-zsh.

git clone --depth=1 https://github.com/romkatv/powerlevel10k.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k

Then you’ll need to set the theme in your .zshrc file, which should be located at the root of your home directory. Here’s a quick screenshot of where the .zshrc should be located after installing oh-my-zshrc.

There should already be a line for ZSH_THEME; simply change it to powerlevek10k

#ZSH_THEME="robbyrussell"
ZSH_THEME="powerlevel10k/powerlevel10k"

Step 10 – Powerlevek10k Configuration

Close your Windows Terminal and relaunch it. You should now be taken through the Powerlevel10k configuration screens. Once complete, you can start using your ZSH shell with oh-my-zsh and the Powerlevel10k theme!

Further Customization

There are tons of configuration options for the Powerlevel10k theme; you can read all about it on the Github repository.

https://github.com/romkatv/powerlevel10k

I’ve actually create my own custom ZSH setup that does most of the work above, except for the font (Step 8). I’ve also add in some custom aliases and commands. It’s very much a WIP!

https://github.com/jordantrizz/zshbop

Enjoy!

Updates

05-09-2022 – Added more detail about .zshrc

DDrafts

macOS Setup

byJordan
May 20, 2021
1 minute read

Draft

This is a draft article, so it’s incomplete!

Table of contents

Finder Preferences

Finder Preferences

Finder -> Preferences

General
- Show these items on the desktop – Click all
Advanced
- Show file extensions.
- Show Folders on Top -> In windows when sorting by name

DDrafts

macOS Setup iTerm2 + Homebrew + ZSH

byJordan
May 19, 2021
1 minute read

Table of contents

Step 1 – Install iTerm2
Step 2 – Install Homebrew

This guide is my setup on my macOS laptop with iTerm + Homebrew + ZSH.

Step 1 – Install iTerm2

Pretty self explanatory! Go get iTerm2 at https://iterm2.com and install it!

Step 2 – Install Homebrew

Again, another easy step! Install Homebrew at https://brew.sh/ at the time of this posting the following command works.

Also note that you can use MacPorts it’s really your choice! I can’t recall why I switched to Homebrew 🙂

 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

This will install the Xcode Command Line Tools as well!

TTidbits

MySQL 8 Increasing table_open_cache in Ubuntu 20

byJordan
May 16, 2021
1 minute read

So if you’ve ever had to increase table_open_cache in Ubuntu 20, you’ll probably find that it doesn’t work the traditional way. Which is mind blowing! But I have a guess as to why. It’s to stop you from blowing up your server from too many open files!

I had updated /etc/mysql/mysql.conf.d/mysqld.cnf with the following

table_open_cache = 8000

However, MySQL was only showing the value set at 4945

mysql> SHOW VARIABLES LIKE '%table_open%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| table_open_cache           | 4945  |
| table_open_cache_instances | 16    |
+----------------------------+-------+
2 rows in set (0.00 sec)

After some Google searching, I found this thread https://askubuntu.com/questions/1241013/cannot-increase-mysql-table-open-cache-in-ubuntu-20-04

Which pointed me to /lib/systemd/system/mysql.service and the line LimitNOFILE

# MySQL systemd service file

[Unit]
Description=MySQL Community Server
After=network.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
User=mysql
Group=mysql
PIDFile=/run/mysqld/mysqld.pid
PermissionsStartOnly=true
ExecStartPre=/usr/share/mysql/mysql-systemd-start pre
ExecStart=/usr/sbin/mysqld
TimeoutSec=600
Restart=on-failure
RuntimeDirectory=mysqld
RuntimeDirectoryMode=755
LimitNOFILE=10000

# Set enviroment variable MYSQLD_PARENT_PID. This is required for restart.
Environment=MYSQLD_PARENT_PID=1

Based on the information in the post, the LimitNOFILE is causing an automatic change to the “table_open_cache” variable. Modifying it to say 20000 will allow you to go with an 8000 table_open_cache. Copy and pasting into WordPress wouldn’t have done this post justice. So I screenshotted it.

After making the changes and restarting MySQL I got table_open_cache set to 8000

TTidbits

NetData Installation or Upgrade – Permission Denied Temp Directory not Executable

byJordan
May 16, 2021
1 minute read

You might have run into the following issue at some point when trying to install or upgrade Netdata.

~> <(curl -Ss https://my-netdata.io/kickstart.sh)
/dev/fd/63: line 167: /tmp/netdata-test.GTtLvJePOp: Permission denied

It’s usually related to your /tmp folder is mounted without execution permission and doesn’t allow execution of programs from this location. A simple work around is to set the $TMPDIR location in your shell to a place that is mount with execution permission. As an example

~> env TMPDIR=/root/tmp /usr/libexec/netdata/netdata-updater.sh

Granted you don’t have to use /root/tmp it can be any location.

UUncategorized

Cyberpanel Openlitespeed Issue with PHP Session Files Cleanup on Ubuntu 18

byJordan
April 13, 2021
1 minute read

There’s an issue with the Cyberpanel installation using Openlitespeed on Ubuntu 18 where the PHP session files aren’t removed by the system installed.

You might see issues in your /usr/local/lsws/log/error.log like this

  60789 2021-04-12 10:58:36.099775 [WARN] [30587] [UDS://tmp/lshttpd/wordp1606.sock] Can not start this external application.
  60790 2021-04-12 10:58:36.099798 [NOTICE] [30586] [billi9809.655340]: Failed to open pid file [/tmp/lshttpd/billi9809.sock.pid]: No space left on device
  60791 2021-04-12 10:58:36.099823 [ERROR] [30586] [billi9809.655340]: Failed to lock pid file for [/tmp/lshttpd/billi9809.sock]: No space left on device
  60792 2021-04-12 10:58:36.099838 [WARN] [30586] [UDS://tmp/lshttpd/billi9809.sock] Can not start this external application.
  60793 2021-04-12 10:58:36.099853 [INFO] [30586] [UDS://tmp/lshttpd/wordp1606.sock] Error Timeout, restart and try again!
  60794 2021-04-12 10:58:36.099862 [NOTICE] [30589] [billi9809.655340]: Failed to open pid file [/tmp/lshttpd/billi9809.sock.pid]: No space left on device
  60795 2021-04-12 10:58:36.099883 [ERROR] [30589] [billi9809.655340]: Failed to lock pid file for [/tmp/lshttpd/billi9809.sock]: No space left on device

The cause is related to the build of of PHP session files in the /tmp folder, which usually is cleaned out by a cronjob in /etc/cron.d/php that calls /usr/lib/php/sessionclean which of course fails to running correctly due to lsphp nor working as intended.

The solution? Create a cronjob to delete the sess_ files. Here’s an example I found online

❯ cat lsphp-session
#!/bin/bash
export PATH=$PATH:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
path_of_session_files=/tmp
find $path_of_session_files -type f -cmin +720 -name "sess_*" -delete

Add this code to a file an placed it in /etc/cron.daily, make sure to chmod u+x so its executable.

That’s it!

UUncategorized

Litespeed Web Server Statistics in Netdata Including Fixes

byJordan
January 6, 2021
1 minute read

It’s pretty simple to get this done, as there’s a plugin built in. As per this guide on the Netdata website.

https://app.netdata.cloud/spaces/lmtca-space/rooms/general/nodes/34d04048-ab58-11ea-81cb-5600029faf80#chartName=menu_ebpf&after=-900&before=0

You simple need to run the following commands to create the necessary configuration.

cd /etc/netdata   # Replace this path with your Netdata config directory, if different
sudo ./edit-config python.d/litespeed.conf

You then update a single line at the bottom of the file. Set path to /tmp/lshttpd/

local:
  path  : 'PATH'

After restarting Netdata you might find that there is no change to your dashboard. Let’s check the logs….

2021-01-06 14:01:56: python.d INFO: plugin[main] : [litespeed] built 1 job(s) configs
2021-01-06 14:02:00: python.d ERROR: litespeed[localhost] : "/tmp/lshttpd/" has no "rtreport" files or dir is not readable
2021-01-06 14:02:00: python.d INFO: plugin[main] : litespeed[localhost] : check failed

Since the file .rtreport is actually symlinked to /dev/shm/lsws/status/.rtreport which is owned by nobdy/nogroup. We simply need to add netdata to the nogroup group.

usermod -a -G nogroup netdata

You should then be able to see the Litespeed section in Netdata Cloud

Setup SSH Key Login on Mikrotik

Created File and Place SSH Key Contents

Set SSH Key for Users

Setup Email Sending on Mikrotik via Postmark

Send a Test Email from Mikrotik

Backup Mikrotik Config and Email – Example 1

Backup Mikrotik Config and Email – Example 2

My Issue

Proxmox and Dell OSMA

Introduction

Step 1 – Enable WSL

Step 2 – Install Ubuntu

Step 3 – Install Windows Terminal

Step 4 – Configure Windows Terminal

Step 5 – Fixing the WSL Home Directory Issue

Step 6 – Install ZSH

Step 7 – Install oh-my-zsh

Step 8 – Install Powerlevel10k Font

Step 9 – Installing Powerlevel10k Theme

Step 10 – Powerlevek10k Configuration

Further Customization

Updates

Finder Preferences

Step 1 – Install iTerm2

Step 2 – Install Homebrew