infrastructure on Kevin Heruer

Migrating from Proxmox VE to TrueNAS Scale

Tue, 13 Sep 2022 19:30:00 +0100

I’ve been using Proxmox VE for over a year now, ever since I built the new server. However, I started to run into limitations mainly because Proxmox VE is not the right tool for my situation. I always thought I needed the flexibility off full blown VM’s and that the storage side of things came after, I have been proven wrong.

I like to keep things separated, this means that I have over 15 VM’s running and most of them are just simple services like a load balancer or a dashboard. Running full VM’s for this is overkill and has a huge impact on the server’s resources, including storage. Because they’re all VM’s they need a boot disk and some storage, the default for a simple Debian server is around 10 to 20GB which is not fully utilised.

Also, most of the work and usage is storage. Nextcloud and Plex are the biggest services running on the server, but far from being the most critical. Home Assistant is the most important service running right now which controls everything in the house.

Knowing this, the idea is to move to TrueNAS Scale. Mainly because TrueNAS is focused on storage rather than being a Virtual Machine first approach. Especially now containerization is the norm, even in enterprise environments.

I do know that Proxmox VE has the option to run LXC, but with LXC being the less popular its safer to go for TrueNAS scale which implements Kubernetes. Plus TrueNAS has the whole storage first approach, which for me is the better option because reliable storage (and more user friendly interface) has become more important for me than flexibility in running environments.

I will still recommend Proxmox VE to people who need full control over their VM’s, it’s especially good for running multiple full blown desktops with remote login (think thin clients etc).

Increasing VM disk size: Final edition

Fri, 23 Jul 2021 21:47:00 +0100

This post merges both A little follow up on this post and Increasing Disk Size on Existing VM and fixes a couple of mistakes.

First off, go to your Proxmox host. Run the resize command:

bash


qm resize <vmid> <disk> <size>

where vmid is the ID in your VM ID, disk is the identifier of the disk inside the VM, it’s most likely to be scsi0. size is the size you want to add or remove, something like +100G or +1T is both valid.

Now go into your VM, run the following command to check if the system has notices a disk resize:

bash


dmesg | grep sda

Where sda is the disk, it could be vda or sdb.

Run the following command to get your disk and display the partitions, remember to change sda if required.

bash


fdisk -l /dev/sda | grep ^/dev

Remember the number of the partition you want to increase, it’s most likely the biggest one.

Run the following command to open parted with the drive:

bash


parted /dev/sda

Run a list to check if everything is recognized correctly and run the following command:

bash


resizepart <partition number, integer only> 100%

You could change the 100% to add only a percentage of the free space and add the remaining space to another partition, but to keep it simple we’ll give it all to this partition.

If it asks to fix the unused space type F to fix it, if it asks for a partition number and size; just put in the values like above.

Ctrl-C out of Parted and run df -h, you’ll notice that you still do not have your extra space! Run pvresize /dev/<block> to start resizing, remember to subsitute <block> with your partition like sda3.

Now run lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv and resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv to actually resize your LVM partition. Remember to replace /dev/mapper/ubuntu--vg-ubuntu--lv with your LVM partition, you can find it by running df -h and looking for the one mounted on /.

That’s it! The drive has been expanded and readily available, no need to reboot!

Sources:
https://pve.proxmox.com/wiki/Resize_disks

Follow up on increasing existing VM disk size

Sun, 21 Mar 2021 13:47:57 +0100

A little follow up on this post: Increasing Disk Size on Existing VM Disk

When extending an LVM system you should also run the following commands:

pvresize /dev/sda3 (replace with your partition)

lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv (replace with your lvm partition)

resize2fs /dev/mapper/dev/ubuntu--vg-ubuntu--lv (again replace with your lvm partition)

Increasing Disk Size on Existing VM Disk

Thu, 21 Jan 2021 17:05:03 +0100

When you’re using your Proxmox server you might run in the problem of not having enough HDD space defined for some of your VMs, using ZFS (pools) it’s really easy to fix this.

Since all my VM disks are simple files you can easily increase their size by running the following command:

bash


qm resize <vmid> <disk> <size>

Let’s say you have a VM with a disk named vm-100-disk-0 and you wan to increase the disk size by 100GB, you just run

bash


qm resize 100 vm-100-disk-0 +100G

and you’re halfway there!

Now you need to tell the VM to use the newly added space, first you need to check if the system sees the new space by running

bash


dmesg | grep sda

Now we need the right partition to add the space to, run the following to check which partitions there are

bash


fdisk -l /dev/sda | grep ^/dev

Remember the number /dev/sda3 for example would be 3.
(Just make sure you’re grepping the right device, vda is also a possibility.)

Now run pared with the right device

bash


parted /dev/sda

It will ask you to fix the unused space, enter F for fix, then run

bash


resizepart 3 100%

This will resize partition (sda)3 to use 100% of the newly added space.

That’s it, no need to reboot!

You can find a more detailed description at the source: https://pve.proxmox.com/wiki/Resize_disks

Nextcloud

Wed, 20 Jan 2021 10:23:09 +0100

What is Nextcloud

Nextcloud is a complete software package to replace most cloud services like Google, iCloud, Dropbox etc. Nextcloud is branched off from the other solution ownCloud because (from what I’ve heard) a couple of devs did not like the direction it was going so they started their own product.

Nextcloud is not just a storage solution, it’s a complete package, it’s not blazingly fast and it’s written in PHP. But it does offer great integration with other services and has tons of plugins, there are plugins for monitoring your Nextcloud instance itself or third party services. It also supports contacts, calendars, Kanban boards, and even complete chat integrations with video.

Why Nextcloud

I mainly use Nextcloud as a Google Drive/Dropbox replacement, but the expandability is what really got me. I’m pretty sure I want more functionality in the future so using a different service like Seafile is not what I’m looking for in the long run, plus Nextcloud being completely free to host yourself without limitations is great.

A simple Samba/WebDAV was not going to cut it either, having a nice web interface is important since having to install some app to use it sucks and sometimes I work on public systems and having to connect to SMB/WebDAV is just not doable while opening a webpage is easy.

It also has apps, like for iOS, I recently moved all my photos from iCloud to my Nextcloud instance and with the auto upload feature I won’t have to worry about missing any photo’s or videos missing from my Nextcloud environment.

Just owning your own data is great, I’m 99% sure Apple won’t do any crazy things with my data, but you never know. Plus I don’t need to pay a montly fee to get more storage, a HDD is really cheap these days and already having a server certainly did help.

How to set it up

You can set up Nextcloud in multiple ways, I went with the manual one because I want to squeeze/optimize every little thing to save resources on my hardware.

I basically followed the documentation for installation on Linux here: https://docs.nextcloud.com/server/20/admin_manual/installation/source_installation.html I installed every required PHP module (and PHP itself) together with MariaDB, Redis, and SMB.

It’s recommended to install imagick, apcu, memcached, and redis module for PHP so you don’t run into any performance issues down the road. You could also always install these at a later moment when you DO run into these performance issues ofcourse.

Instead of following the Apache configuration I followed the Nginx one: https://docs.nextcloud.com/server/20/admin_manual/installation/nginx.html just because I’m more familiar with Nginx and I personally think it’s less resource hungry and faster. I changed some default things like client_max_body_size to 10G and ofcourse the server_name.

Instead of using SQLite I use MariaDB, it’s much faster and stable for long term use, Nextcloud also only recommends SQLite for development environments.

Optimization

Make sure to read https://docs.nextcloud.com/server/20/admin_manual/installation/server_tuning.html a lot of points here lead to a smooth server, I still need to optimize some things like PHP-FPM but at least make sure you’re using MariaDB/MySQL and Redis caching. These two will speed up things by a lot.

Next check out https://docs.nextcloud.com/server/20/admin_manual/configuration_server/caching_configuration.html it will explain/suggest the best caching settings for your Nextcloud instance. Remember PHP runs best when properly cached/optimized. Just running PHP and PHP-FPM with default settings is just OK, and that’s not good enough for most in production.

At last, if you’re seriously thinking about using Nextcloud as your main cloud storage/Dropbox replacement make sure you check out https://docs.nextcloud.com/server/20/admin_manual/installation/harden_server.html and be sure to run over HTTPS.

Oh and run in RAID1 or ZFS Mirror, I advise using ZFS even though it uses more RAM it’s worth it. Being able to have a pool you can modify however you want and having an abstract layer around your hardware is great and (in theory) I won’t lose any data if any (1) HDD fails in a mirror. Just don’t run striped RAIDs even when the 80% usable capacity is so tempting.

PS. redundancy isn’t a backup, make sure you have one. :)

PPS. this runs in a VM with 3GB of RAM and an intel i5 10500.

Proxmox Tip

Mon, 18 Jan 2021 17:51:06 +0100

When you’re using Proxmox for free without any subscription you may see

bash


TASK ERROR: command 'apt-get update' failed: exit code 100

The issue here is that by default Proxmox is using subscription only repos, you can remove this “error” by either getting a subscription (duh), or change the repo to pve-no-subscription. This repo is not as stable as the subscription one they claim, but for a home server it should be enough. If you really need the stability you’re probably running a critial environment which probably is worth paying for the subscription.

To change the repo edit your /etc/apt/sources.list and set the repo to pve-no-subscription.

bash


deb http://ftp.debian.org/debian buster main contrib
deb http://ftp.debian.org/debian buster-updates main contrib
# PVE pve-no-subscription repository provided by proxmox.com,
# NOT recommended for production use
deb http://download.proxmox.com/debian/pve buster pve-no-subscription
# security updates
deb http://security.debian.org/debian-security buster/updates main contrib

Source: https://pve.proxmox.com/wiki/Package_Repositories#sysadmin_no_subscription_repo