TrueNAS SCALE + Docker + Kubernetes - how to do it?

TrueNAS CORE and Jail - what is it all about?

Originally, the idea used TrueNAS CORE to add applications, functionalities were so-called Plugins. You can install Nextcloud, Plex Media server, and quite a few other Plugins this way. Anyway, if someone does not find what he needs, he can always run an empty Jail and install inside what he needs there. Jail is based on the fact that you fire up a "tethered" application, limited to a specific directory on disk. It has access to the same RAM to the same processor to the same system kernel. They even share disk space although, as I mentioned, Jail is trapped in its own directory and does not have access to the entirety of the host disk. The separation of software and system seems to provide security.

 

The undoubted advantage of the solution is its simplicity and very low resource overhead. Unlike classic virtualization, each process inside Jail is like any other process running and visible on the host system. Of course, the host processes are not seen by Jail.

 

And what was the harm in letting it be as it was, one might ask?

 

The disadvantage that can really hinder the free use of this solution is the fact that Jail-e also share with the host the kernel of the system they use. This makes it necessary for all applications in Jail-e to be compatible with the host system. TrueNAS CORE is based on FreeBSD and Jail-e running on it must only be compatible with FreeBSD but also generally with its particular version. That is, an application update may not be possible without a system update, and a system update often requires updating the Jail-i or just sometimes ...breaks them.

Docker

The solution envied in TrueNAS CORE is Docker. It solves the ills associated with Jail. Docker allows you to build a minimal operating system containing only the elements needed to support the application for which it is created. Importantly, it also contains all the dependencies, its own application-specific versions of libraries and everything needed to run it. It is decoupled from the host system solving the problem of incompatible software versions of libraries, etc. Not only do the system versions not have to be compatible but the operating systems can be different.

 

The entire operating system with HTTP server or database takes tens to hundreds of MB max. This is a huge difference compared to a virtual server

 

Such a container is so universal that we can freely transfer it to any Docker-compatible computer and fire it up there without worrying about incompatibility.

TrueNAS SCALE docker - and what's the deal?

Once we've made a general prediction about why Jail isn't cool for several reasons and Docker is, let's get back to TrueNAS. In theory, the simplest thing would be to simply deploy Docker to TrueNAS. It's just that, as I mentioned TrueNAS CORE is on FreeBSD and despite the advantages of this system, unfortunately, extremely simplifying.... FreeBSD doesn't know how to Docker.

 

That's why, among other things, the TrueNAS SCALE branch was fired up. The main technical difference is that it is based on Debian. Debian and Ubuntu based on it, on the other hand, play great with Docker.

 

Also digressing, an important element that differentiates CORE and SCALE is the ability to scale the file system by combining a minimum of three nodes into a cluster, which is supposed to make the cluster not only resilient to single-node failures, but also faster.

 

Kubernetes

Today's complex and heavily loaded services are most often broken up into so-called micro-services. This logically separates e.g. databases from HTTP servers, cache servers or load-balancers etc. This makes that by firing multiple instances of such a micro HTTP service on several servers, we enable almost instant scalability and at the same time increase resistance to failure of one of the servers.

 

Everything beautiful but... it turns out that in practice such containers for one application per startup are often several. In addition, as part of redundancy, this would have to be replicated to other nodes. Fire additional containers as needed... it gets busy.

 

Orchestrators come to the rescue here. For example, SWARM, docker-compose or Kubernetes, just used in TrueNAS.

 

He is the one in the background who takes care of maintaining the right number of containers, creating internal networks for communication between containers, redirecting queries to the right containers, allocating disk resources and much more that we don't have to think about. At least ne the beginning how everything works.

 

Of the currently visible at first glance disadvantages of this solution, we can mention the much greater complexity of the solution in hijacking from Jail. The second disadvantage but no longer the containerization itself but the TrueNAS version release is the lack of redundancy. At least in the native version. The point is that containers are often duplicated across several physical servers for reasons of both reliability and load scalability. That's not how it would work in this case. Probably it could be solved somehow with more or less interference with the guts of the system, like most things in open source anyway, but that's not the point here.

 

Lab

I used TrueNAS SCALE 22.12 Bluefin for the presentation. As of May 2023, it was necessary to manually change the line to Bluefin because the earlier version had trouble updating applications from outside the native IXSystems list. In addition, I have to admit that just as at the beginning you could feel the underdevelopment of SCALE from the user interface side, it looks more and more sensible and nice from release to release. I would also add that the lab is a virtual machine with NVMe disks because containers and especially initializing them can be very hard on the disks for the disks as well.

 

Summary

This is just the basis of kobernetes or containers in general is a mega-complex topic but I encourage you to have fun.

 

Would I put production containers with critical data on it. Well, not likely. Would I plant containers on it for personal use? Yes, I think it's a great starting point to start playing with Docker/kubernetes.

 

 

If you would like to learn more about TrueNAS write to us. We will tell you how it works and why it is worth it?