Two Years

Alot has changed since I last posted here,

Back then I was running the server on a 4 core with 8GB ram, the network relied on a SOHO off the shelf router.

Since then my network has gone threw a number of changes:

The main server was upgraded to a Dual CPU 6 core Xeon with 128 GB ram, I made the switch last year around this time to ZFS as my primary storage file system and wont ever look back at mdadm.

I had went through and made ‘compile cluster’ of three Gen 1 Raspberry pis running Gentoo, these ran relatively well but I never found a use for them outside of running AI battles of freeciv.

The above compile cluster was recently returned from the dead with a new batch of Gen 3 Pis all running 64 bit kernels. Two sit headless on a shelf operating my radius, dns and local ntp services, the third is my current workstation, and their shared portage repo. I also run Gentoo in a vm on the server ready to cross compile as needed.

The SOHO router started to over heat and shutdown, this was in the middle of a week I was working from home, I had just recently gotten the new server online and still had the ‘bones’ of the old one, Since then I have ran a few roll your own router distros:

4 core 8GB:
IPcop
*pfsense wouldn’t boot
IPFire, longest and most stable for this run ran for more then a year without problems.

I also tested switching From my tried and true KVM to ESX as school would provide it for a year. After a month I changed back, ESX seemed slow on my hardware for all my Linux machines but seemed much more optimized for windows, to bad I don’t run windows.. I did how ever get Mac OSX to install.

Around six months or so ago, my wife’s father happened on a rack system that was getting tossed, the gentleman who ran it passed away and his wife knew nothing about it and just wanted it gone, he asked if I wanted it.

From that I acquired:
42U Rack
Powerconnect 5448 48+4 port switch
Supermicro ‘NAS’ server 4 core Xeon with 8GB ram (Upgraded to 32GB) with 16 2TB drives
i5 with 16GB ram

Dual Core AMD, cant remember the ram as I haven’t used it.

Since then I racked My Dual Xeon and the router, after having issues with the 4 core router, I decided to upgrade to the i5 For a router. as When I went to the ‘roll your own’ I started down the squid(av,cache), snort, etc.. route and it eats alot of ram if you let it.

i5 16GB:
IPfire again long run until a friend at work showed me..
PFsense, would run for a few weeks then suddenly loose wan/route and i couldn’t get it back without re installing this lead to going back to.
IPFire, around this time 114 was out and both the installer and upgrader had issues I decided to check out
OPNsense and have run it ever since.

Everything racked from top down:

Switch
Dual Xeon
Router
SPACE
Supermicro Nas (on the bottom)

Network topology changed aswell, given now with the ‘roll your own’ distros I can have completely separate networks I have and mainly for now kept with the IPFire schema:

LAN, for everything Wired
Wireless, Selfexplanitory
L I have three access points in the house now, the original 1750AC a gifted 1900AC
both running as access points WPA-Enterprise, using the PIs radius for auth, then
an old 600n for legacy WPA2 with a direct line to the internet and nothing else.
Excom, Servers and such

LAN can access anything and any of the three networks, but both Wireless and Excom(unicated) can only talk to the internet and themselves.

For now I will leave this as my update.

Love Virtual Machines and Hypervisors

Ello,

As anyone who checked recently knows I had a HDD failure where My servers Operating system Hard Drive Failed, The Bios would not even recognize the drive being plugged in, So I figured i would be down for a while and set up a small web server in one of my Raspberry Pi’s.

What I did not know was how cheap mechanical Hard drives had become over the last year or so, The boot drive was a 500GB and well within warranty, so I had already planned to send it back to get it repaired/replaced, on a off trip to a local computer store I checked the price for another drive($64 and change) in the mean time and bought it on sight.

Seeing as with most of my recent servers I have switched to running different services in separate VMs I.E.: This Webserver in one, The IRC server in another, etc…

So Once I had the server back up bringing the services up was cake, just load each VM from the most current backup and were back where we were.

Feels good to have a decent back up stratagy.

Jace

Branching(Bridging?) the Server

Well another sort or accomplishment under my belt, I have successfully added a third nice to Server. Set it up for bridging, after trial and error, a lot of aggravations, and finally bucking and asking for help.

I now have a working topology as such:

Nic 1 — LAN 1 — Host only < for the underlying host, media server and storage system. Nic 2 -- LAN 1 -- br0 KVM set one, original servers including this web server. Nic 3 -- LAN 2 -- br1 KVM set two, new servers on separate wan connection. All working so far after a few hiccups. Love this stuff! EOL

Cron’d Back up!

Well this Weeks mini project was to get a automated backup of all my Virtual machines on my KVM host, the first hurdle was to shut down the running VMs this is completed by executing virsh shutdown x where x is the domain number of the VM, I normally never have more then five machines running so I started the script with

 

#!/bin/bash

virsh shutdown 1

virsh shutdown 2

virsh shutdown 3

virsh shutdown 4

virsh shutdown 5

 

after this the VMs shutdown now I figured Compression would be key as most of these VMs have forty gigabyte hard drives if not bigger but normally only use about half.

 

Originally I started with tar.bz2 with the following command

sudo tar jcvf VM-Backup.tar.bz2 dir_1 dir_2

Sadly this took forever!! upon a little research it was due to the fact that bzip2 isn’t SMP compliant otherwise known as using multiple cores or CPUs

so next I tried Gzip with this command

sudo tar zcf VM-Backup.tar.gz dir_1 dir_2

To the same effect so I started to invest time in google until i found a SMP compatible BZIP2 here: http://compression.ca/pbzip2/ and was finally happy with the results with this command

sudo tar cfv VM-Backup.tar.bz2 –use-compress-prog=pbzip2 dir_1 dir_2

This used all cores available and maxed them during compression, for being a quad core system it cut the compression time roughly down to a third of what it was.

Now that things are starting to look up I decided I wanted to Log when the script ran and tag the output files with the date and time of Backup this was done by making a Variable for date

DATE=$(date +%m%d%Y-%H%M)

at this point what ever you put $DATE in it will output MMDDYYYY-HHmm so I wrote a few echo lines like this one

echo “Nightly Backup Started for $DATE” > /location/of/log

After all was said and done it was brought to my attention a question, “How do you check your backups for data integrity?” so I tagged on the end of all the compression scripts an md5 hash output

&& md5sum * > VM-Backup-$DATE.MD5 

you may notice the $DATE again I wanted to make sure the same MD5 matched the Backup file, I then thought about the space this is going to take up and figured one backup being kept is good enough to each time this script is ran it changes directory into the back up directory and deletes all files with

cd /location/of/backup/

rm *

and finished off with a reboot as I plan to run this weekly, this is my finished script changed for system anonymity

#!/bin/bash

DATE=$(date +%m%d%Y-%H%M)

#echo $DATE

echo “Nightly Backup Started for $DATE” > /location/of/log

cd /location/of/backup/

rm *

virsh shutdown 1
virsh shutdown 2
virsh shutdown 3
virsh shutdown 4
virsh shutdown 5

#sudo tar jcvf VM-Backup-$DATE.tar.bz2 dir1 dir2 && md5sum * > VM-Backup-$DATE.MD5

#sudo tar zcf VM-Backup-$DATE.tar.gz dir1 dir2 && md5sum * > VM-Backup-$DATE.MD5

sudo tar cfv VM-Backup-$DATE.tar.bz2 –use-compress-prog=pbzip2 dir1 dir2 && md5sum * > VM-Backup-$DATE.MD5

DATETWO=$(date +%m%d%Y-%H%M)

echo “Nightly Backup Complete for $DATETWO” > /location/of/log

 

sudo reboot

 

 

Enjoy and leave me tips if you have suggestions or kudos