tuber thoughts

There was a teeny 15 second blackout. Afterwards, the power was just all wrong all over the house. While using the microwave, after a few seconds the power would dip drastically and the lights would dim. I thought "omg the microwave is dead already?!" its not even a year! I thought it was shorting or something and causing such a massive draw and everything in the house dipped. We later use the coffee maker and it cause a similar problem with the lights. After some testing I found that some outlets were sitting at 106 volts while others were at 136 volts. WHAT! When I turned the microwave on. the voltage dips down to 93volts!!! I started shutting off breakers and I tested about 4 individual circuits before determining that it was the whole house and therefore something wrong from the power company.

To keep it short here is a few noteworthy things. Here is being serviced by SoCal Edison. Don't bother with "partial outage". Just say it is a full outage. Otherwise you just keep getting a bot and they just assume that it is a tripped breaker. They also will just keep testing the meter thinking "well meter gets power therefore no problem!" (oh but the neutral doesn't actually go to the meter sooo...?) Don't bother with voltage issue as they... i don't know since the only way to report that is via the website which usually doesn't work. They definitely outsource according to The Wise One as he heard accents every time he called and he kept getting hung up on whenever he asked for a supervisor. (6+ times)

The verdict is a "Open Neutral" as in the reference 0 voltage from each 110v leg was gone. That means it was floating 0 reference and effectively just ground. Ground and Neutral/Return is tied together at the breaker panel. That means the "return power" was just going through the earth. Now this area is actually dry and a little rocky so earth isn't really that terrific of a conductor anyways.

The nearest distrubution box was missing LOL! They had to dig it up. And in the end, something about the neutral "being disconnected" OK. I would never know since I wasn't home at the time. Here are the pictures of my neighbor's yard.

I've begun descending into this madness of containerization and virtualization. I did a tiny bit of it at work for a test machine and omg was it a HUGE help. This is all because of LXC containers. "oops... uhhh... screw it" kill the container and start over which only takes about 30seconds to 1 minute. The resource overhead of a LXC container is so small compared to a VM. < insert real metrics here: container, VM, bare metal>

Yes baremetal is a point of comparison. I listened to a video from either Level1Tech or TechTechPotato, i forget who, talking to a VMware person. He mentioned that the VMs in some cases would perform better than baremetal because windows no longer had to deal with drivers because the hypervisor did everything instead. I believe it. The VM would not have to deal with waiting for hardware to initialize. There are lots of little other things that add up. Of course I'm not dealing with 100-core servers and actual server motherboards that actually take a minutes just to get to the bootloader to just start up but you get the picture. For VMs vs Containers, I'm still learning. Containers have much less overhead. They simulate the conceptual environment instead of simulating the hardware. For VMs you can pass hardware along to the VM using things like IOMMU and SR-IOV. Concepts I am aware of but haven't fully delved into yet.

The mental exercise of migrating some of the servers between nodes is making me realize what the purpose of a Storage Area Network / SAN is for now. LTT said recently in a video about how a SAN is a role, not a piece of hardware. That comment made me more perceptive of it and I'm seeing why it exists now. or at least I think I know what it exists now.

For example, take hypervisor A and B. These are badass servers with lots of cores and ram and storage. They are working together in a cluster. Virtual machine, VM, is working on A but something happens and A goes down. If it was replicated across B as well then B can continue the work. The problem is that this means you have two copies of your VM sitting on systems. One on A and one on B. If you add a third system, SAN, that only handles storage, then your VM can reside on that computer instead. Or maybe instead of having the whole VM on the SAN, you put the large dataset your VM was working with.

A applicable example for me is Jellyfin. I can put all of the media on the SAN, then that means I can move the Jellyfin container between my two servers. Lets say that I don't need very much horsepower at the moment so I shut the second server off entirely and move all services to one server to save power. Cool! Of course that doesn't work in my case as I would need 3 servers then. The SAN would just be the same as a NAS + networking fun.

Here is my bigger challenge. Minimal downtime while I learn how to migrate all services to being hypervised, containers, etc. Starting original layout:

• Ubuntuserver
  • Mumble - High Priority, daily affected avg: 6 users
  • Web
  • Jellyfin - Medium Priority, daily affect avg: 3users
  • Nextcloud
  • various games
  • openvpn client
  • QBittorrent
• Rosie
  • Zoneminder (maybe Frigate later)
  • Home assistant
  • zwave-js-ui

Hardware changes: Adding 1tb NVME drive to each server. Adding ECC ram to 64GB.

That's it for now. will need a seperate post (:

list of links to not lose

• https://gist.github.com/crundberg/a77b22de856e92a7e14c81f40e7a74bd
• https://forum.proxmox.com/threads/usb-passthrough-to-a-container-lxc.101741/
• https://medium.com/@konpat/usb-passthrough-to-an-lxc-proxmox-15482674f11d

outline:

• trying to get ecc cuz price and stuff back in november 2022
• didn't work for ubuntuserver, no boot, but works for drunkmary, boots
• ecc works for drunkmary. turns out (i discover july 2023 why...) actually it doesn't but it at least boots and can use it. just wont do the actual ECC cuz ryzen G
• come july 2023 - tries again for ECC ram cuz its EVEN cheaper. ubuntuserver still doesn't work after even more bios updates. but this time i try put the sticks in rosie and it works!
• tried a different mobo, replacing a Asrock x570 Phantom Gaming 4 with Asrock x570 Pro 4 because this is what Rosie. doesn't work! ):
• suspect lemon cpu in ubuntuserver (3600xt) vs rosie (3700x). I buy a 3700x off ebay because that would be helpful for live migrations via proxmox in the future. Do i need live migration? no. but it would be awesome and fun to learn! :D
• found some lines to actually report if multibit ECC is working in linux. Will have to paste here.
• Bonus confirmation: I gave RubberDuck the x570 PG4 to swap on his personal rig to troubleshoot suspect mobo failure. After the swap i tested the ECC memory in his just for kicks (3600x). it boots fine! This is more evidence that my 3600xt is a lemon for ecc.

LOL so backlogged, so slow, much stuff inbetween. Alot has occured since Part 2 back in Sept 2022. I had to do a more unified simple solution for the Wise One (dad's meme nickname)

mostly updates on progress.

outline:

• lel ended up getting the topton one with a newer gen cpu. decided i want opnsense and the 6x2.5 gbE was too enticing.
• https://mikrotik.com/product/hap_ac3 for dad
• openwrt was trouble but ended up ok
• VLANs DO NOT WORK, probably because SORTA beta, not full release
• expected to work now since the full release is out and 2023 openwrt kernel i think has built in dsa support now too
• started working installing conduit at goldwater
• found the bananapi 3
• flawless
• struggling with opnsense and openwrt on the topton unit. in regards to vlans... probably not the hardwares fault but my understanding of bridges + vlans

spent all day with this issue at work. dns entries was 10.10.1.2 (ns2) and 10.10.1.1 (ns1)

windows apparently DOESNT fall down to the next server if it doesnt work. THANKS WINDOWS (':

flipping the dns entries (and a ipconfig /flushdns on one system) fixed it.

https://community.spiceworks.com/topic/292367-why-wouldn-t-a-computer-query-the-2nd-dns-server-if-the-primary-was-off-line

PS C:WindowsSystem32> nslookup ad ns1
Server:  ns1
Address:  10.10.1.1

Non-authoritative answer:
Name:    ad
Addresses:  10.10.2.104
          ...
          ...
          ...
          ...
          ...
          ...

PS C:WindowsSystem32> nslookup ad ns2
Server:  ns2
Address:  10.10.1.2

*** ns2. can't find ad: Server failed