I lost a raspberry pi for a couple months. Just couldn’t find the darn thing. It was working fine serving away.

Turns out it was in the basement. I forgot I moved it to be closer to my tilt Hydrometer.

To be fair, those things are really tiny!

Oh, good thing you wrote this. I’m about to give one of my printers away and there’s still a pi inside

Cups?

OctoPrint (3d printer)

Memory unlock!

YEAH it’s bash.org

Ooo, another bash mirror. Nice!

For a while, the windows machine I was using kept randomly becoming unresponsive (both via network, as well as the keyboard+mouse) and I just could not figure out why. Everytime it did, I’d have to force a shutdown by holding the power button until it shutoff, then press it again to start back up. Sometimes this happened while I wasn’t home, so it had to just stay offline until I was.

I got sick of having to do this; so I replaced the power button with a transistor and a RPI. The PI would ping the server every 5min; if it failed to get a response 3 times in a row, it would trigger the transistor for 10sec, release for 3sec, then press and release again; forcing a poweroff then starting the machine again. It’d also write these events to a log file. I think it was around twice a week ish.

Never did figure out what the cause of the lockup was; but it stopped when I replaced windows with Debian.

Had a zyxel nas that got unresponsive after 10-15days of uptime, thankfully it hat a scheduled shutdown/reboot setting. Runs since 2014 with a reboot every Mondy 3am.

Heh, my router is on one of those timer plug things. No matter how frozen, it will reboot.

I’ve got an extra and don’t know what to do with it. I feel like its bad to pull power unexpectedly once a week though. Especially with a single HDD storage setup. I’d like to setup a RAID config, but its a 28TB drive and additional drives would cost me about $$$$$$$.

I have a NAS that had a similar problem, every week or two it would stop responding. From the logs it was clear that the machine was running fine after going non-responsive, but either the NIC or something else network related was crashing. However, the logs weren’t capturing the exact issue. I wrote a script to ping Google every 10 minutes, write a bunch of debug data to disc if it failed, and then reboot.

And it has never happened in the year since. I have no idea what the root cause was, but sending those pings out seems to have fixed it.

Never did figure out what the cause of the lockup was

the mysteries of closed-source software

I also had a machine that would randomly crash on Windows, but not on Linux.

I’m pretty sure it was some sort of hardware failure, as it also sometimes failed to boot Linux. But if it booted Linux, it would be stable until I shut it down.

Sometimes Linux handles faults better than Windows

Typically dmesg shows error when the hardware is acting up

Using an RPi with a transistor almost seems professional… ITAPPMONROBOT

Note to any future lemmings who find this thread troubleshooting the same symptom…

1. Rule out unswitched (non-chipset, direct CPU lanes) PCI connectivity and power issues by testing without specific PCI components. Unstable PCI errors can be missed during post and present later after boot, and Windows is especially bad at handling it transparently. Common culprits include GPU sag, dubious riser extensions, and anemic PSUs 2. If there is a “high bandwidth” feature toggle for RAM in your BIOS settings that is currently on, try turning it off 3. If your system drive is an NVMe blade with a Phizon controller, check if the manufacturer software has a firmware update available. 4. If the freeze is regular but not persistent, try turning the polling rate down on your mouse, and if it’s wired, try to make sure it’s plugged into one of the board’s SOC-hosted USB 2.0 ports

Yeah getting rid of windows solves many issues, LOL. I have always refused to run any server with windows, outside of being forced to at work. 😁

I just wrote a comment about a very similar setup! Glad I wasn’t the only one haha

Had a problem where my ports would suddenly close. At first I thought it was Telekom, then the modem, then the NAS.

After much investigation I discovered that restarting my router would solve it.

After some more weeks I discovered that the crawling bots were triggering some stateful tplink firewall that would close everything until rebooted.

So the router in the middle of the setup was the problem, not the edges.

Well at home i only have problems with spiders in my racks, not like the racks i have seen at work. Highlights are toiletpaper, chipmunks+nuts and worst in a school server rack a gun.

The gun is for the printer if it tries its bullshit one more time

Nice chonky heatsink, goddam

I used this, among other things, for my personal Blender renderfarm. Cooling was a must.

haha looks like you’re aaalmost at the point where water cooling is compulsory.

Is it a Threadripper? I built a couple of those for rendering, such absolute beasts.

I honestly don’t remember now as this was over 10 years ago.

10 years!! aren’t you woried about dried up thermal paste? I think Re-pasting CPU/GPU every 2~3 years or so is a good practice to prolong longevity.

Ah, sorry for the confusion - that server’s been out of commission for a few years already.

Cat vomit… my cat hit my raspberry pi.

Sounds like a cat. The entire place to vomit and it finds the most expensive/inconvenient thing to lay that stomach lasagna on.

One day, our internet just stopped working entirely. Our ISP was fine, router good, but no outside connection and I couldn’t access local devices. I have two Unraid servers. I took a quick glance at them, they were powered on, all looked ok. I decide to power cycle everything. Still no internet. Then I realize that I can still access Google from my phone when on WiFi… hmm. That would mean… an issue with pi-hole!

I begin to try to diagnose from my phone with a hotspot to my computer. I’m getting warmer but some things just aren’t making sense. I go back and stare at the server running pi-hole. Then it fucking hits me.

I have the pretty odd and totally not ideal home network. I have plans to upgrade everything but currently I have two wifi networks tied to the same WAN. One network is run off of access points so I can get Ethernet to odd areas of the property. The Unraid server hosting pi-hole was connected to an AP and the Ethernet cable I used didn’t have a clip, so it just barely unseated itself from it, breaking the connection. The router+pi-hole config meant also breaking the internet if anything happened to pi-hole or that server. I swapped it for a better cable and it all instantly started working.

Spent that weekend sorting out the network so it doesn’t happen again. It was definitely a head-scratcher.

It’s always dns at least half the time

Pihole is cool, but fucked me over as well more than once (not the pihole itself but the host). It’s on its own hardware now, but I plan to add a second one to reduce the single point of failure.

Haven’t thought of that tbh. Would they both just share configuration? Curious how leasing would be split/shared between them. Off to research!

There’s some sync mode but I haven’t really looked into it yet.

Makes sense.

I’ve been running two piholes for a few years:

I let the router handle dhcp, while having it hand out each of the piholes IPs as DNS #1 and DNS #2. Those piholes settings are mirrored using Nebula-Sync, so any change made to one pihole gets copied over to the other automatically.

Both of those piholes are setup to wrap outbound DNS reqests in DOH as well, with the router blocking port 53 outbound. So all DNS traffic leaving the network has to be DOH/DOT or it just gets dropped. Currently using NextDNS and quad9 as my upstream DOH providers.

Note: the cloudflared method for DOH is outdated. I’m still using it via the docker image “image: cloudflare/cloudflared:2025.11.1”

I believe This (dns-crypt) is the more recent method.

Oh sweet this is super helpful! Doesn’t seem too complicated. Thanks!

Yep I’ve had it randomly stop working if it has to serve clients outside it’s subnet

I mean, would you want it to serve clients outside of your subnet? I guess maybe if you had something like a guest network. But I tend to take the “fuck it, you get basic service and nothing else” approach to things like guest networks.

Isn’t it a potential security issue to have your pihole serve things on any subnet, because it allows an improperly configured firewall to stab at your pihole? I think you can specify any additional subnets it will serve in the settings somewhere. But IIRC, the default behavior is to intentionally drop traffic from unknown subnets. But that’s just off the top of my head.

I keep one Physical Pihole and one in a container, be sure yeah, this happens.

Mine would be when I had a 48 bay disk array / JBOD fail on me… badly. After a storm, it killed the larger card that allowed for me to get many of the drives into a PCIe 16x slot, and I was relegated down to only getting ~8 disks per box made from spare hardware. A single box I got 16 going. Add to this mix an SSD for mid-line caching.

These were all running bcache on top of mdraid… One single mount.

Yes I understand how obnoxiously stupid it was to run RAID6 on a 48 disk volume. It was almost all just stuff I could re-acquire over time, not irreplaceable things.

I just HAD to solve this one though.

In come several spare chassis / mobo etc…. get a bunch of drives powered and on /dev/ , move to the next.

A couple spare gigabit switches…

several gigabit NICs…

two explicit paths for each machine…

a bit of iSCSI magic, and one machine now had the physical disks all exposed to it… mdadm –assemble blah blah, bit of UUID chaos…

It’s surprising that while a bit speed limited (I think I got just around 110MB/sec reads), it was nicely performant for what a huge mess of wires and disks just strewn out around my rack.

Managed to evacuate all I needed without much issue once I got that going. Now, I try to keep my arrays under 16 drives at a time, or keep a very rigid policy of “I can lose this and don’t care” vs “this box gets RAID10 and/or offsite backups nightly”.

back up your critical stuff people!

48 disks seems a nightmare in power consumption, failure rate, and overall cabling management…

What was the total capacity?

Power isn’t so bad really. These were all 2tb disks, so ended up with about 40tb total after shenanigans.

Cabling was atrocious. I’d originally been using several pulled 4 disk SAS backplanes from servers thrown out of my work, just screwed down to some pine boards in a stack.

Zip ties and sata/sas cables were the rule.

Overall the drives had 4 small power supplies initially, 300watt iirc. Cheap.

Somewhere I have an old photo from the wiring during the “rescue” but I have to look around.

Disk failure was actually not an issue for ages before. A specific thunderstorm with 2 nearby tornadoes is what caught me this time. Before that, the occasional single disk every 6 months or so.

I still have running arrays using 15 year old disks, many with more than 10 years of that powered on. If you find good models, they can last ages.

for me it was the fact that the desktop case i used for the server had the power button on top and it was big enough to be pushed in, when my cat walked over it…

my cat currently loves hitting the “Sleep"-button on my keyboard every time he jumps on my table -.- at least i can wake it again, but it IS annoying. Haven’t had the time yet to reconfigure that button to instead run a soundfile of me scolding my cat lol.

You don’t need a mouse for a server. Just SSH into it, duh.

Now now don’t sshush the server, it’s doing the best it can, okay? It gets hot in that box…

Well, if it could think outside the box every once in a while, that would be great y’know.

At my first-ever tech job back in 2000, I was given a little Celeron desktop computer to do my work: mostly writing some ColdFusion and cutting up images in Photoshop. For the most part, the machine worked fine, except whenever I scrolled down on a webpage in Netscape, the box’s PC speaker would start screaming. I tried to ignore it, but I was in a small room with two other nerds and it was getting really annoying.

One day, my frustrated colleague decided it was time to investigate. As part of the process, he noticed that there was very little air coming out of the power supply fan port. “Must be a busted fan and that’s causing an overheating alarm” he declared. We unplugged the box, popped it onto my desk, and opened it up. The internals were pristine. No dust, nothing. The computer was practically brand new after all.

We were just about to crack open the power supply (not recommended, but we were getting desperate) when my colleague noticed that something was wedged inside the fan… it was the biggest (dead, thankfully) cockroach I have ever seen, at least 10cm long. For context, this was in Vancouver, Canada. Those just don’t exist there.

My colleague jammed his screwdriver in there to grind up the cockroach carcass, then plugged the box back in. After a shittone of dessicated cockroach guts spewed out the back, my little computer was operating normally.

As best we could guess, the monster crawled into the fan in Malaysia when as was assembled and somehow got pinned in there, died of starvation, and then stowed away to Canada.

And this is the way bioinvasor species spread - most of the time they don’t come delivered inside of PSUs but simple crates, which improves their survivability lol

Not self hosted but at work: after an update to both our firewall cluster nodes they both acted wonky, restarted 5 min after booting, ssh connection wasn’t working half the time, dropped packages… the problem persisted through multiple reboots, support had no idea.

the fix was to pull the plug entirely, wait 10 seconds then power them on again. on both nodes we had this behavior and both worked again after losing power for a few seconds.

seems that even for business appliances “have you tried to turn it off and on again” is a viable solution.

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

Fewer Letters More Letters AP WiFi Access Point DNS Domain Name Service/System ISP Internet Service Provider NAS Network-Attached Storage NVMe Non-Volatile Memory Express interface for mass storage PCIe Peripheral Component Interconnect Express PSU Power Supply Unit RAID Redundant Array of Independent Disks for mass storage RPi Raspberry Pi brand of SBC SBC Single-Board Computer SSD Solid State Drive mass storage SSH Secure Shell for remote terminal access

11 acronyms in this thread; the most compressed thread commented on today has 4 acronyms.

[Thread #80 for this comm, first seen 18th Aug 2026, 08:10] [FAQ] [Full list] [Contact] [Source code]

Also mouse not pictures

I was frantically searching for it, then i opended the post to See if the comments found it …

I’m sorry. Have this picture of a lime hawk-moth caterpillar I saw the other day.

plantmonster.net/…/PXL_20260813_122831209.jpg

I hosted shit for about ten years before really understanding how local dns worked.

Imagine a primitive society that discovered literally every major modern scientific achievement, but somehow missed the wheel. That’s basically how my first setup went lol.

I have heard of local DNS multiple times now and been selfhosting for many years, but I’m good with my 192.168.178.39 :)

It opened the door to pihole and unbound, then dns certs, man… everything’s so nice now lol

Well hello there, fellow Fritz-Box user :D

I have a feeling there’s a point in time where most non-it self-hosters got fed up with ads and installed pihole, then learned about DNS. I have a feeling that started about five years ago.

Now it’s probably hand in hand with how necessary a pihole is.

I do not self host. I just wandered in here. Do you have suggested resources to learn more about DNS? I feel like generally searching shit sucks to get an answer like that lately.

A dns server knows what and where things are. That’s its only real job, to tell who’s asking where say, eff.org is.

“Ay homie, where’s eff.org?”

over here - 173.239.79.200

“thanks, bud!”

The issue is, the dns server you use by default is under the control of whoever your provider is, and as a result, they get all of your dns lookups like the example above as well as every porn site, every piracy site, everywhere.

Installing your own, or simply moving to one your provider has no business with like 9.9.9.9 can help keep your dns queries more private, go a step further and use encrypted dns, as many providers supply, and you’ll be even further covered.

Installing your own, however, comes with the added perk of being able to apply your own dns entries. Suddenly local addresses can have custom names without ever touching a hosts file. Neat! Use DNS certs and you’ve got local SSL!

I would go find something applicable to feed you for a dns 101, but my old resources don’t exist. I just know stuff from years of doing it and failing over and over until things work.

Thanks man

No mouse problem, but I’ve had to take measures to prevent cat-related downtime. Cats and power-button at the top is not a good combination.

When I had this problem I disconnected the power button on the motherboard.

Cant you just assign a different action to that power button instead of messing with it internally?

Connect to spray bottle aimed at… well nope.

You can for a click, but no way for the long press, as that’s handled on a lower level

You got a point. Cats are sneaky 😅

yeah what if cat does a sudo shutdown -r now then we’re really in trouble

Taht’s too, but that would still be better than biting the cables.

Did you reconnect it everytime you had to power it on?

Nah just jam a screwdriver in there to jump the pins.

You shut down your PC though?

On one computer I had a recessed reset button, so I used that as power.

Another computer I added a wire to an external button double stick taped to the back of the monitor so I can turn on both at the same time.

You sound like you are one step short of wiring up a giant lever to dramatically pull down when you want to power on the computer.

I would like and subscribe

Throw the second switch!

A lot of newer motherboards have a separate power button next to the case pinouts.

Instead of cat proofing, you were supposed to get a server room dog…

I have two server room dogs. An English pit, which is my seizure dog, and a Jack Russel. They shed, especially in the summer months, and the floor sweeper gets a daily work out, as well as keeping it out of equipment enclosures. It’s not that I hate cats, they are nice as a pet in your house, just not mine. LOL

Aha, I disabled the LEDs by unplugging them from the motherboard, you could do the same with the power sw plugs

They make sprung flip-top covers specifically for this reason. It’s just a little adhesive square/circle that you stick over the power/reset button, and it has a flip-top that automatically springs closed again.

Oh, my housemates’ cat hits the front power button when he leaps off my tower. I’ve got some leather taped over it. My uptime approaches 1! I love your little weight setup, too!

My radiator is on top so they’ve learned that’s about the only surface in the house where they’re not allowed to sleep

Meowy Guard. en.wiktionary.org/wiki/molly-guard

I have a small piece of acrylic sheet taped over the power and reset buttons on my server for that reason. At least I don’t have to worry about mice.

I’ve got Pfsense installed on a device with 6 RJ45 ports. 0 is wan, 1 is lan0, 2 is lan1, etc up to 4. Why I have multiple lans is somewhat tangential.

One day it rebooted due to a power outage and WAN worked but lan1 and 2 and 3+ were messed up. I’m still not entirely sure what happened, but the second physical Ethernet port was non responsive at boot and Pfsense just assigned the ports as if it never existed. I moved all the cables over one except LAN and it worked just fine. Except that of course in the troubleshooting i had moved all the assignments around and broke it thoroughly, so it took a while.

Then you tell people about it on IRC and become known as “the dead mouse guy”.

But the maximum nickname length on the IRC server is limited, so you settle for “deadmau5” as your nick.

Not selfhosting, but I remember standing in a server room with a hands’ width of water on the ground (and rising), and hastily shutting down servers where the water was already in the case. It were big towers (Compaq Proliant, if anyone remembers), and luckily the board, drives, and PSU were in the top part of the big tower cases, the drive bays at the bottom were empty.

We got all equipment out safely, and we did not get electrocuted, either. We left with the last equipment when the water level was approaching the knees.

Did you try turning the faucet off first to stop the water from rising while you were working?

The “faucet” was water literally streaming in through the broken window during a thunderstorm. One of my coworkers went outside and shielded the window with a wooden board to reduce the flow.

Reminds me of a time at my old job. It was a small office and this was before “the cloud” was a buzzword, so we had a little server room running a few boxes for AD, Exchange, and the thick end of our thin clients.

This “server room” was basically a large closet behind my office, retrofitted with its own AC.

We had residential units on the higher floors of our building. Residents occasionally thought to themselves “you know what makes a great hanging rack? Sprinkler pipes!” Hijinks ensued.

That was a fun call to get at 10PM.

Shoutout to the facilities guy who immediately thought to run a makeshift tarp over the server rack, fashioned from trash bags. No permanent damage.

My server once randomly kernel panicked once (and only once) and it took me an embarrassingly long time to figure out that a ram stick (too poor for ECC) had gone bad. Also took all my proxmox backup server backups with it because the validation would always fail (also due to the ram stick). Still worked fine otherwise lol

I thought that heatsink and fan was a spindle of CDs! I thought that was an interesting place to store them I hadn’t seen before…

Fail2ban crashing on 3 seperate days i tried setting it up then one day i did the exact same thing an it worked

One time when I was 10 my PS2 wouldn’t work so I threatened to punch it if it didn’t work again. It didn’t work so I punched it. Then it worked.

Loving the character development arc.

lesson learned, apparently 🤣

Ahh yes, the “Impact Fix” works everytime til it doesn’t. Had a monitor with a loose wire to the power button. For almost a year a forceful tap in the correct spot would make it work, til one day

Percussive maintenance

Fitting a NAS on top of my cupboard. Been reconsidering putting it there though, may hide it under a table somewhere instead.

I should note that I haven’t built the nas yet because of these space constraints.

Goddamn I hate mice. I just had to tear out the entire contents of a combine cab to get rid of their piss and shit, and fix the wiring they chewed off.

I had to modify the wiring in a - fairly new - tractor once.

I opened up an inspection panel and found a pair of pliers in there, used them to fix the wiring and took them out.

I gained a tool and fixed an annoying rattle too.

You wouldn’t believe how many rusty old vicegrips I own because of this sort of thing. Hell, I held a seeder meterbox on with vicegrips this spring, I’m one to talk. Couldn’t get a new holder clip for it, and the meterbox was cracked so I ordered a new one and limped by with the vicegrips until we were done.

The mouse did not survive interrogation. Also mouse not pictured.

I thought it was the huge heat sink that looks like it extends past the side plane of the cover. Reminds me of something I did with a CoolAir Cosmos tower. I mounted two 300 cfm fans over each CPU and took the side panel, neatly cut out the area around the fans, then 3d printed a scoop, with a slot for filtration, to fit on the side panel and allow the fans to breathe.

I do not care to admit how many times when I was first getting into home labbing that I could not figure out why I could not reach a pi or the internet. All because I either killed an SD card for pihole or just plain out forgot to put one in!

I took my SD card out of the rpi to reinstall home assistant. Terribly annoyed because I had no backup of my configuration.

When reinstalling my SD I found out it was the ethernet cable that wasn’t plugged in.

I am so afraid of an SD card dying and killing my HA so I have a HD setup. I have one good SD card of the 10’s of cards I have killed and it is now running my ssh honey pot on a pi 0 2.

Yeah, that was the last iteration of the setup. I had two SD’s break before I figured out they aren’t suited for running high read/write systems. Lessons learned during homelabbing.

I should really make a dd image of my card for the Pi I use for all my switch serial connections.

I dunno if that counts or if it’s the weirdest cause I rarely have any issue, but one day, I wanted to deploy speedtest tracker via docker on my 1st NAS..

So I tried, but it didn’t work (I forgot the reason). So I try to check the logs, but it turns out my NAS was insanely slow.

When I rebooted it, it couldn’t stop beeping and telling me there was a hardware problem, so I tried everything I could do to fix it, but nothing worked, so I ended up reinstalling the whole OS

Keep in mind this was the cost to see I had a bad result at the speedtest lmao 😭

Btw, I also lost all my important files, yayyy!

Not exactly self-hosting, but I’ve worked at two different academic labs that had unique issues and small budgets.

The first was on top of a mountain at 3km elevation. The low air pressure meant that all hard drives had a lifespan of no more than a year before starting to have issues. This was in the pre-SSD days. The “solution” was to boot from a floppy, never compress data, and have a student return the drives to a low elevation and swap them for the prior ones every six months.

The second was at a place that was using old Datacube MaxVideo hardware that we would never have the budget to replace. It would crash if the room temperature got above about 18C, so we set the thermostat to 15C. Which doesn’t sound too bad until you’ve tried sitting at a desk for 8 hours where everything is 15C and the heat is just slowly leaving your body into the chair, the desk, and the keyboard. And then you walk outside where it is 32C and instantly start sweating because you’re still in several layers and fingerless gloves.

I had a screen get burn in (forgot to turn it off, oops) removed it from the laptop, waited 3 days, put it back in and it was flawless again.

I had a laptop that refused ethernet and would only ever use wifi… When the ethernet was connected. No wifi unless that cord was connected. Confirmed it was, indeed, wifi by using a dummy connector.

Bought a 10tb hdd that refused to read anything at all, opened it for giggles/curiosity and found a piece of scotch tape stuck to one of the disks and one of 4 disks completely shattered. The disk was still factory sealed (by appearance, anyways; they still had the antitamper stickers)

Optical or roller ball?

I have a strong suspicion that the answer is mammilion

That sounds like a lot. Is it more than a bajillion, or less?

Many cells

Back in the day of BBS’ my buddy had a wonky modem. If I picked up phone , pursed my lips and did a squeaky sound the modem worked and I could log into BBS.

mouse not pictured

Got my hopes up for nothing!

It’s OK I gotchu

Fun fact - he’s called that because this same thing happened to him with a dead mouse in the pc case.

Na, he was the mouse. There was so much RGB, this is what he became.

I stared at that picture like it was a Where’s Wally for like 10 minutes before I came to check the comments and saw this note in OP’s message…

We are cut of the same cloth.

I got a little fanless atom PC from work that was being tossed because it “didn’t work.” It did work you would just have to apply pressure to it before boot because the CPU must have been coming unsoldered. I just stuck it on the bottom of like 30 pounds of gear and never had a problem.

How on earth did you diagnose that one?

There was an era where the BGA chips giving up the ghost was a common enough occurrence and pushing on things was a relatively normal troubleshooting step

I had a 4 disk raidz1 array of four identical drives, and one of them was having problems so I pulled it. I was able to resilver to learn more about the drive specs, so I ran some tool that was allegedly read-only. It was not read-only, and after running it, all of the partitions tables of all three disks that were still in the machine were missing. Scared the shit out of me. I got the bright idea to put the failling disk back in and dd the first few blocks of it onto each of the three that had their partition tables wiped, and OMG, it worked. I was able to resilver them, even with the failing disk limping along, and get back to 100% data integrity.

it is frying here during summer in Australia. I have to buy a seperate airconditioner for the server room and pipe it into the intake of my home lab/nas so it would not shut down on me

My last home I put the server in the closet with my heat pump water heater. They pretty much balanced each other out.

“not enough space in the used server for a standard power unit”

I wanted to put a dedicated graphics card in there for interference experiments - but the server is, well, a server - and those high availability PUs are expensive and a hell to cable manage

But a Dremel I had at hand … And a second PU as well. Now my server has a second power unit which is only powering the graphics card and is short circuited to be always-on.

self-made problems require self-made solutions.

My WAN connection would just randomly start losing packets and would not recover, not full disconnect just lots of retries. Didn’t impact LAN just WAN. Unplugged the cable or disabling then re-enabling the interface would fix it for a coupe of days to a week then it would come back.

Time consuming to diagnose as switching cables, ports, etc would require waiting days to see if it fixed it.

Eventually had the ISP come out and swap the fiber media converter and fiber integrity. But again it came back.

After several weeks I stumbled on a FreeBDS bug that was exactly my issue…I was running OpnSense and my gateway had I225s. Applied the scripts and it worked.

I used to have a Ryzen 2700X based system as a server. At the time (no clue if it’s still an issue), there was a bug in the interaction between early Ryzen chips and Linux that caused the OS to lock up intermittently. I used a ATmega32u4 board to connect to the reset switch on the PC and as a USB device with a systemd unit that sends a ping to the board every 30 seconds. If the board doesn’t see a ping for a minute, it reboots the PC. It was janky as hell, but it worked for 2+ years!

That reminds me of the old story where they positioned a PC opposite from the server and taped a pen to the CD drive so that opening the CD tray presses the reset button of the server. The PC would automatically detect when the server freezes and open the CD tray. Probably just a legend but still a great example for how a silly and simple solution that just works can often times be as good if not better than a sophisticated but over engineered solution

so you reinvented watchdog?

yes, it’s a zip-tie-and-duck-tape watchdog that cost me $5

most BIOSes have it as an option though

I don’t know about “most”. It hasn’t been an option on any of the consumer motherboards I’ve used in my homelab.

The backup server at my parents house really did not want to run. Something kept breaking to the point that no web interface or SSH worked. Now I’m not new to self hosting, so I ran my fair share of HDMI and USB cables across my space to debug something on my server. But at my parents place it’s just that extra bit annoying, since I only bring my laptop. Finding a screen is doable, but the keyboard got me more than once. The only one they have kicking around has a freaking PS/2 cable… not even my salvaged backup hardware is old enough to support that. More than one time I had to wait for friends to get back from work so I could borrow their keyboard just to press ‘any’ key.

I decided to handle the issue by getting an IP KVM, which should help me deal with that, right? Wrong! I installed a new drive and that piece of hardware goes into some secure boot loop or whatever that is too fast for my KVM to turn on…

Not on a computer but I had a JVC TV that started blanking a few times a minute whenever my dvd player was connected. Whilst trying to fix this I attempted to update the TV firmware. That didn’t fix a thing but I noticed that the blanking went away when the USB drive with the firmware update was plugged in. So I just plugged a cheapo USB drive in and left it there for as long as I owned that TV and dvd player.

Way back in the olden days, we had a hard drive with out any backups go bad. I determined that it was the circuit board. So I took a working drive and basically connected the bad driver’s platters to the working drive’s circuit board.

It worked and I was able to back up the data and restore it on a new drive.

also mouse not pictured

😤😤😤

I’ve had a mouse burn my generator coils while going in while running