I am back on this. After experimentation, I have a handy little overload circuit that handles DC, AC, and pulsey noisy DC, regardless of polarity, has tight trip accuracy, and trips in about 1/4 second. When sensing AC or DCC, it will be a bit slower owing to how it has to average the current with an oscillating waveform. The more over the current limit you are, the faster it trips. So if the trip point is 3A and you load it with 3.1A, it might take 1/2 second to trip. But if you are well over the line (like 3.5A), or have a dead short, it will trip almost instantly. It uses the ACS712 Hall Effect chip for current sensing so there is no voltage drop and no heat.
Ain't no Arduino, no programming. Just a comparator chip and conventional analog electronics.
A 3D rendering of it is below (from the PC board layout program). It's a little 2" x 3" board. It can run on 12V (nominal, like 9 to 14v) DC or AC, it has a built-in 8 amp relay for switching off the power, and it can be set to trip anywhere from about 0.5 to 5 amp. It has its own trip indicator LED and reset button, but you can also connect another one so it can indicate and be reset from a panel.
You just put it inline with either of the two main wires going to the track (output from power pack, throttle, DCC booster, old Lionel transformer, whatever), and then set the overload point by turning an adjustable trimmer on the board.
I have a philosophy question for the group, but I already think I know what the answer will be. The way I originally designed it, on an overload, it energized the relay to cut off the track power. But now I have changed it so that when everything is OK, it energizes the relay and turns the track power ON. If there is an overload, it de-energizes the relay to cut power off. My thinking was that if this board is ever disconnected or it blows up, the "fail safe" mode should be "track power off". The only drawback is that you have a relay activation every time it is turned on. But seeing as how modern relays are good for tens of thousands of activations, I think if it ever fails, it is better to fail in the "off" state.