Besides ground cable on engine block check to make sure there is a battery ground to a chassis part from the negative side. Mine had same symptoms and that was the fault which also destroyed the regulator.
Sounds like a bad switch to me as well. We used relays and a distribution buss per the mad enterprises specs(madelectrical.com), 14.50 volts at the head lights vs 11 or so stock and the dash head light switch now only has to energized the relay. Lights are bright and steady.
Its a ground fault. You don't have a proper ground. You can replace all the switches in the world,but its a ground problem. The lights fluctuate with the engine rotation at idle. When you are doing 2500 rpms or normal driving the fluctuation ceases. Its a bad ground.
there is a circuit breaker in headlight switch that heats up, opens current path (and turns lights off) to headlights, cools down and closes switch ad nauseum. the circuit breaker gets tired out and then normal headliight current starts exercizing it. listen near the switch knob and you can usually hear it clicking at the rate lights are going on and off. replace headlight switch as other poster advised.
You didn't say which lights. I had the same problem with the dash lights in my '70 Cougar. Also the 4-way amp in my sound system would cut out. I cured both problems with a GM 12SI alternator. I used a kit from M.A.D. Enterprises, already mentioned in this thread. The part no. is for a 1983 Z-28.
Check for a grounding strap from engine to firewall, to be sure, but if it's your dash lights you're talking about the cause is that low amp Ford alternator.
I also have the M.A.D. relay kits on the headlights; works great.