With a fully charged battery it should read 12-13 volts as minimum. Any less and it's not fully charged.
When you turn on the key power is sent to the field coil of the alternator and the ignition coil Both of these draw a lot of current. This can draw the battery voltage down a volt or two.
If your headlight comes on when you turn the key on it can draw the voltage down even more.
I would first start by charging the battery.
Once charged test for voltage at the battery, this will give you a baseline. Next check voltage at the red wire on your toggle switch. It should read the same as the battery. if not you have problems between the battery and switch.
Now with the switch on, check voltage at the brown and blue wires, It may read a bit less than the battery. At this point recheck the battery. Compare this reading with the reading At the brown and blue wires. If low the switch is bad. Being a new switch it should be ok.
Now check voltage at the ignition coil, engine stop switch in the run position of course. The voltage should be the same as battery voltage, or at least with in .2 or .3 of battery. If not you have problems between the toggle switch and the coil. Often the fuses or engine stop switch. The engine stop switch can be taken out of the housing and cleaned. Fuses if they are the stock round glass fuses are crap. they were crap when new. Change them for blade type fuses. Either a new fuse box or individual inline fuse holders.
Another thing you can try is to unplug the voltage regulator. This stops the current flow through the field coil of the alternator. This should decrease the voltage drop.
I might suggest starting at one end of your bike and working to the other. Checking each connection in the electrical system. Pull them apart inspect for corrosion and dirt. Clean them On the female side of the connectors you can pull them from the housing by releasing the locks. A bit of a squeeze will tighten them. When reassembling a bit of dielectric grease will prevent corrosion.
Leo