Don't try to use muffs without something to prevent the muff from pushing back on the levers. The wind and drag from the muffs will easily apply the brakes or make the clutch slip.
I have been using silk glove liners inside snowmobile gloves to keep my hands warm. With the Barkbusters hand guards I have, this is good down to around freezing. There is a noticeable loss of dexterity from the snowmobile gloves, but that is unavoidable. The silk liners do more than one would think to keep the fingers warm and dry. And don't forget your neck! Your neck will loose more heat faster than any other part of your body. I wear a Buff in cool weather, and a micro fleece neck warmer in cold weather. Sometimes both together.
Some general cold weather tips: Wool is your friend, cotton will kill you. Do not wear any cotton in cold weather as it will absorb your perspiration and then conduct your precious body heat away. Wool, silk, and synthetics only! Loose layers trap air, so start with close fitting silk or synthetics against your skin to wick perspiration away. Then a wool layer for trapping air, and synthetic on the outside. Your outer layer must block the wind, and stop any water from entering. Add more layers as required, being sure that each layer is looser than the previous or you will compress the lower layers and loose the ability to trap air.
When riding longer distances in cold weather, I make sure to eat extra carbs for the energy, drink more water, and I do some iso-metric exercises when riding to keep the blood moving.