One of the first things that I noticed about doing astronomy is that given the same daytime conditions, clear nights are colder than cloudy nights. Clouds blanket Earth and trap heat, so there really is a basis to this observation. Winter nights are longer, so observing stars all night can mean 12+ hours in the cold.
As a working observational astronomer, I have spent many 12+ hour winter nights at the telescope. Sometimes I'm lucky and it is relatively warm, but I have observed all night on many nights cold enough to make even Al Gore support global warming.
To enjoy the winter night sky, dress warmly. Don't just throw a parka on over your T-shirt. Wear many layers under the parka.
I typically wear 3 to 4 pairs of long johns, some wind resistant pants, and then some ski pants. Heavy long socks also help the legs and feet. For my torso, I might wear a half dozen or so long and short sleeve T-shirts underneath a couple fleece or sweatshirt layers. Then I have a down vest, occasionally two, and my down parka. A hat is also crucial; don't leave your head unprotected. Your fingers and toes cannot be warm if your head is not. I can gain 20 pounds getting dressed for observing. If you are wearing too much, you can always save a layer or two for the later part of the night when it gets colder.
Don't dress like you would for skiing. Wear much more. When skiing, the exercise warms you. Astronomy is less active, so wear more to keep warm.
Proper dress can help you enjoy stargazing on even the coldest winter nights. Put on your layers, get out there, and enjoy the night sky.