Make sure that you don’t have any drafts. Holes around windows can cause loss of much-needed heat, a little bit of electrical tape around your windows can make a BIG difference. Also, proper insulation is a great investment. While the initial price might be steep, what you save of electricity will cover it in the long run. If you can’t replace your windows for double paned, consider covering them, curtains, can do a great job.
Finally, make sure that your heating system is running efficiently. If you are going with a space heater, some models need less energy than others. If you have central, have a technician come at the beginning of the season. A small maintenance bill can save you the cost of replacing the whole unit!