Do car batteries freeze? And if so, what can I do?

The answer in short is yes, flooded wet cell batteries do freeze.
In extremely cold climates, batteries stored outdoors may be subjected to freezing. Freezing usually results in irreparable damage to the battery plates.
Cold engines need more cranking energy to get started in the winter, which can really put a strain on car batteries. Is your battery up to the task?
The Optima  YELLOWTOP® battery is protected from freezing all the way down to -30F and the Optima  REDTOP® battery,  all the way down to -50F.
However, that protection is only good if the batteries are fully-charged!

If your car, truck, boat, lawnmower or other seasonal use vehicle has been in storage for a few months, it wouldn't be a bad idea to check the voltage of your battery and make sure it is still maintaining a charge.  Fully-charged, the REDTOP battery (and 34M BLUETOP®) will measure about 12.6-12.8 volts and BLUETOP and YELLOWTOP batteries (except the 34M BLUETOP) will measure about 13.0-13.2 volts.

The lower the voltage in your battery, the less protection it has from freezing temperatures.  If you find your battery has been frozen, do not attempt to charge it until it has been thawed out at room temperature for at least 24-36 hours.

Car Batteries and Cold Weather