We review the formation and evolution of compact binary stars consisting of white dwarfs (WDs), neutron stars (NSs), and black holes (BHs). Binary NSs and BHs are thought to be the primary astrophysical sources of gravitational waves (GWs) within the frequency band of ground-based detectors, while compact binaries of WDs are important sources of GWs at lower frequencies to be covered by space interferometers (LISA). Major uncertainties in the current understanding of properties of NSs and BHs most relevant to the GW studies are discussed, including the treatment of the natal kicks which compact stellar remnants acquire during the core collapse of massive stars and the common envelope phase of binary evolution. We discuss the coalescence rates of binary NSs and BHs and prospects for their detections, the formation and evolution of binary WDs and their observational manifestations. Special attention is given to AM CVn-stars — compact binaries in which the Roche lobe is filled by another WD or a low-mass partially degenerate helium-star, as these stars are thought to be the best LISA verification binary GW sources.