He was the only child of İrma Felekyan (Toto Karaca) of Armenian origin, a popular opera, thatre and movie actress, and Mehmet İbrahim Karaca of Azeri origin. His first group was called Dynamites and was a classic rock cover band. Later he joined Jaguars, an Elvis Presley cover band. In 1967, he started to write his own music, forming the band Apaşlar (Apachees), his first Turkish-language group. In 1969, Karaca and bass-player Serhan Karabay left Apaşlar and started an original Anatolian group called Kardaşlar (Brothers).
In 1972, Karaca joined the group Moğollar (Mongols) and wrote one of his best-known songs, "Namus Belası". However, Cahit Berkay, the leader of Moğollar, wanted an international name for his band, and he left for France to take the group to another level. Karaca, who wanted to continue his Anatolian beat sound, left Moğollar and started his own band Dervişan (Dervishes) in 1974. Karaca and Dervişan sang poetic and progressive songs.
In the 1970s, Turkey's image was damaged by political violence between supporters of the left and the right, separatist movements and the rise of Islamism. As the country fell into chaos, the government suspected Cem Karaca of involvement. At times he was accused of treason for being a separatist thinker and a Marxist-Leninist. The Turkish government tried to portray Karaca as a man, who was unknowingly writing songs to start a revolution. One politician was quoted as saying, "Karaca is simply calling citizens to a bloody war against the state." Dervişan was ultimately dissolved at the end of 1977. He later founded in 1978 Edirdahan, an acronym for "from Edirne to Ardahan"; the westernmost and the easternmost provinces of Turkey. He recorded one LP with Edirdahan.
In early 1979, he left for West Germany for business reasons. Turkey continued to spin out of control with military curfews and eventually a military coup on September 12, 1980. General Kenan Evren took over the government and temporarily closed all the nation's political parties. After the coup, many intellectual people, including writers, artists and journalists, were arrested. A warrant was issued for the arrest of Karaca by the government of Turkey.
The state invited Karaca back to the country several times, but Karaca, not knowing what would happen upon his return, decided not to come back.
While he was in Germany, his father died, but he could not return to attend the funeral. After some time, the Turkish government decided to strip Cem Karaca of his Turkish citizenship, keeping the arrest warrant active.
Several years later, in 1987, the prime minister and leader of the Turkish Motherland Party, Turgut Özal, issued an amnesty for Karaca. Shortly afterwards, he returned to Turkey. His return also brought a new album by him, Merhaba Gençler ve Her zaman Genç Kalanlar ("Hello Young Ones and Everyone Who'd Always Felt Young"), one of his most powerful works. His return home was greeted happily by his fans, but during his absence, Karaca had lost the youthful audience and acquired few new listeners. He died on February 8, 2004 and was interred at Karacaahmet Cemetery in Üsküdar district of Istanbul.