Carly Simon was born in New York City, New York. Her father was Richard L. Simon (co-founder of Simon & Schuster), a pianist who often played Chopin and Beethoven at home. Her mother was Andrea Louise Simon (née Heinemann), a civil rights activist and singer. Her father was Jewish and her mother was of German, French, and Afro-Cuban descent.
Simon was raised in the Riverdale neighborhood of the Bronx, New York City and has two older sisters, Joanna (b. 1940) and Lucy (b. 1943), and a younger brother, Peter (b. 1947). They were raised as nominal Catholics, according to a book of photography Peter published in the late 1990s. She attended Riverdale Country School. She also briefly attended Sarah Lawrence College and joined Alpha Gamma Delta, before dropping out to pursue music.