Yolen was born at Beth Israel Hospital, the first child of Isabel Berlin and Will Hyatt Yolen. She and her family moved to California when she was young but returned to New York a few years later. After her younger brother was born, her father joined the army and served on the European front during WWII. Yolen spent her childhood taking piano lessons, ballet dancing and writing a neighborhood newspaper with her brother. Yolen spent her teen years in Westport, Connecticut, and became involved in choir, basketball, debate, language clubs and the school paper. She graduated from Smith College with several publications in literary magazines already to her name.