Cardozo was born on January 1, 1837 in Charleston to a free black mother, Lydia Weston and a Jewish father, Isaac Cardozo. He received his early education from schools for free blacks. After leaving school at the age of 12, he worked as an apprentice and journeyman carpenter. At 21 he traveled to Glasgow, Scotland to pursue a college education. After graduating, he moved to London and studied at several Presbyterian seminaries. Upon returning to the United States, he served as a minister in Connecticut until 1865 when he left the ministry and went to work for the American Missionary Association. The A.M.A. sent him home to Charleston where he founded Avery Institute. He was a delegate to the Constitutional Convention of 1868, and that same year was elected Secretary of State and became the first African-American elected to a statewide office in the U.S. He was elected State Treasurer in 1872 and re-elected in 1874 and 1876. He was forced to leave office and politics with the overthrow of the State Reconstruction government by white Democrats. A massive state investigation into the purported frauds of the Republican government was launched by the white Democrats. Cardozo was one of only three individuals targeted for prosecution. He was convicted of conspiracy to issue a fraudulent pay certificate and served a short time in jail but was subsequently pardoned. He then moved his family to Washington, DC where he worked for the U.S. Treasury Department and then became one of the leading educators in the public schools of the District of Columbia. A school named in his honor still operates. Cardozo died on July 22, 1903.