Lee was born in Abbeville County, South Carolina. He was born a slave and belonged to General Samuel J. McGowan. When McGowan joined to Confederate Army, Lee accompanied him as his servant throughout the war and was twice wounded. After the war, he moved to Edgefield County and was elected to the General Assembly in 1868. In 1870 and 1872 he was elected to represent Aiken Count y in the legislature. In 1872, he became the first African-American Speaker of the House serving for two years before leaving the Assembly to run unsuccessfully for Attorney General in 1874. Lee had been admitted to the bar in 1872. He and George Clyde with some association with Robert Brown Elliott opened a law office and soon dominated the court dockets in Aiken. In 1876 Lee was elected solicitor and held that office until 1878 when he resigned and moved to Charleston to practice. His resignation was caused by allegations of corruption which led to his indictment and conviction in the fall of 1878. He was released from jail in November of 1878, and returned to Charleston and again practiced law. He rose to become one of the most successful black lawyers in the state. He mentored a number of younger black lawyers. In 1891, he was appointed Brigadier General of the black state militia, where he served until his death on April 3, 1895 in Charleston.