Rev. Robert Allen Castleman and Mary Morrison (Lee) Castleman move back to Virginia in 1860

In 1860, just prior to the outbreak of the Civil War, Mary wrote:

“His [Rev. Castleman] mother just died in Virginia and left him, six slaves. He cannot conscientiously set them free; for he cannot tell what would become of them. He cannot send them to Liberia; he has not the means. He cannot bring them here [Harrisburg, PA] in the existing state of feeling on the subject. His conclusion is, he must go back to Virginia, take a country parish and take care of these slaves.”

The years during and immediately after the Civil War were difficult for this large family. Practically destitute, they were loaned a small farm which allowed them to feed their family. Both Mary and her husband sought out teaching jobs.

"In 1865, Rev. Castleman was inexplicably shot and killed by an unknown person while returning home from a neighbor’s house. His obituary noted, “He had no enemy, capable of such a deed; it is supposed he was the victim of an appalling mistake.” [source: Mrs. Mary Lee Castleman Remembering Herndon's History: A Prominent Woman’s Influence on The Herndon Seminary and St. Timothy’s Episcopal Church.]

Rev. Castleman's obituary reads: "An affectionate husband and devoted father, a faithful pastor, and preacher of Christ and Him crucified."

• On Feb 27, 1860, Abraham Lincoln said he favored "deportation" of slaves. 
• In 1862 In 1862, when Congress ended slavery in the District of Columbia they set aside $600,000 for freed slaves to be deported. Lincoln's Commissioner of Emigration planned to send people to a list of countries that were not even their native lands such as Equador, Honduras, French Guiana, etc.

Mary Morrison (Lee) Castleman was the first cousin of Robert E. Lee