The Hoge House was originally built in 1838 by Charles Brown using enslaved labor in the work. Brown operated a ferry on the Kanawha River. His son, Tallyrand Brown, inherited it around 1848 and rented it out before selling it in 1852 to Captain John Bowyer, a War of 1812 veteran and Putnam County justice of the peace who later served in the West Virginia House of Delegates. In 1857, the house was purchased by James W. Hoge, a lawyer and judge who represented Putnam County at the 1861 Virginia Secession Convention—initially voting against secession in April but signing the ordinance in June. Hoge lived in the house until his death in 1882. The house was relocated behind the Putnam County Courthouse in 2004.