Castleton is a town in Rutland County, Vermont, United States. Castleton is about 15 miles to the west of Rutland, and about 7 miles east of the New York/Vermont state border. The town had a population of 4,717 at the 2010 census. Castleton University is located there, with roots dating to 1787. Fort Warren was located in Castleton.