The Sergeantsville Inn began as a private residence in the early 1700's in a small town called Skunktown. When the United States Post Office came to town, they would not legitimize the name of Skunktown and the town was renamed sregeantsville after the Sergeant family.

 With the addition of the two wings in the 1830's it became a grain and feed store. Before the resttaurant opened in the early 1900's the building was also used as a grocery store , ice cream parlor, feed store and a pelt trading center.

 Although the building has gone through several owners, the original structure and additions have been preserved. The small stone building adjacent to the Inn was used as the town's ice house. In 1983 it was then joined to the main building and now serves as one of the Inn's four dining areas. Three of the dinign rooms, the library, Ice House and the Wyeth Room still use the original fireplaces.