The Phinizy-Segrest House, also known as the Hamilton-Phinizy-Segrest House, is located at 250 South Milledge Avenue (Tax Parcel No. 12-2-D2-B-002).
 

This two-and-a-half-story building represents the Greek Revival style with elaborate Victorian detailing. The symmetrical H-shaped plan has a central main section with a two-tiered porch and one-and-a-half-story wings. Of frame construction and clad in weatherboard, it has a hipped metal roof. The porch, which was raised to accommodate an English basement, features an elaborate balustrade, porch supports, and lace-like brackets, all of cast iron. Dr. Thomas N. Hamilton began construction of the house in 1857 but did not live to see the building completed in 1858. His widow, Sarah Hamilton, resided there for a time before Dr. and Mrs. H. C. White rented the house. In 1890 the widow of Ferdinand Phinizy bought the house, remodeled the interior extensively, and installed gas chandeliers.

Her granddaughter, Mrs. Robert T. Segrest, sold the house to Phi Mu Sorority in 1964. Phi Mu rehabilitated the property, replaced the existing straight flight of stairs with a curving double set, added wings to the building, and constructed dormitory space at the rear. An original balustrade atop the portico was removed. The source of the ironwork remains uncertain, but according to local historians, the grillwork supposedly was shipped from Philadelphia on the last train South before the Civil War. Athens had two foundries at this time, however, where the design could have been duplicated.

The Phinizy-Segrest House has been locally designated as a Historic Landmark (January 8, 1991).