Miller's Point got its name when the land was sold to Edmund Miller in 1825, who developed it as a whaling station. However the whaling operations were stopped in the 1850s and, for almost a century, the coast remained unused apart from occasional fishermen.In the 1920s, the land was bought by the Molteno family of Cape Town who used it as a family retreat and closed it off from public access. In 1961, the Moltenos donated the land to the city for conservation. The land has now been integrated into the Table Mountain National Park and the family's manor is now a restaurant complex.
The area around the former manor is now a recreational area, and a portion of the land is leased to the Cape Boat and Ski-boat Club. The rest of the property (extending from the Swartberg Mountains down to the False Bay coast) is now a nature reserve, with some provision for holiday accommodation.The point itself has two beaches, several sheltered coves and tidal pools.