The most southerly geographical features of various types are listed here.
Berkner Island is further South than any of these, but doesn't reach sea level and is fully ice covered and may be considered an under-sea mountain.