Of all modern land-based mammals, hippopotamuses are the closest to whales.
To understand when whales and land-based mammals diverged, a useful first step is to compare whales to other modern-day animals.
Molecular and skeletal evidence suggest that whales are most closely related to hoofed mammals and in particular the hippopotamuses, with which they probably shared a common ancestor 54 million years ago.
Although we cannot say with certainty what this common ancestor was, it was likely closely related to a land-based carnivore called Pakicetus, which appeared around this time. This creature had teeth and ear structures very similar to modern whales.
Pakicetus is one of many fossils discovered recently in Pakistan from the Eocene period (55 to 34 million years ago), at which time Pakistan was under the prehistoric Tethys Sea. These fossils provide a startling account of the transition between early land animals and modern whales.
In particular they are related to the artiodactyls. The artiodactyls are the even-toed ungulates (hoofed animals) whose weight is borne about equally by the third and fourth toes, rather than mostly or entirely by the third as in odd-toed ungulates (perissodactyls) such as horses. Artiodactyls include pigs, hippopototamuses, sheep and cattle.
In 1985 Vincent Sarich had conjectured that whales were descended from artiodactyls based on molecular evidence.
Further evidence came in 1999 based on DNA evidence. Norhiro Okada and colleagues at the Tokyo Institute of Technology studied DNA snippets called SINEs (short interspersed repetitive elements) that confirmed the closeness of whales and artiodactyls on the tree of life.
The clincher came in 2000 when two different groups of paleontologists [Thiewissen et al. 2001] and [Gingerich et al. 2001] found the ankle bones of two different kinds of primitive whales from Pakistan and showed their morphological relationships to the ankle bones of artiodactyls. Artiodactyls and whales have an ankle bone, called the astralagus, with a characteristic "double-pulley" shape that is unique in the mammal world. All other mammals have a single-pulley ("single-spooled") ankle structure.
Alan Boyden of Rutgers University initially provided evidence of the closeness of hippopotamuses and whales by looking at antibody-antigen reactions [Boyden and Gemeroy 1950]. At this time it wasn't clear whether whales were directly descended from the artiodactyls or whether they were a cousin species sharing a common ancestor. This last possbility was excluded by later fossil evidence.
Hippopotamuses are closer whales than they are to other artiodactyls such as pigs and cattle.
Pakicetus was discovered by Philip D. Gingerich in 1981 in Pakistan, in a region that was once under the Tethys sea in the Eocene [Gingerich and Russell 1981].
Pakicetus on Wikipedia - some debate about whether it was semi-aquatic or not.
Certain cusps on teeth can identify them as mammals rather than reptiles. These early mammals had complexly cusped grinding molars, but modern whales have either very simple teeth or none at all. [Wong article in SciAm pages 188-9 2002].
Pakicetus had a thickened bulla better adapted for either hearing underwater or picking up vibrations from the ground. It still retained a mammal-like eardrum [Wong article in SciAm 2002].
[Boyden and Gemeroy 1950] Boyden, A. and Gemeroy, D. 1950 "The relative positions of Cetacea among the orders of Mammalia as indicated by precipitin tests", Zoologica 35:145-151.
[Gingerich and Russell 1981] Philip D. Gingerich, D. E. Russell (1981). "Pakicetus inachus, a new archaeocete (Mammalia, Cetacea) from the early-middle Eocene Kuldana Formation of Kohat (Pakistan)". Univ. Mich. Contr. Mus. Paleont 25: 235-246.
[Gingerich et al. 2001] Philip D. Gingerich, Munir ul Haq, Iyad S. Zalmout, Intizar Hussain Khan and M. Sadiq Malkani, "Origin of Whales from Early Artiodactyls: Hands and Feet of Eocene Protocetidae from Pakistan" in Science (2001) Vol. 293 no. 5538 pp. 2239-2242. link.
[Thewissen et al. 2001] Thewissen, J.G.M., E.M. Williams, L.J. Roe and S.T. Hussain, "Skeletons of terrestrial cetaceans and the relationship of whales to artiodactyls" Nature 413:277-281.