We study the structure and stellar populations of the luminous elliptical galaxies dominating fossil groups and compare them with the brightest galaxies in ordinary groups. Despite being over-luminous, the fossil central galaxies do not show boxy stellar isophotes which are usually associated with luminous elliptical galaxies. Boxy isophotes, according to the numerical simulations, are produced in gas poor mergers. The isophotal shapes of the fossil central galaxies, therefore, suggest a gas rich merger for fossil central galaxies. Using a two-component spectral fitting, we show that the dominant stellar population of the fossil and non-fossil galaxies is old and the second population is either old or intermediate age. However, the second stellar component (recently-formed stars) in fossil central galaxies is significantly more metal poor than that in the brightest galaxies of non-fossil groups.