The focus of this paper is the effect of spatial non-uniformity in the ambient flow on the forces acting on a rigid sphere when the sphere Reynolds number, $Re$, is in the range 10 to 300. Direct numerical simulations (DNS) based on a pseudospectral methodology are carried out to solve for the unsteady three-dimensional flow field around a sphere which is either held stationary or allowed to translate freely under the hydrodynamic forces. The various components of the total force, namely the inertial, steady viscous, and history forces, are systematically estimated in the context of linearly varying straining flows. The inertial forces are isolated by computing the rapid changes in the drag and lift forces in response to a rapid acceleration of the ambient flow. It is shown that the inertial forces arising due to convective acceleration at moderate Reynolds numbers follow the inviscid flow result. While the effect of temporal acceleration depends only on the sign and magnitude of the acceleration, the effect of convective acceleration is shown to depend also on the initial state of the ambient flow. A simple theoretical argument is presented to support the numerical observations. It is also shown that the effect of convective acceleration on the steady viscous force can be realized on a slower time scale. The results show that the history kernels currently available in the literature are not adequate to represent the effect of non-uniformity on the history force.