Taylor series expansion is often used in the downward continuation of potential field, and its performance depends on the accuracy and reliability of the vertical or radial partial derivatives (VPDs or RPDs) of potential field parameters. In order to avoid the singularity on spherical boundary and the uncertainty to the computational results by using the closed analytic kernel function to solve the partial derivative, considering the fact that all kinds of gravity observations behave as a type of limited spectrum bandwidth signal after being filtered, this research proposes to express the kernel function of the Poisson integral for the external gravity anomaly by a spherical harmonic series expansion, which is then truncated into a band-limited summation that has the same spectrum range as the gravity observation. After that, we derive a set of band-limited formulas to calculate the high-order RPDs, which are modified and applied to the downward continuation of the gravity anomaly by Taylor series expansion. The formulas are validated using the ultra-high-degree geopotential model EGM2008 by a two-stage procedure. The numerical tests of the band-limited formulas and the Taylor series expansion downward continuation model show that the proposed band-limited formulas have good reliability and validity, and are superior to other models in terms of stability and accuracy.