Objectives: The most widely used model for Polarimetric Interferometric Synthetic Aperture Radar (PolInSAR) in forest height inversion is the random volume over ground (RVoG) model. However, there are many unknown parameters in the model, and it is difficult to solve the parameters based on single baseline PolInSAR data. Methods: In view of this, a forest height inversion method that combines the PolInSAR observation information of single-baseline neighboring pixels is proposed, based on the traditional RVoG model, assuming that the forest height and extinction coefficient remain unchanged within the neighboring pixels, and that the ground-body magnitude ratio parameter changes with pixel changes. The method can be understood as sacrificing part of the spatial resolution for rich observation information, which solves the problem that the parameter inversion based on the RVoG model in the single-baseline configuration has to rely on a priori information. Results: The proposed joint-neighborhood pixel single-baseline forest height inversion algorithm is experimentally validated using PolInSAR data covering the Krycklan coniferous forest experimental area and the Mabounie rainforest experimental area, and the results show that, compared with the traditional solving strategy, the root mean square error of the proposed method are reduced by 25% and 19% for the coniferous forest experimental area, and by 23% and 15% for the rainforest experimental area. in the coniferous forest and 23% and 15% in the tropical rainforest, respectively. Conclusions: Therefore, the single-baseline PolInSAR forest height inversion algorithm with joint domain pixels can obtain higher accuracy compared to the traditional solution strategy and is more suitable for forest height inversion.