Abstract: Objectives: Phase filtering is one of the key technologies in interferometric synthetic aperture radar (InSAR) data processing, and the quality of interferograms significantly affects the accuracy of subsequent processing steps. The traditional phase filtering methods with manual parameter adjustment not only have low filtering accuracy but also low computational efficiency. Deep learning-based phase filtering methods have great potential, but current convolutional neural networks (CNN) based methods overlook the non-local features causing limiting accuracy. Transformer-based networks have strong global modeling capabilities and can extract non-local features from interference fringes, but they struggle with handling local features of interference fringes. Methods: To effectively improve the filtering effect, this paper proposes a joint local and non-local phase filtering method for InSAR images combined with the advantages of convolutional neural network (CNN) and Transformer network. First, based on the strong global feature extraction capability of the robust Transformer network, a PFCT network structure is proposed for phase filtering. The network performs both local and nonlocal filtering at the same time, avoiding the accuracy limitation problem caused by the neglect of nonlocal features of interference fringes in the existing methods. Then, a new complex loss function is proposed to guide the network to maintain the integrity of interference fringes while improving filtering performance. Results: In the simulation data experiment, the mean square error index was 15.5% lower than the suboptimal algorithm, and the structural similarity index was 5.3% higher. In the true data experiment, the residue removal index was 1.8% higher than the suboptimal method. The experiments show that the PFCT network model only slightly reduces computational efficiency, but achieves better filtering results than other methods, effectively maintaining the stripe structure while maintaining the filtering effect. Conclusions: The proposed method has a great filtering effect and powerful generalization ability on InSAR phase filtering.