Abstract: A method based on local HSV image and shape of object to recognize object is proposed for robot tracking.After the color segmentation,use the knowledge of shape of objects to recognize objects.The robot tracking result testifies the availability of the method.