Planetary remote sensing images are an important data source for planetary observations and are the basis for qualitative and quantitative analysis of the planet’s surface. Analyzing the features of the planet’s surface based on remote sensing images and recognizing and classifying topographic features from massive planetary remote sensing data are significant fundamental tasks in planetary science research. In this new era for deep space exploration and development, multiple missions from different countries and agencies are being implemented. Accordingly, enormous amount of data will be obtained, and this situation requires using automatic target recognition and terrain classification technologies. This study systematically reviews and summarizes the research progress and advances of topography and landform recognition and classification technologies using planetary image data since the start of lunar and deep space exploration missions. First, the moon, Mars, and other planetary exploration missions and the acquired image data are briefly described. After a short introduction to the research progress of general target recognition and classification techniques, the applications of these techniques using the image data of the moon, Mars, and other planets are then elaborated as follows. (1) For lunar images, review of target recognition and classification progress is detailed in three aspects: recognition of the circular structure (i. e., crater), recognition of linear structure (e.g., wrinkle ridge), and terrain classification of the lunar surface. (2) For Mars images, the detailed advances including recognition of tectonic (e. g., crater and volcano), aeolian (e.g., slope streak, sand dune, and dust devil track), fluvial landforms (e.g., channel and gully), and other features (e.g., rock), as well as terrain classification of the Martian surface, are elaborated. (3) For target recognition and classification from other planetary images, the study introduces the research advances on other terrestrial planets (e.g., Mercury and Venus) in the solar system and asteroids that have been explored. Specifically, the asteroid parts are elaborated according to different exploration approaches: close flyby, orbiting, anchoring, and sample acquisition. Finally, future research directions of target recognition and classification using planetary image data are discussed. The future research directions include (1) target recognition and classification using multi-source data: data from different types of sensors, data of different resolutions, and data from different platforms and time; (2) automatic recognition and classification using unsupervised approach; and (3) multi-task image intelligence applications. Achieving high-precision automatic recognition and classification of the planetary surface is still challenging because of the complex environment and featureless texture of the planetary surface. In the future, automatic recognition and classification will surely play increasingly important roles in supporting planetary exploration engineering missions and scientific research through the continuous improvement in data quality and development of related field technologies.