In recent years, forest fires occur frequently around the world, which severely damage the structure and function of the forest ecosystem. The initial assessment of burn severity could provide a quantitative basis for rapid implementations of post-fire restoration measures. In the last decades, remote sensing-based models have become an appropriate choice to assess burn severity, which generally require a certain amount of field survey data. However, this requirement could not be sufficiently satisfied in the first moments after fire, since the field survey work would cost a substantial amount of time and labor. The absence of field survey data in the initial assessment of burn severity would largely limit the efficient application of remote sensing technologies. In this study, a transfer learning algorithm (i.e., SSTCA, semi-supervised Transfer Component Analysis) was employed to propose an initial assessment model of burn severity to improve the time-efficiency of traditional remote sensing-based models. Firstly, the SSTCA algorithm was applied to project a series of new features from original spectral features of remotely sensed data. Based on these projected features, a Support Vector Regression (SVR) model was then trained using historical field survey data from source areas (i.e., Bear fire on June 27, 2002 and Mule fire on July 11, 2002). Thereafter, the SSTCA-SVR model was transferred to the initial assessment of burn severity of a target area (i.e., Lushan fire on March 30, 2020). Finally, the performance of this proposed model was quantitatively compared with those of some traditional models (i.e., dNDVI-, dLST-, dNBR-, and SVR-based models). Results showed that original spectral features of remote sensing images over source and target areas were quite different. After the SSTCA projection, projected features of source and target samples have a similar distribution pattern in the new features-based space. Meanwhile, in the initial assessment of burn severity, dNDVI- and dNBR-based models have overestimated burn severity levels with low accuracies (i.e., overall accuracy was from 20.80% to 24.80% and Kappa value was between 0.01 and 0.06). Compared with them, the dLST-based model has a better performance with an overall accuracy of 34.80% and a Kappa value of 0.19. Although SVR-based model has shown a promising performance with an overall accuracy of 58.00% and a Kappa value of 0.48, this model has overestimated the burn severity levels in some regions of burned areas. The assessment results of burn severity levels using SSTCA-SVR model has the best performance with an overall accuracy of 71.20% and a Kappa value of 0.64. We conclude that the application of a transferring learning algorithm would be helpful for building an assessment model of burn severity with a good transferring ability. In this way, more accurate results could be obtained in the initial assessment of burn severity, and the response of post-fire management might be accelerated after forest fires.