首页 >  , Vol. , Issue () : -

摘要

全文摘要次数: 103 全文下载次数: 117
引用本文:

DOI:

10.11834/jrs.20210420

收稿日期:

2020-09-23

修改日期:

2021-07-19

PDF Free   EndNote   BibTeX
高分遥感共性产品生产系统关键设计
张正, 李宏益, 胡昌苗, 唐娉
中国科学院 空天信息创新研究院
摘要:

以众多遥感卫星的发射与行业应用系统为代表,遥感应用技术在近年间取得了快速发展,国产高分系列卫星等平台所提供的数据越来越丰富,以遥感共性产品作为信息源的应用系统也已经覆盖了多个行业。然而这两者之间的桥梁,即由国产卫星数据生产标准化系列遥感共性产品的能力,仍显得相对薄弱,极大限制了国产遥感数据的使用率与影响力。这一能力的改善需要从产品体系、产品算法与生产系统等多方面共同推进。本文从高分遥感共性产品生产系统的角度,针对系统所面对的来自数据集成、算法集成、生产编排与云计算等方面的诸多挑战,提出了一套完整的系统关键设计。系统在容器化集成运行的基础上,涵盖了集群、数据、算法、权限、参数、工作流、生产等一系列关键环节的关键技术。系统研发与生产实践表明,本系统可以流畅的集成各类共性产品算法,实现稳定高效的业务化生产平台,助力于国产卫星产品服务能力的有效提高。

GaoFen Quantitative Remote Sensing Production System: Core Design
Abstract:

Objective: With the launches of various remote sensing satellites and the establishments of numerous industrial application systems, recent years have witnessed a fast development of remote sensing science and technologies. Satellite series like GaoFen have pushed the richness of data to a new level and quantitative-product-driven application systems are becoming increasingly influential in many disciplines. By contrast, the bridge between data and application, namely the production capability of quantitative products, seems weak, which greatly limit the usage and influence of GaoFen data. The improvement of production capability comes from multiple aspects including product hierarchy, algorithm model, and production system. From the perspective of production system, we propose in this paper an integrated system design that responses to four main challenges of the system: uniform data access, heterogeneous algorithm integration, layered workflow orchestration and cloud infrastructure adaptation. Method: The system is based on algorithm containerization, where executable algorithms and all its dependencies are encapsulated so that it can run uniformly and consistently on different infrastructures without worrying about the complexity caused by deployment. This helps the system to manage diverse remote sensing algorithms in a uniform manner. We employ the Kubernetes container orchestration platform to automate the execution, scaling, and management of containerized algorithms. A containerized cluster consists of multiple master nodes, lots of computing nodes, multiple data centers, and multiple algorithm repositories is constructed to support the system and to cope with the high computing and data throughput density of remote sensing algorithms. To improve load balancing, each algorithm repository is further divided into several sub-repositories. User defined role base access control for algorithms is set up to protect intellectual properties of algorithm owners. A recommended algorithm image architecture is introduced to standardize algorithm encapsulation. A set of nine properties are abstracted to uniformly describe any data entity parameter of an algorithm and this ensures that suitable input data can be found for user uploaded algorithms to run in the system. For scenarios of both data visualization and quantitative computing, a multi-scenario data organization strategy is proposed to avoid excessive data operations such as projection transform or subdivision. The business logic of the system, from user order creation to product calculation, is detailed for clear implementation. Sometimes the production involves workflow batches and we propose a stratified workflow aggregation strategy to optimize workflow execution. Result: The system has been used for large-scale production of various kinds of GaoFen quantitative remote sensing products, including surface reflectance (REF) product, normalized difference vegetation index (NDVI) product, leaf area index (LAI) product, and surface albedo (Albedo) product. These products fully cover China area for eight successive years from 2013 to 2020 with quantities of more than five million and with storages of nearly 300TB. The proposed system completes the production task smoothly and efficiently. Conclusion: During the routinely support for many large-scale production tasks, each part of the system performed consistently with the system design proposed in this paper, which demonstrates that the paper can help to build a stable and efficient quantitative remote sensing production system on cloud native infrastructures and essentially improve the capability of serving high-quality quantitative product.

本文暂时没有被引用!

欢迎关注学报微信

遥感学报交流群 分享按钮