Subjects: Astronomy submitted time 2024-03-26 Cooperative journals: 《天文学进展》
Abstract： Time series reconstruction is a crucial data processing step in time domain astronomy and serves as the foundation for fitting light curves and conducting time domain analysis. For many large-field time domain surveys, it is necessary to complete this computational process within a single exposure cycle. With the rapid increase in astronomical data, existing methods for astronomical data processing struggle to simultaneously meet the accuracy and efficiency requirements of time-series reconstruction. The memory-based computing general-purpose distributed framework, Spark, holds the potential to improve the efficiency of this process. However, applying Spark directly often encounters issues. MapReduce distributed models like Hadoop and Spark require relatively independent tasks among distributed cluster nodes and minimal data transfer across nodes during execution. Otherwise, frequent communication becomes an efficiency bottleneck for the application of the model. However, due to the presence of boundary problems in cross-matching, it is inevitable to transmit newly added data at the boundaries, severely restricting the concurrency of the model and reducing the acceleration ratio in practical parallel model applications. Therefore, we propose a non-blocking asynchronous execution flow, where each distributed process handles continuous processing exclusively for independent sky regions. The delayed batch appending of additional identification tasks from block-edge newly added celestial bodies in other nodes is determined based on the progress of each process. This ensures that identification calculations are not omitted, thereby improving concurrent efficiency while maintaining algorithm accuracy. Additionally, a research study was conducted on different join strategies between two tables, examining them from both theoretical and experimental perspectives. Furthermore, a join-free strategy was proposed. Finally, the design of an efficient time-series reconstruction system based on the Spark distributed framework validates the aforementioned research. Experimental results demonstrate a significant improvement in the efficiency of the proposed time-series reconstruction algorithm compared to previous research, laying a solid foundation for the analysis of astronomical time-series data in time-domain astronomy.
Subjects: Astronomy submitted time 2024-03-22 Cooperative journals: 《天文学进展》
Abstract： Time series reconstruction is a crucial data processing step in time domain astronomy and serves as the foundation for fitting light curves and conducting time domain analysis. For many large-field time domain surveys, it is necessary to complete this computational process within a single exposure cycle. With the rapid increase in astronomical data, existing methods for astronomical data processing struggle to simultaneously meet the accuracy and efficiency requirements of time-series reconstruction. The memory-based computing general-purpose distributed framework, Spark, holds the potential to improve the efficiency of this process. However, applying Spark directly often encounters issues. MapReduce distributed models like Hadoop and Spark require relatively independent tasks among distributed cluster nodes and minimal data transfer across nodes during execution. Otherwise, frequent communication becomes an efficiency bottleneck for the application of the model. However, due to the presence of boundary problems in cross-matching, it is inevitable to transmit newly added data at the boundaries, severely restricting the concurrency of the model and reducing the acceleration ratio in practical parallel model applications. Therefore, we propose a non-blocking asynchronous execution flow, where each distributed process handles continuous processing exclusively for independent sky regions. The delayed batch appending of additional identification tasks from block-edge newly added celestial bodies in other nodes is determined based on the progress of each process. This ensures that identification calculations are not omitted, thereby improving concurrent efficiency while maintaining algorithm accuracy. Additionally, a research study was conducted on different join strategies between two tables, examining them from both theoretical and experimental perspectives. Furthermore, a join-free strategy was proposed. Finally, the design of an efficient time-series reconstruction system based on the Spark distributed framework validates the aforementioned research. Experimental results demonstrate a significant improvement in the efficiency of the proposed time-series reconstruction algorithm compared to previous research, laying a solid foundation for the analysis of astronomical time-series data in time-domain astronomy.
Subjects: Agriculture, Forestry,Livestock & Aquatic Products Science >> Basic Disciplines of Agriculture submitted time 2017-11-09 Cooperative journals: 《中国生态农业学报》
Abstract：本文基于生态系统服务价值, 从粮食安全角度出发, 通过计算粮食耕地盈亏量、粮食耕地超载指数和 补偿系数, 建立了耕地生态补偿模型, 通过量化县域耕地生态补偿, 对河北省环京津地区耕地生态补偿问题进行了研究。研究结果表明: 1)环京津耕地“生态供给”与“生态消费”存在典型的“空间异位”现象, 其中耕地“生态消费”主要集中于环京津南部地区, 而耕地“生态供给”则主要集中在环京津西部地区。2)2014 年环京津耕地生态系统服务价值为4.480 5×1010 元, 整体呈现不能自足的态势, 总赤字金额为7.834×109 元。3)环京津地区中,河北省张北县、兴隆县、蔚县、尚义县和涞源县等17 个县市表现为盈余, 其余各县市均呈现为赤字状态。其中滦南县需支付的耕地生态补偿量最高, 为5.173×107 元, 其次为玉田县和东光县, 分别为4.864×107 元和4.849×107 元。虽然遵化市、滦平县、曲阳县可获得补偿, 但其耕地生态条件也仅仅表现为“紧平衡”, 仍需受到广泛关注。4)以粮食安全角度为出发点, 2014 年环京津区急需获得耕地生态补偿的县为张北县、蔚县、尚义县、阳原县、涞源县和曲阳县, 需支付耕地生态补偿的县为滦南县、玉田县、献县、吴桥县和定州市, 唐县、涞水县和丰宁满族自治县既不需获得也不需支付耕地生态补偿。与前人的研究相比, 本研究以生态补偿为切入点对河北省环京津地区耕地补偿问题进行了研究, 研究结果对于促进环京津地区经济发展、耕地生态环境保护与耕地资源持续高效利用发挥着重要作用。同时, 此方法的运用可为类似地区生态补偿的量化研究提供参考, 为其他地区以生态价值量确定耕地保护指标提供依据, 对耕地生态补偿价值机制的研究有指导意义。
Subjects: Agriculture, Forestry,Livestock & Aquatic Products Science >> Basic Disciplines of Agriculture submitted time 2017-11-09 Cooperative journals: 《中国生态农业学报》
Abstract：本文基于生态系统服务价值, 从粮食安全角度出发, 通过计算粮食耕地盈亏量、粮食耕地超载指数和 补偿系数, 建立了耕地生态补偿模型, 通过量化县域耕地生态补偿, 对河北省环京津地区耕地生态补偿问题进行了研究。研究结果表明: 1)环京津耕地“生态供给”与“生态消费”存在典型的“空间异位”现象, 其中耕地“生态消费”主要集中于环京津南部地区, 而耕地“生态供给”则主要集中在环京津西部地区。2)2014 年环京津耕地生态系统服务价值为4.480 5×1010 元, 整体呈现不能自足的态势, 总赤字金额为7.834×109 元。3)环京津地区中,河北省张北县、兴隆县、蔚县、尚义县和涞源县等17 个县市表现为盈余, 其余各县市均呈现为赤字状态。其中滦南县需支付的耕地生态补偿量最高, 为5.173×107 元, 其次为玉田县和东光县, 分别为4.864×107 元和4.849×107 元。虽然遵化市、滦平县、曲阳县可获得补偿, 但其耕地生态条件也仅仅表现为“紧平衡”, 仍需受到广泛关注。4)以粮食安全角度为出发点, 2014 年环京津区急需获得耕地生态补偿的县为张北县、蔚县、尚义县、阳原县、涞源县和曲阳县, 需支付耕地生态补偿的县为滦南县、玉田县、献县、吴桥县和定州市, 唐县、涞水县和丰宁满族自治县既不需获得也不需支付耕地生态补偿。与前人的研究相比, 本研究以生态补偿为切入点对河北省环京津地区耕地补偿问题进行了研究, 研究结果对于促进环京津地区经济发展、耕地生态环境保护与耕地资源持续高效利用发挥着重要作用。同时, 此方法的运用可为类似地区生态补偿的量化研究提供参考, 为其他地区以生态价值量确定耕地保护指标提供依据, 对耕地生态补偿价值机制的研究有指导意义。
Hits
YES
Hits
Downloads
Comment