A Comparison Method of Massive Power Consumption I

A Comparison Method of Massive Power

Consumption Information Collection Test Data

Based on Improved Merkle Tree

Enguo Zhu1(&),Fangbin Ye2,Jian Dou1,and Chaoliang Wang2

1China Electric Power Research Institute,Beijing,China

krauser3c@https://www.wendangku.net/doc/ac7194440.html,

2State Grid Zhejiang Electric Power Company Electric Power Research

Institute,Hangzhou,China

Abstract.It is necessary to conduct comparison test of massive electricity data

on the uni?ed interface platform for ensuring that the power consumption

information collection system can provide data for kinds of business systems

stably and accurately.In this paper,a method of data comparison for mass data

of power energy data acquire system based on improved Merkle Tree is pro-

posed to solve the problem that the traditional one to one comparison method

cannot adapt to massive data test.Merkle Tree is improved by forming all the

subtrees into complete binary tree.Based on the improved algorithm mentioned

above,the improved Merkle Tree is constructed for the source data from power

consumption information collection system and the publish data from uni?ed

interface platform.Through the comparison of Merkle Root and its sub nodes,a

fast consistency check is achieved,and the optimal data blocking strategy and

multithreading optimization method are put forward.This method greatly

improves the ef?ciency of the uni?ed interface platform for the consistency of

mass data.

Keywords:Power consumption information collectionáHash algorithm

Improved Merkle treeáData consistency

1Introduction

As an important data source of power marketing,power consumption information collection system provides business and data support for many application systems.As the range of data acquisition expands,the volume of data provided to the other systems also increases greatly[1–4].The State Grid Corporation proposed to establish the uni?ed interface platform for power consumption information collection system,so as to achieve the interface data transmission between the power consumption information collection system and other business systems.At this stage,the volume of data transferred is huge,and this data may be used in the important business areas such as electricity settlement.In order to avoid business anomalies caused by data variation,it is necessary to ensure data consistency between source data and interface data to pass data consistency test for uni?ed interface platform[5,6].Because the data provided by the power consumption information collection system has the characteristics of large ?Springer Nature Singapore Pte Ltd.2018

Q.Zhou et al.(Eds.):ICPCSEE2018,CCIS902,pp.401–415,2018.

https://https://www.wendangku.net/doc/ac7194440.html,/10.1007/978-981-13-2206-8_33