基于以太网机群的海量高光谱数据处理系统平台设计

Platform design for massive hyperspectral data processing based on internet cluster

摘要: 在一些大型图像处理系统中，通过构建服务器机群网络，可实现多台服务器并行处理图像数据流。针对高光谱遥感数据及处理算法特点，设计了基于以太网的服务器机群海量高光谱数据处理系统平台，平台由7台服务器和1台千兆网交换机构组成，其中选择CPU核数不低于8核的4台HP刀片服务器作为并行处理服务器，在服务器机群软件设计中，通过任务分解、算法流水化处理及网络负载均衡等方法来实现并行处理。该系统已应用在某遥感数据处理系统中，多台服务器同时写入SAN盘阵，可实现6 000 Mbps的写入能力。

Abstract: In some large-scale image processing systems, multiple servers can achieve parallel processing image data stream to complete the complex data operations by building server cluster network. An Ethernet-based server cluster for huge hyperspectral image processing system was designed based on image characteristics. In order to achieve parallel processing, the task decomposition, algorithm pipelined processing and network load balancing were used in software design. The system is used in a hyperspectral image data processing system and the date rate can achieve 6 000 Mbps.