About two month ago I had been attending Hadoop Summit 2015 in Brussels. Frank Cremer (Port of Rotterdam) and Mansour Raad from ESRI had delivered an interesting presentation on Hadoop’s use to manage output of extensive network of radar and AIS stations:
The Port of Rotterdam is one of the busiest ports in the world. Monitoring and managing the resulting ship traffic is the responsibility of the Rotterdam Port Authorities. Therefore the Port has an extensive network of radar and AIS stations, continuously acquiring positional and telemetry data from all ships within range. This data is stored in Hadoop and processed through map/reduce jobs queried from web maps. Our presentation focuses on specific aspects of this dataset, its processing and its integration into web maps. This integrated approach serves as a showcase for bringing Hadoop`s capabilities to business users through web maps.
This case study is essentially a good model for processing output from myriads of sensors generating continuously a huge amount of data, quite a common problem in military, weather forecasting and SCADA domains. Internet of Things (IoT) is coming and with IoT comes the problem of handling output of multiple sensors. So far, Hadoop is the only only cost-sensible and scalable Big Data management solution capable to withstand massive data assault. I agree with conclusions of Hadoop’s market analysis that rumors of Hadoop’s demise are indeed greatly exaggerated.