Those of you who have already the chance of integrating a huge amount of data as Smart Meters have already faced situations when the Data is too huge to consider it with sample data and extrapole it to the whole flow rate.
It is also the case whenever your data is not perfect. And most of the time, your data is not what you expected, the data is not perfect. The sensors are not perfect, receiving SMS or radio data is never perfect.
You need to know better your data quality speaking or quantity speaking. You will need to control its consistency.
But the process data will not make it alone. We will need to use other tools: Analytics and Statistics are our friends. You will make your own specific program or integrate a Business Intelligence tool or
But how to make a real communication, efficiently, between your data and tools.
For the PI System, there is one solution to help the CIO, Business Analyst and IT Engineers to communicate around the process data and the calculated data.
The Data Wrangling : data cleaning matters.
PI Integrator is the solution that helps interact massively.
But is not miracle, we will not to practice, involve scientists to understand the business analysts and work together including IT Team.