RTDIP and LF Energy
By providing frameworks and reference implementations, LF Energy minimizes pain points such as cybersecurity, interoperability, control, automation, virtualization, and the orchestration of supply and demand.
What is Real Time Data Ingestion Platform
Organizations need data for day-to-day operations and to support advanced Data Science, Statistical and Machine Learning capabilities such as Optimization, Surveillance, Forecasting, and Predictive Analytics. Real Time Data forms a major part of the total data utilized in these activities.
Real time data enables organizations to detect and respond to changes in their systems thus improving the efficiency of their operations. This data needs to be available in scalable and secure data platforms.
Real Time Data Ingestion Platform (RTDIP) is the solution of choice leveraging PaaS (Platform as a Service) services along with some custom components to provide Data Ingestion, Data Transformation, and Data Sharing as a platform. RTDIP can interface with several data sources to ingest many different data types which include time series, alarms, video, photos and audio being provided from sources such as Historians, OPC Servers and Rest APIs, as well as data being sent from hardware such as IoT Sensors, Robots and Drones.
Why Real Time Data Ingestion Platform?
Real Time Data Ingestion Platform (RTDIP) enables users to consume Real Time Data at scale, including historical and real time streaming data. RTDIP has proven to be capable of ingesting over 3 million sensors in a production environment across every geographical location in the world.
The Real Time Data Ingestion Platform can be run in a customers own environment, allowing them to accelerate their cloud deployments while leveraging a proven design to scale their time series data needs.
RTDIP also provides a number popular integration options, including:
- Rest API
- Python SDK
These options allow users to integrate with a wide variety of applications and tools, including:
- Data Visualization Tools such as Power BI, Seeq, Tableau, and Grafana
- Data Science Tools such as Jupyter Notebooks, R Studio, and Python
- Data Engineering Tools such as Apache Spark, Apache Kafka, and Apache Airflow
RTDIP is architected to leverage Open Source technologies Apache Spark and Delta. This allows users to leverage the power of Open Source technologies to build their own custom applications and tools in whichever environment they prefer, whether that is in the cloud or on-premise on their own managed Spark Clusters.