RTDIP (Real Time Data Ingestion Platform) is a scalable, industry-standard solution for the ingestion and processing of data from a vast array of sources and provides easy access to high volume, historical and real time process data for analytics applications, engineers, and data scientists wherever they are.
Create streaming and batch ingestion pipelines according to requirements of the source, transforms popular industry formats and outputs data according to the needs of the data consumer.
Enables users to perform complex real time queries on datasets within the platform.
Deploy pipelines to popular orchestration engines to schedule and execute jobs according to your preference.
RTDIP works seamlessly across multicloud platforms.
RTDIP is compliant with the Apache Open Source license, enabling maximum flexibility in adaptation and re-use.
RTDIP can provide datasets for intelligent curation and query by AI agents.
RTDIP Ingestion Pipeline framework has been built to simplify ingesting time series data with the ability to create streaming and batch ingestion pipelines according to the requirements of the source of the data and the needs of the consumer.
RTDIP data ingestion can include time series, alarms, video, photos, audio, and input from hardware including IoT Sensors, Robots, and Drones. Sources are components that connect to source systems and extract data from them.
RTDIP has the components to perform transformations on your data. These will target certain data models and common transformations that sources or destination components require to be performed on data before it can be ingested or consumed.
RTDIP destination components allow for connecting and writing data to destination systems including Delta, Eventhub, Kafka, Kinesis and Rest API.
Use RTDIP’s Query Builder feature to perform complex real time queries, including aggregation on datasets within the platform.
Raw facilitates performing raw extracts of time series data.
Latest queries provides the latest event values.
Resample enables changing the frequency of time series observations.
Interpolate takes Resample one step further to estimate the values of unknown data points that fall between existing, known data points. The available interpolation methods are forward fill, backward fill and linear interpolaton.
Time Weighted Averages provide an unbiased average when working with irregularly sampled data.
Interpolation at Time works out the linear interpolation at a specific time based on the points before and after.
Circular Averages computes the circular average for samples in a range.
Circular Standard Deviations computes the circular standard deviations for samples assumed to be in the range.
Use RTDIP data to drive intelligent decision making: using your preferred BI tool, understand what needs to be maintained and when, activity on industrial sites, optimal times for power generation, and much more.
Learn More
RTDIP is being used on a global scale and is backed by trusted partners including Shell, Databricks, Innowatts and LF Energy. Leverage the benefits of a world class solution.
Contribute to the development of this world-class platform. Together, we can build a solution that will revolutionise how data is captured, interpreted and leveraged.
Learn More
RTDIP on GitHub
RTDIP on DockerHub
RTDIP on PyPI
RTDIP on Slack
Copyright © 2016 - 2023 RTDIP
