Skip to content

Python Delta Local Pipeline

This article provides a guide on how to execute a simple Delta Table copy locally without Spark using the RTDIP SDK. This pipeline was tested on an M2 Macbook Pro using VS Code in a Python (3.10) environment.


This pipeline job requires the packages:


Name Description
PythonDeltaSource Reads data from a Delta Table.
PythonDeltaDestination Writes to a Delta table.


Below is an example of how to read from and write to Delta Tables locally without the need for Spark

from import PythonDeltaSource
from import PythonDeltaDestination

source = PythonDeltaSource("{/path/to/source/table}").read_batch()

destination = PythonDeltaDestination(source, "{/path/to/destination/table}", mode="append").write_batch()