Skip to content

Databricks Secret Scope

DatabricksSecrets

Bases: SecretsInterface

Reads secrets from Databricks Secret Scopes. For more information about Databricks Secret Scopes, see here.

Parameters:

Name Type Description Default
spark SparkSession

Spark Session required to read data from a Delta table

 required
vault str

Name of the Databricks Secret Scope

 required
key str

Name/Key of the secret in the Databricks Secret Scope

 required
Source code in src/sdk/python/rtdip_sdk/pipelines/secrets/databricks.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
class DatabricksSecrets(SecretsInterface):
    '''
    Reads secrets from Databricks Secret Scopes. For more information about Databricks Secret Scopes, see [here.](https://docs.databricks.com/security/secrets/secret-scopes.html)

    Args:
        spark: Spark Session required to read data from a Delta table
        vault: Name of the Databricks Secret Scope
        key: Name/Key of the secret in the Databricks Secret Scope
    '''    
    spark: SparkSession
    vault: str
    key: str

    def __init__(self, spark: SparkSession, vault: str, key: str):
        self.spark = spark
        self.vault = vault
        self.key = key

    @staticmethod
    def system_type():
        '''
        Attributes:
            SystemType (Environment): Requires PYSPARK on Databricks
        '''        
        return SystemType.PYSPARK_DATABRICKS

    @staticmethod
    def libraries():
        libraries = Libraries()
        return libraries

    @staticmethod
    def settings() -> dict:
        return {}

    def get(self):
        '''
        Retrieves the secret from the Databricks Secret Scope
        '''        
        dbutils = get_dbutils(self.spark)
        return dbutils.secrets.get(scope=self.vault, key=self.key)

    def set(self):
        '''
        Sets the secret in the Secret Scope
        Raises:
            NotImplementedError: Will be implemented at a later point in time
        '''          
        return NotImplementedError

get()

Retrieves the secret from the Databricks Secret Scope

Source code in src/sdk/python/rtdip_sdk/pipelines/secrets/databricks.py 
55
56
57
58
59
60
def get(self):
    '''
    Retrieves the secret from the Databricks Secret Scope
    '''        
    dbutils = get_dbutils(self.spark)
    return dbutils.secrets.get(scope=self.vault, key=self.key)

set()

Sets the secret in the Secret Scope

Raises:

Type Description
NotImplementedError

Will be implemented at a later point in time
Source code in src/sdk/python/rtdip_sdk/pipelines/secrets/databricks.py 
62
63
64
65
66
67
68
def set(self):
    '''
    Sets the secret in the Secret Scope
    Raises:
        NotImplementedError: Will be implemented at a later point in time
    '''          
    return NotImplementedError

system_type() staticmethod

Attributes:

Name Type Description
SystemType Environment

Requires PYSPARK on Databricks
Source code in src/sdk/python/rtdip_sdk/pipelines/secrets/databricks.py 
38
39
40
41
42
43
44
@staticmethod
def system_type():
    '''
    Attributes:
        SystemType (Environment): Requires PYSPARK on Databricks
    '''        
    return SystemType.PYSPARK_DATABRICKS