Skip to content

Weather Forecast V1 Multi

SparkWeatherCompanyForecastAPIV1MultiSource

Bases: SparkWeatherCompanyForecastAPIV1Source

The Weather Forecast API V1 Multi Source is used to read 15 days forecast from the Weather API. It allows to pull weather data for multiple stations and returns all of them in a single DataFrame.

URL for one station: https://api.weather.com/v1/geocode/32.3667/-95.4/forecast/hourly/360hour.json

It takes a list of Weather Stations. Each station item must contain comma separated Latitude & Longitude.

Examples

["32.3667,-95.4", "51.52,-0.11"]

Parameters:

Name Type Description Default
spark SparkSession

Spark Session instance

 required
options dict

A dictionary of ISO Source specific configurations (See Attributes table below).

 required

Attributes:

Name Type Description
stations list[str]

List of Weather Stations.
api_key str

Weather API key.
language str

API response language. Defaults to en-US.
units str

Unit of measurements. Defaults to e.
Source code in src/sdk/python/rtdip_sdk/pipelines/sources/spark/the_weather_company/weather_forecast_api_v1_multi.py 
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
class SparkWeatherCompanyForecastAPIV1MultiSource(
    SparkWeatherCompanyForecastAPIV1Source
):
    """
    The Weather Forecast API V1 Multi Source is used to read 15 days forecast from the Weather API. It allows to
    pull weather data for multiple stations and returns all of them in a single DataFrame.

    URL for one station: <a href="https://api.weather.com/v1/geocode/32.3667/-95.4/forecast/hourly/360hour.json">
    https://api.weather.com/v1/geocode/32.3667/-95.4/forecast/hourly/360hour.json</a>

    It takes a list of Weather Stations. Each station item must contain comma separated Latitude & Longitude.

    Examples
    --------
    `["32.3667,-95.4", "51.52,-0.11"]`

    Parameters:
        spark (SparkSession): Spark Session instance
        options (dict): A dictionary of ISO Source specific configurations (See Attributes table below).

    Attributes:
        stations (list[str]): List of Weather Stations.
        api_key (str): Weather API key.
        language (str): API response language. Defaults to `en-US`.
        units (str): Unit of measurements. Defaults to `e`.
    """

    spark: SparkSession
    options: dict
    spark_schema = WEATHER_FORECAST_SCHEMA
    required_options = ["stations", "api_key"]

    def __init__(self, spark: SparkSession, options: dict) -> None:
        super(SparkWeatherCompanyForecastAPIV1MultiSource, self).__init__(
            spark, options
        )
        self.spark = spark
        self.options = options
        self.stations = self.options.get("stations", [])
        self.api_key = self.options.get("api_key", "").strip()
        self.language = self.options.get("language", "en-US").strip()
        self.units = self.options.get("units", "e").strip()

    def _pull_data(self) -> pd.DataFrame:
        """
        Pulls data from the Weather API and parses the JSON file for multiple stations

        Returns:
            Raw form of data.
        """

        result_df = None
        for station in self.stations:
            parts = station.split(",")
            lat, lon = parts

            df = self._pull_for_weather_station(lat, lon)
            df["latitude"] = lat
            df["longitude"] = lon

            if result_df is not None:
                result_df = pd.concat([result_df, df])
            else:
                result_df = df

        return result_df

    def _validate_options(self) -> bool:
        for station in self.stations:
            parts = station.split(",")

            if len(parts) != 2 or parts[0].strip() == "" or parts[1].strip() == "":
                raise ValueError(
                    f"Each station item must contain comma separated Latitude & Longitude. Eg: 10.23:45.2"
                )

        return True