load_data¶

This module contains functions for loading and processing data from JSON and Excel files.

prepshot.load_data.check_schema(params_info)[source]¶

Validate that params.json declares a compatible _schema_version.

Raises a RuntimeError with a clear migration hint if the file is missing the stamp or carries a different version than this release supports.

Parameters: params_info (dict) -- Parsed contents of params.json.
Return type: None

prepshot.load_data.compute_cost_factors(data_store)[source]¶

Calculate cost factors for various transmission investment and operational costs.

Parameters: data_store (dict) -- Dictionary containing loaded parameters.
Return type: None

prepshot.load_data.extract_config_data(config_data)[source]¶

Extract necessary data from configuration settings.

Parameters: config_data (dict) -- Configuration data for the model.
Returns: Dictionary containing necessary configuration data.
Return type: dict

prepshot.load_data.extract_sets(data_store)[source]¶

Extract simple sets from loaded parameters.

Parameters: data_store (dict) -- Dictionary containing loaded parameters.
Return type: None

prepshot.load_data.load_excel_data(input_folder, params_info, data_store)[source]¶

Load input data based on the provided parameters.

The function dispatches on "format":

"format": "long" (default) -- load from a .csv file in tidy form (dimension columns first, value column last). See read_long_csv().
"format": "table" -- load from a .csv file with multiple value columns, returned as a DataFrame so consumers can use groupby, column-by-name access, etc.

Each entry may also declare "required": false and a "default" value. If the file for an optional parameter is missing, the loader silently substitutes the default (or an empty dict if none is given) and logs a debug message. Required parameters with missing files still terminate the process.

The legacy function name is kept for backwards-compatible imports; despite the _excel_ in the name, all on-disk inputs are CSV as of v1.5.0.

Parameters

input_folder (str) -- Path to the input folder.
params_info (dict) -- Dictionary containing parameter names and their corresponding file information.
data_store (dict) -- Dictionary to store loaded data.

Return type

None

prepshot.load_data.load_json(file_path)[source]¶

Load data from a JSON file.

Parameters: file_path (str) -- Path to the JSON file.
Returns: Dictionary containing data from the JSON file.
Return type: dict

prepshot.load_data.process_data(params_info, input_folder)[source]¶

Load and process data from input folder based on parameters settings.

Parameters

params_info (dict) -- Dictionary containing parameters information.
input_folder (str) -- Path to the input folder.

Returns

Dictionary containing processed parameters.

Return type

dict

prepshot.load_data.read_long_csv(filename, dropna=True)[source]¶

Read a long-format ("tidy") CSV input file.

The convention is: dimension columns first, value column last. For example a 2-dim input carbon_tax looks like:

zone,year,value
BA1,2020,0
BA1,2025,5
BA2,2020,0

The returned dict matches the shape produced by the wide-format reader for the same parameter:

1 dimension column -> {key: value} (scalar keys)
2+ dimension columns -> {(d1, d2, ...): value} (tuple keys)

The ORDER of the dimension columns in the CSV determines the order of elements in the output keys, so model-side lookups work unchanged regardless of which format the file is in on disk.

Parameters

filename (str) -- Path to the CSV file.
dropna (bool) -- If True, rows with any NaN are dropped before keying.

Returns

Mapping from dimension key (or tuple of keys) to the value.

Return type

dict