API Reference

Provisional-Final module

pystagegate.prov_fin.load_summary_data(config: dict, dataset_key: str) → DataFrame

Load and validate summary data from a CSV file.

Args:

config (dict): A dictionary configuration. dataset (str): A string key value for the dataset to load.

Returns:

df (pd.DataFrame): A pandas DataFrame containing the selected data.

pystagegate.prov_fin.merge_final_migration_data(immigration_df: DataFrame, emigration_df: DataFrame, config: dict) → DataFrame

Merge immigration and emigration dataframes on specified columns and calculate net migration.

Args:

immigration_df (pd.DataFrame): The immigration DataFrame. emigration_df (pd.DataFrame): The emigration DataFrame. config (dict): A dictionary configuration.

Returns:

merged_df (pd.DataFrame): A pandas DataFrame containing the merged data with net migration.

pystagegate.prov_fin.provisional_scot_aggregate(provisional_scot_df: DataFrame, config: dict) → DataFrame

Aggregate the provisional Scotland migration data by summing counts for each combination of local authority, year, direction, and age.

Args:

provisional_scot_df (pd.DataFrame): The provisional Scotland migration DataFrame. config (dict): A dictionary configuration.

Returns:

aggregate_df (pd.DataFrame): A pandas DataFrame containing the aggregated data.

pystagegate.prov_fin.provisional_scot_cartesian_merge(provisional_scot_df: DataFrame, config: dict) → DataFrame

Create a cartesian product of unique values for the provisional Scotland data and merge it with the original DataFrame.

Args:

provisional_scot_df (pd.DataFrame): The provisional Scotland migration DataFrame. config (dict): A dictionary configuration.

Returns:

merged_df (pd.DataFrame): A pandas DataFrame containing the cartesian product merged with the original data, with missing values filled with 0.

pystagegate.prov_fin.regional_breakdown(df: DataFrame, config: dict, nation: str = None) → tuple[DataFrame, DataFrame]

Performs aggregations by Age and Local Authority on both a whole country (GB) profile and nation profile.

Args:

df (pd.DataFrame): The input migration DataFrame. config (dict): A dictionary configuration. nation (str, optional): A one letter code for nation, must be one of ‘E’, ‘S’, ‘W’. Defaults to None for GB analysis.

Returns:

tuple:

• age_agg (pd.DataFrame): Migration data aggregated by age.

• la_agg (pd.DataFrame): Migration data aggregated by age and local authority.

pystagegate.prov_fin.squared_difference(df: DataFrame, prefix: str, prov_col: str, fin_col: str) → DataFrame

Create squared difference estimates for migration data using provisional and final estimates.

Args:

df (pd.DataFrame): The input migration DataFrame. prov_col (str): The provisional estimate column name. fin_col (str): The final estimate column name.

Returns:

df (pd.DataFrame): A pandas DataFrame containing the difference and squared difference estiamtes.

pystagegate.prov_fin.subset_provisional_data(provisional_df: DataFrame, config: dict) → DataFrame

Subset the provisional migration data for the specified year and rename columns.

Args:

provisional_df (pd.DataFrame): The provisional migration DataFrame. config (dict): A dictionary configuration.

Returns:

subset_df (pd.DataFrame): A pandas DataFrame containing the subsetted and renamed data.

Utility module

pystagegate.utils.join_paths(root: str, paths: dict) → dict

Join the folder root from the config to a dict of paths from the config

Args:

root (str): The root value from the config. paths (dict): A paths value from the config.

Returns:

joined_paths (dict): A new dict with the joined root and path values.

pystagegate.utils.load_config(path: str) → dict

Load a JSON configuration file.

Args:

path (str): The file path to the JSON file.

Returns:

config (dict): The loaded JSON configuration as a dictionary.

Validation module

pystagegate.validate.create_gx_context(df: DataFrame, df_key: str)

Create a Great Expectations context for the given DataFrame.

Args:

df (pd.DataFrame): The DataFrame to validate. df_key (str): The key for the DataFrame in the configuration.

Returns:

tuple: A tuple containing the Great Expectations context, expectation suite, and batch definition.

pystagegate.validate.prov_fin_validate(df: DataFrame, df_key: str, config: dict)

Validate the given DataFrame against the configuration.

Args:

df (pd.DataFrame): The DataFrame to validate. df_key (str): The key for the DataFrame in the configuration. config (dict): The configuration dictionary containing validation rules.

Returns:

validation_results: Validation result object.