API Reference

API Reference#

Pipelines#

pystagegate.pipelines.national_profile(config: dict | str) → tuple[DataFrame, DataFrame]#

Compute year-on-year squared differences in final migration data.

Args:: config (dict | str): Configuration dictionary or path to configuration file.
Returns:: tuple[pd.DataFrame, pd.DataFrame]: DataFrames containing year-on-year squared differences and adjusted squared differences.

pystagegate.pipelines.prov_fin_main(config: dict | str) → DataFrame#

Compute sums of squared differences between provisional and final migration data.

Args:: config (dict | str): Configuration dictionary or path to configuration file.
Returns:: pd.DataFrame: DataFrame containing sums of squared differences between provisional and final migration data.

pystagegate.pipelines.sex_ratio_main(config: dict | str) → tuple[DataFrame, DataFrame, DataFrame]#

Compute sex ratio analysis for final migration data, including national and local authority levels.

Args:: config (dict | str): Configuration dictionary or path to configuration file.
Returns:: tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]: DataFrames containing sex ratios, national sex ratios, and squared differences.

Provisional-Final analysis#

pystagegate.prov_fin.merge_final_migration_data(immigration_df: DataFrame, emigration_df: DataFrame, config: dict, sex_ratio: bool = False) → DataFrame#

Merge immigration and emigration dataframes on specified columns and calculate net migration.

Args:: immigration_df (pd.DataFrame): The immigration DataFrame. emigration_df (pd.DataFrame): The emigration DataFrame. config (dict): A dictionary configuration.
Returns:: merged_df (pd.DataFrame): A pandas DataFrame containing the merged data with net migration.

pystagegate.prov_fin.nation_breakdown_sqdiff(df: DataFrame, config: dict, nation: str) → tuple[DataFrame, DataFrame]#

Performs aggregations and squared difference calculations by age and Local Authority on national profiles.

Args:

df (pd.DataFrame): The input migration DataFrame. config (dict): A dictionary configuration. nation (str): A one letter code for nation, must be one of ‘E’, ‘S’, ‘W’.

Returns:

tuple:

age_agg (pd.DataFrame): Migration data aggregated by age.
la_agg (pd.DataFrame): Migration data aggregated by age and local authority.

pystagegate.prov_fin.provisional_scot_aggregate(provisional_scot_df: DataFrame, config: dict) → DataFrame#

Aggregate the provisional Scotland migration data by summing counts for each combination of local authority, year, direction, and age.

Args:: provisional_scot_df (pd.DataFrame): The provisional Scotland migration DataFrame. config (dict): A dictionary configuration.
Returns:: aggregate_df (pd.DataFrame): A pandas DataFrame containing the aggregated data.

pystagegate.prov_fin.provisional_scot_cartesian_merge(provisional_scot_df: DataFrame, config: dict) → DataFrame#

Create a cartesian product of unique values for the provisional Scotland data and merge it with the original DataFrame.

Args:: provisional_scot_df (pd.DataFrame): The provisional Scotland migration DataFrame. config (dict): A dictionary configuration.
Returns:: merged_df (pd.DataFrame): A pandas DataFrame containing the cartesian product merged with the original data, with missing values filled with 0.

pystagegate.prov_fin.squared_difference(df: DataFrame, type_1: str, type_2: str, type_1_total: str, type_2_total: str, output_name: str = 'output') → DataFrame#

Create squared difference estimates for migration data using two estimate types and their totals.

Args:: df (pd.DataFrame): The input migration DataFrame. type_1 (str): The first estimate column name. type_2 (str): The second estimate column name. type_1_total (str): The first estimate total column name. type_2_total (str): The second estimate total column name. output_name (str): Substring to denote the outputted squared difference columns. Defaults to “output”.
Returns:: df (pd.DataFrame): A pandas DataFrame containing the difference and squared difference estiamtes.

pystagegate.prov_fin.subset_provisional_data(provisional_df: DataFrame, config: dict) → DataFrame#

Subset the provisional migration data for the specified year and rename columns.

Args:: provisional_df (pd.DataFrame): The provisional migration DataFrame. config (dict): A dictionary configuration.
Returns:: subset_df (pd.DataFrame): A pandas DataFrame containing the subsetted and renamed data.

Sex Ratio analysis#

pystagegate.sex_ratio.clean_and_mask(sex_ratio_df: DataFrame, config: dict) → DataFrame#

Clean and mask the sex ratio DataFrame.

Args:: sex_ratio_df (pd.DataFrame): The DataFrame containing sex ratio data. config (dict): Configuration dictionary.
Returns:: pd.DataFrame: The cleaned and masked DataFrame.

pystagegate.sex_ratio.compute_sex_ratio(sex_ratio_df: DataFrame, config: dict, caps: tuple[float, float] = None, mask: bool = True, male_sex: str = 'Male', female_sex: str = 'Female') → DataFrame#

Compute sex ratios for immigration and emigration across both configured years.

Args:: sex_ratio_df (pd.DataFrame): Pivoted DataFrame with (measure, sex, year) columns. config (dict): Configuration dictionary containing global parameters. caps (tuple[float, float], optional): Lower and upper caps for sex ratio. Defaults to None. mask (bool, optional): Whether to apply quality masking to the sex ratio calculations. Defaults to True.
Returns:: pd.DataFrame: The DataFrame with added sex ratio columns for each migration type and year.

pystagegate.sex_ratio.pivot_sex_ratio_frame(sex_ratio_df: DataFrame, config: dict)#

Pivot the sex ratio DataFrame over sex and year.

Args:: sex_ratio_df (pd.DataFrame): The DataFrame containing sex ratio data. config (dict): Configuration dictionary.
Returns:: pd.DataFrame: A pivoted DataFrame

pystagegate.sex_ratio.sex_ratio_helper(sex_ratio_df: DataFrame, migration: str, year: int, caps: tuple[float, float] = None, mask: bool = True, male_sex: str = 'Male', female_sex: str = 'Female') → DataFrame#

Helper function to compute sex ratio for a given migration type and year.

Args:: sex_ratio_df (pd.DataFrame): Pivoted DataFrame with (measure, sex, year) columns. migration (str): Migration type, either ‘imm’ or ‘em’. year (int): Year for which to compute the sex ratio. caps (tuple[float, float], optional): Lower and upper caps for sex ratio. Defaults to None. mask (bool, optional): Whether to apply quality masking to the sex ratio calculations. Defaults to True.
Returns:: pd.DataFrame: The DataFrame with the added sex ratio column for the specified migration type and year.

pystagegate.sex_ratio.sex_ratio_ssq(la_df: DataFrame, national_df: DataFrame, config: dict) → DataFrame#

Compute sum of squared differences in sex ratios for national vs local authority data.

Args:: la_df (pd.DataFrame): DataFrame containing local authority sex ratio data. national_df (pd.DataFrame): DataFrame containing national sex ratio data.
Returns:: pd.DataFrame: A DataFrame containing the squared sex ratio for local authorities compared to the national average.

pystagegate.sex_ratio.year_agg_merge(merged_df: DataFrame, config: dict) → DataFrame#

Aggregates year totals for specified years and merges them for comparison.

Args:: merged_df (pd.DataFrame): The merged DataFrame containing immigration and emigration data. config (dict): Configuration dictionary containing dataset variable mappings and global parameters.
Returns:: pd.DataFrame: A DataFrame containing the squared differences for immigration, emigration, and net migration counts between the two specified years.

pystagegate.sex_ratio.year_agg_sqdiff(merged_df: DataFrame, config: dict) → tuple[DataFrame, DataFrame]#

Computes the squared differences for immigration, emigration, and net migration counts between two specified years and aggregates the results.

Args:: merged (pd.DataFrame): The merged DataFrame containing immigration and emigration data. config (dict): Configuration dictionary containing dataset variable mappings and global parameters.
Returns:: tuple[pd.DataFrame, pd.DataFrame]: A tuple containing the merged DataFrame with squared differences and the aggregated DataFrame with scaled sums of squared differences.

Utilities#

pystagegate.utils.load_config(conf_id: str | dict) → dict#

Load a JSON configuration file.

Args:: conf_id (str | dict): The file path to a JSON file or a dict object
Returns:: config (dict): Configuration dictionary.

pystagegate.utils.load_summary_data(config: dict, dataset_key: str) → DataFrame#

Load and validate summary data from a CSV file.

Args:: config (dict): A dictionary configuration. dataset (str): A string key value for the dataset to load.
Returns:: df (pd.DataFrame): A pandas DataFrame containing the selected data.

pystagegate.utils.write_outputs(output_path: str, output_pairs: list[tuple[DataFrame, str]] | tuple[DataFrame, str]) → None#

Write output DataFrames to CSV files.

Args:: output_path (str): The directory path where the output files will be saved. output_pairs (list[tuple[pd.DataFrame, str]] | tuple[pd.DataFrame, str]): A list of tuples or a single tuple containing a DataFrame and the corresponding output file name.
Returns:: None

Validation#

pystagegate.validate.validate(df: DataFrame, df_key: str, config: dict) → None#

Validate a DataFrame against the schema defined in the configuration.

Args:: df (pd.DataFrame): DataFrame to validate. df_key (str): Key identifying the dataset in the configuration. config (dict): Configuration dictionary containing validation rules.
Raises:: ValueError: If the dataset key is not found in the configuration. pandera.errors.SchemaError: If the DataFrame does not conform to the schema.
Returns:: None