leaf_engine.adapt.load
This module contains function to load standard Adapt outputs into their corresponding data stores.
Attributes
Functions
|
Filters moves with lanes that are not in the analytics.lane table or that |
|
|
|
|
|
|
|
Inserts lane plans into DB. |
|
Inserts lane quick ref into DB. |
|
Reads, transforms, filters, validates network moves and makes insertion |
|
Inserts observation patterns into DB. |
|
Inserts should be flex into DB. |
|
|
|
Reads carrier churn file into pd.DataFrame. |
|
Reads consolidated flex file into pd.DataFrame. |
|
|
|
Reads lane_plan file into pd.DataFrame. |
|
Reads lane quick ref file into pd.DataFrame. |
|
Reads network moves file into pd.DataFrame. |
|
Reads observation patterns file into pd.DataFrame. |
|
Reads should be flex file into pd.DataFrame. |
|
Transforms network moves to match DB constraints. |
Module Contents
- leaf_engine.adapt.load.filter_network_moves(moves_df: pandas.DataFrame, company_id: int, batch_date: str, incl_record_types: list[str]) pandas.DataFrame
Filters moves with lanes that are not in the analytics.lane table or that have been previously inserted.
- Parameters:
moves_df (pandas.DataFrame) –
company_id (int) –
batch_date (str) –
- Return type:
- leaf_engine.adapt.load.insert_carrier_churn(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False) None
- leaf_engine.adapt.load.insert_consolidated_flex(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False) None
- leaf_engine.adapt.load.insert_lane_adapt_detail(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False)
- leaf_engine.adapt.load.insert_lane_plans(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False) pandas.DataFrame
Inserts lane plans into DB.
Read the lane plans file(s) from the company’s data directory.
- leaf_engine.adapt.load.insert_lane_quick_ref(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False) None
Inserts lane quick ref into DB.
Read the lane quick ref file(s) from the company’s data directory.
- leaf_engine.adapt.load.insert_network_moves(company_id: int, company_name: str, batch_date: str, equipment_class: str, record_type: str, incl_record_types: list[str], dry_run: bool = False) None
Reads, transforms, filters, validates network moves and makes insertion calls.
Before inserting records, filters existing moves from the same batch_date from moves DataFrame, then deletes existing moves for company_id from batches other than batch_date.
- leaf_engine.adapt.load.insert_observations_patterns(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False) None
Inserts observation patterns into DB.
Read the observation patterns file(s) from the company’s data directory.
- leaf_engine.adapt.load.insert_should_be_flex(company_id: int, company_name: str, batch_date: str, equipment_class: str, dry_run: bool = False) None
Inserts should be flex into DB.
Read the should be flex file(s) from the company’s data directory.
- leaf_engine.adapt.load.is_allowed_column(col_name)
- leaf_engine.adapt.load.read_carrier_churn(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads carrier churn file into pd.DataFrame.
- Parameters:
- Return type:
- leaf_engine.adapt.load.read_consolidated_flex(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads consolidated flex file into pd.DataFrame.
- Parameters:
- Return type:
- leaf_engine.adapt.load.read_lane_adapt_detail(company_id: int, company_name: str, batch_date: str, equipment_class: str)
- leaf_engine.adapt.load.read_lane_plans(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads lane_plan file into pd.DataFrame.
- Parameters:
- Return type:
- leaf_engine.adapt.load.read_lane_quick_ref(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads lane quick ref file into pd.DataFrame.
- Parameters:
- Return type:
- leaf_engine.adapt.load.read_network_moves(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads network moves file into pd.DataFrame.
- Parameters:
- Return type:
- leaf_engine.adapt.load.read_observations_patterns(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads observation patterns file into pd.DataFrame.
Combines weekly and daily since the data is stored in separate files but the schema is the same. Only the format of the pattern is different.
- Parameters:
- Return type:
- leaf_engine.adapt.load.read_should_be_flex(company_id: int, company_name: str, batch_date: str, equipment_class: str) pandas.DataFrame
Reads should be flex file into pd.DataFrame.
- Parameters:
- Return type:
- leaf_engine.adapt.load.remove_unwanted_columns(df)
- leaf_engine.adapt.load.transform_network_moves(moves_df: pandas.DataFrame, company_id: str, batch_date: str, equipment_class: str, record_type: str, inplace: bool = True) pandas.DataFrame
Transforms network moves to match DB constraints.
NOTE that by default these transformations are in-place: the DataFrame passed as the first argument will be mutated (this is because creating a copy will have a high memory cost). Use the inplace parameter to change this behavior.
- Parameters:
- Return type:
- leaf_engine.adapt.load.LaneAdaptDetaiLoadException
- leaf_engine.adapt.load.LanePlansLoadException
- leaf_engine.adapt.load.NetworkMoveLoadException
- leaf_engine.adapt.load.caller
- leaf_engine.adapt.load.logger
- leaf_engine.adapt.load.planning_caller