leaf_engine.etl.transform

Functions to transform the shipments DataFrame.

Includes functions for transforming the shipments DataFrame output by the ETL pipeline into DataFrames that can be inserted into the analytics DB via the API.

Functions

`to_execution_graph_shipments`(shipments_df)	Transforms execution shipments to fit execution graph data model.
`to_graph_lanes`(→ pandas.DataFrame)	Transforms company network lanes to fit graph data model.
`to_locations`(→ pandas.DataFrame)
`to_raw_lanes`(→ pandas.DataFrame)
`to_raw_shipments`(→ pandas.DataFrame)
`to_records`(→ List[dict])	Transforms a DataFrame to a list of dictionaries.

Module Contents

leaf_engine.etl.transform.to_execution_graph_shipments(shipments_df: pandas.DataFrame)

Transforms execution shipments to fit execution graph data model.

Parameters:

shipments_df (pd.DataFrame) – DataFrame containing execution shipments.
analytics.v_execution_shipment. (See) –

Returns:

DataFrame with columns that fit the graph data model.

Return type:

pd.DataFrame

leaf_engine.etl.transform.to_graph_lanes(lanes_df: pandas.DataFrame) → pandas.DataFrame

Transforms company network lanes to fit graph data model.

Parameters:

lanes_df (pd.DataFrame) – DataFrame generated by calling the Postgres function
LeafCRUDCaller.get_company_network_lanes. (analytics.get_company_network_lanes. See) –

Returns:

DataFrame with columns that fit the graph data model.

Return type:

pd.DataFrame

leaf_engine.etl.transform.to_locations(df: pandas.DataFrame) → pandas.DataFrame

Parameters:: df (pandas.DataFrame) –
Return type:: pandas.DataFrame

leaf_engine.etl.transform.to_raw_lanes(df: pandas.DataFrame, company_id: int, batch_date: str, record_type: str, db_cols: List[str]) → pandas.DataFrame

Parameters:

df (pandas.DataFrame) –
company_id (int) –
batch_date (str) –
record_type (str) –
db_cols (List[str]) –

Return type:

pandas.DataFrame

leaf_engine.etl.transform.to_raw_shipments(df: pandas.DataFrame, company_id: int, batch_date: str, record_type: str, db_cols: List[str]) → pandas.DataFrame

Parameters:

df (pandas.DataFrame) –
company_id (int) –
batch_date (str) –
record_type (str) –
db_cols (List[str]) –

Return type:

pandas.DataFrame

leaf_engine.etl.transform.to_records(df: pandas.DataFrame) → List[dict]

Transforms a DataFrame to a list of dictionaries.

Removes all non-null/inf values.

Parameters:: df (pandas.DataFrame) –
Return type:: List[dict]