leaf_engine.etl.tasks.pull_lighthouse_data

Functions

normalize_lanes(→ pandas.DataFrame)	Normalizes lighthouse lanes data for clustering and UUID pipeline tasks.
pull_lighthouse_data(→ List[pandas.DataFrame])	Pulls lighthouse shipper data for internal run. Filters down to only latest
upload_file(→ None)	Uploads lanes DataFrame to g-drive.

Module Contents

leaf_engine.etl.tasks.pull_lighthouse_data.normalize_lanes(lanes_df: pandas.DataFrame) → pandas.DataFrame

Normalizes lighthouse lanes data for clustering and UUID pipeline tasks.

Parameters:

lanes_df (pd.DataFrame) – Lanes DataFrame.

Returns:

Normalized lanes DataFrame.

Return type:

pd.DataFrame

leaf_engine.etl.tasks.pull_lighthouse_data.pull_lighthouse_data(params) → List[pandas.DataFrame]

Pulls lighthouse shipper data for internal run. Filters down to only latest batch.

Parameters:

adapt_params (dict) – Adapt params.

Returns:

DataFrame’s containing lighthouse lanes.

Return type:

DataFrame

leaf_engine.etl.tasks.pull_lighthouse_data.upload_file(params, lanes_df: pandas.DataFrame) → None

Uploads lanes DataFrame to g-drive.

Parameters:

• params (dict) – Params.

• lanes_df (pd.DataFrame) – Lanes DataFrame.

Returns:

None

Return type:

None