leaf_engine.etl.concat
======================

.. py:module:: leaf_engine.etl.concat

.. autoapi-nested-parse::

   Concatenates multiple datasets into a single shipments DataFrame, and post-
   concatenation deduplication logic (either custom logic, implemented in a
   shipper's `custom_drop_duplicates.py` or the default logic).



Functions
---------

.. autoapisummary::

   leaf_engine.etl.concat._custom_drop_duplicates
   leaf_engine.etl.concat._default_drop_lane_duplicates
   leaf_engine.etl.concat._default_drop_shipment_duplicates
   leaf_engine.etl.concat._describe_lane_datasets
   leaf_engine.etl.concat._describe_shipment_datasets
   leaf_engine.etl.concat._drop_duplicates
   leaf_engine.etl.concat.concat_pipeline


Module Contents
---------------

.. py:function:: _custom_drop_duplicates(df, module_name) -> pandas.DataFrame

.. py:function:: _default_drop_lane_duplicates(df) -> pandas.DataFrame

.. py:function:: _default_drop_shipment_duplicates(df) -> pandas.DataFrame

.. py:function:: _describe_lane_datasets(df)

.. py:function:: _describe_shipment_datasets(df)

.. py:function:: _drop_duplicates(df: pandas.DataFrame, lane_level) -> pandas.DataFrame

.. py:function:: concat_pipeline(input_dfs: List[pandas.DataFrame], lane_level=False) -> pandas.DataFrame