leaf_engine.etl.load
====================

.. py:module:: leaf_engine.etl.load


Functions
---------

.. autoapisummary::

   leaf_engine.etl.load.api_insert
   leaf_engine.etl.load.graph_insert
   leaf_engine.etl.load.insert_records
   leaf_engine.etl.load.load_pipeline


Module Contents
---------------

.. py:function:: api_insert(df: pandas.DataFrame, company_id: int, company_name: str, batch_date: str, run_type: str, record_type: str, dry_run: bool = False, overwrite: bool = False)

   Performs mapping, filtering, transforming and inserting of records.

   Takes in the shipments dataframe, maps it to the OpenAPI schema, filters out
   any UUIDs that already exist in the database, reshapes the mapped df to a list of
   dicts (one per row), and validates each dict using the OpenAPI schema.

   Inserts records into the database once they've all passed
   validation.


.. py:function:: graph_insert(company_id: int, dry_run: bool, overwrite: bool) -> None

.. py:function:: insert_records(df: pandas.DataFrame, table_name: str, company_id: int, batch_date: str, record_type: str, dry_run: bool, logger: logging.Logger, crud_caller: leaf_engine.io.api.crud_caller.LeafCRUDCaller)

.. py:function:: load_pipeline(df: pandas.DataFrame, run_type: str, dry_run: bool = False, overwrite: bool = False) -> pandas.DataFrame