mlm_insights.mlm_dask.interfaces package

Submodules

mlm_insights.mlm_dask.interfaces.dask_data_reader module

class mlm_insights.mlm_dask.interfaces.dask_data_reader.DaskDataReader(name: str, file_path: List[str] | str = '', data_source: DataSource | None = None, **kwargs: Any)

Bases: DataReader

Data Reader base class for Dask execution engine. Dask-based readers must derive from this class and implement the abstract methods.

abstract read(schema_provider: SchemaProvider, **kwargs: Any) Any

Read the data from the data-source and return engine specific data frame.

Parameters

schema_providerSchemaProvider

Engine specific schema provider. Use it to call get_engine_schema() to get engine specific schema to pass to the respective read methods

kwargs:

Key-value pair for dynamic arguments

Returns

dataframe: Any

Engine specific DataFrame

Module contents