q-Ph nddlmZddlmZddlmZddlmZddl m Z erddl m Z mZdddddZddZdS)) annotations)partial) TYPE_CHECKINGNpyarrow) DataFrame LazyFrameT)allow_pyarrow_filter batch_sizedspa.dataset.Datasetr boolr int | Nonereturnr c~tt||}tj|j||S)a' Pickle the partially applied function `_scan_pyarrow_dataset_impl`. The bytes are then sent to the polars logical plan. It can be deserialized once executed and ran. Parameters ---------- ds pyarrow dataset allow_pyarrow_filter Allow predicates to be pushed down to pyarrow. This can lead to different results if comparisons are done with null values as pyarrow handles this different than polars does. batch_size The maximum row count for scanned pyarrow record batches. )r r)r_scan_pyarrow_dataset_implplr _scan_python_functionschema)r r r funcs h/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/polars/io/pyarrow_dataset/anonymous_scan.py_scan_pyarrow_datasetr sB. -rj I I ID < - - 4!5 .   with_columnslist[str] | None predicate str | Nonen_rowsrc ddlm}d}|r4ddlm}m}m} m} ddlm} m } m } t|t| | | ||| | d}||d}|||d<|r||j |fi|S||jd i|S) a Take the projected columns and materialize an arrow table. Parameters ---------- ds pyarrow dataset with_columns Columns that are projected predicate pyarrow expression that can be evaluated with eval n_rows: Materialize only n rows from the arrow dataset batch_size The maximum row count for scanned pyarrow record batches. Returns ------- DataFrame r) from_arrowN) to_py_dateto_py_datetime to_py_timeto_py_timedelta)DateDatetimeDuration)par%r&r'r!r"r#r$)columnsfilterr )polarsr polars._utils.convertr!r"r#r$polars.datatypesr%r&r'evalr(headto_table)r rrrr r _filterr!r"r#r$r%r&r' common_paramss rrr*s 6"!!!!!G             >========= $$("0(#2     !-@@M&0 l# <z'"'&::M::;;; :kbk22M22 3 33r)r r r rr rrr ) r r rrrrrrr rrr) __future__r functoolsrtypingrpolars._reexport _reexportrpolars.dependenciesrr(r,rr rrr+rrr:s"""""" ------,++++++++ "&! :=4=4=4=4=4=4r