q-Ph^ddlmZddlZddlZddlZddlZddlmZmZddl m Z m Z m Z m Z ddl mZmZmZddlmZmZddlmZddlmZdd lmZmZmZmZmZmZmZddlm Z!dd l"m#Z$dd l%m&Z&m'Z'dd l(m)Z)m*Z*dd l+m,Z,m-Z-ddl.m/Z/m0Z0m1Z1m2Z2ddl3m4Z4ddl5m6Z6m7Z7ddl8m9Z9ddl:m;Z;ddlZ>ddl?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJddlKmLZLmMZMddlNmOZOmPZPmQZQmRZRmSZSmTZTmUZUmVZVmWZWmXZXmYZYmZZZm[Z[m\Z\m]Z]m^Z^m_Z_m`Z`maZambZbmcZcmdZdmeZemfZfmgZgmhZhddlimjZjddlkmlZlmmZmmnZnddlkmoZpddlkmqZrddlsmtZtddlumvZvddlwmxZxddlymzZzddl{m|Z|dd l}m~Z~mZdd!lmZdd"lmZmZeje5dd#lmZmZdddn #1swxYwYer ddlZdd$lmZmZmZdd%lmZdd&lmZmZdd'l}mZeje5dd(lmZmZdddn #1swxYwYdd)l"mZmZmZdd*l%mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm&Z&mZmZmZmZmZmZmZmZmZmZdd+lkmZdd,lmZdd-lmZejd.kr dd/lmZmZndd/lmZmZejd0krdd1lmZndd1lmZejd2krdd3lm1Z1ndd3lm1Z1ed4Zed5ZdGd9ZdHd=ZdIdDZGdEdFZdS)J) annotationsN) CollectionMapping)datedatetimetime timedelta) lru_cachepartialreduce)BytesIOStringIO)and_)Path) TYPE_CHECKINGAnyCallableClassVarNoReturnTypeVaroverload) functions)ParquetMetadataPartitioningScheme)_AioDataFrameResult_GeventDataFrameResult)negate_duration_stringparse_as_duration_string)deprecate_renamed_parameterdeprecate_streaming_parameter deprecatedissue_deprecation_warning)wrap_parquet_metadata_callback)parse_into_expressionparse_into_list_of_expressions)serialize_polars_object)LazyPolarsSlice)issue_unstable_warningunstable) _is_generatordisplay_dot_graph extend_boolfind_stacklevelis_bool_sequence is_sequence issue_warningnormalize_filepathparse_percentilesqualified_type_namerequire_same_type)wrap_df wrap_expr)DTYPE_TEMPORAL_UNITSN_INFER_DEFAULTBoolean CategoricalDateDatetimeDurationEnumFloat32Float64Int8Int16Int32Int64Int128NullObjectStringTimeUInt8UInt16UInt32UInt64Unknownis_polars_dtypeparse_into_dtype) DataTypeGroup)_PYARROW_AVAILABLEimport_optional subprocess) polars_cloud)pyarrow)PerformanceWarning) CompatLevel) GPUEngine) LazyGroupBy)InProcessQuery)DEFAULT_QUERY_OPT_FLAGSforward_old_opt_flags)Schema)by_dtypeexpand_selector) PyLazyFrameget_engine_affinity) AwaitableIterableSequence)IOBase)IOLiteral) QueryOptFlags)PyExprPyPartitioning) DataFrameDataTypeExpr)AsofJoinStrategyClosedIntervalColumnNameOrSelector CsvQuoteStyle EngineType ExplainFormatFillNullStrategyFrameInitTypesIntoExprIntoExprColumnIpcCompression JoinStrategyJoinValidationLabelMaintainOrderJoin Orientationr PlanStagePolarsDataTypePythonDataTypeQuantileMethodSchemaDefinition SchemaDictSerializationFormatStartBySyncOnCloseMethodUniqueKeepStrategy)numpy)CredentialProviderFunction)ParquetFieldOverwrites) ) Concatenate ParamSpec)r )Self)r )r!TPenginersreturnc.|dkrtn|S)Nauto)rbrs V/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/polars/lazyframe/frame.py_select_enginers$*f$4$4  &@path5str | Path | IO[bytes] | IO[str] | PartitioningScheme1str | Path | IO[bytes] | IO[str] | PyPartitioningcLt|ttfrt|St|tjr|St|t r|jStt|ddr|Sdt|d}t|)Nwritez!`path` argument has invalid type z), and cannot be turned into a sink target) isinstancestrrr1iorfr_py_partitioningcallablegetattrr3 TypeError)rmsgs r_to_sink_targetrs$d $$ !$''' D") $ $  D, - -$$ '$.. / / x2Ed2K2Kxxxnnr streamingbool background new_streaming_eager(Callable[[Any, int | None], None] | Nonec0t|tx}p|dk}|s|dvsd|}t||s|s|r|rtdtd}|rd}|sdSt ddd }|st}t |j| S) Ngpu)rcpuz in-memoryr old-streamingrzInvalid engine argument engine=zUGPU engine does not support streaming or background collection, disabling GPU engine.categoryF cudf_polarsz*GPU engine requested, but required packagezPlease install using the command `pip install cudf-polars-cu12` (or `pip install --extra-index-url=https://pypi.nvidia.com cudf-polars-cu11` if your system has a CUDA 11 driver).) err_prefixinstall_message)config)rrY ValueErrorr0 UserWarningrSr execute_with_cudf) rrrrr is_config_objis_gpurrs r_gpu_engine_callbackrs *&)<<Zdd?Zdd@ZddAZddBZddDZddEZdddGZddJZddKZ ddLZ!ddMZ"e# ddNd(ddQZ$e#dddSZ$e#dNd(ddUZ$ dd'd(ddXZ$dd`Z% ddbdcddiZ&e'e(djddddddddddddkde)dlddZ*e'e(ddddddddddddddddkdde)dddZ+dddZ,dddddddZ-ddddZ.e/dddddd dZ0e/dddddd dZ1e(ddddddddddddddke)dd dZ2e#dddddddddddke)d d dZ3e#dddddddddddkde)d d dZ3e'e(dddddddddddkde)d d dZ3e#dke)dddZ4e#ddke)dddÄZ4e'ddke)dddńZ4ddƄZ5e#ddddddddkddddNddkde)dɜdd߄Z6e#ddddddddkdddddkde)dddZ6ddddddddkdddddddke)dddZ6e#dddddkddddNdke)d ddZ7e#dddddkddddke)d ddZ7dddddkdddddke)d ddZ7e#ddddddddddddddddkddddNdke)dddZ8e#ddddddddddddddddkddddke)dddZ8ddddddddddddddddkdddddke)dddZ8e#dddkddddNdke)d ddZ9e#dddkddddke)ddd Z9dddkdddddke)d dd Z9e:d  ddddddddddddd ddZ; dddddddddddddd ddZ<ddZ=ddZ>ddd dZ?d!d"dZ@ddZAddd#d!ZBd$d#ZCd$d$ZDd%d'ZEd%d(ZFdd)d&d,ZGe/dd-d.dd/dd0d'd9ZHe/dd-d.dddd:d:dd;d<d(dCZIdddddddDdEdddddddFd)dXZJe/dYdZd[ d*dddEd]dddddd^ d+deZKeLdEdfd,dhZMd%diZNd%djZOe:dkd-dmZPddd.dpZQddd/dsZRddtZS d0ddvd1dzZTdd2d|ZUd3d"d~ZVd3d"dZWd3d"dZXddZYddZZe:dddZ[d4d5dZ\e:dd6d5dZ]d!d7dZ^ d8ddd9dZ_d:dZ`d0d;dZad0d;dZbddZcddZdddZeddZfddZgddZh d<d=dZid>dZj ddddd?dZk dd@dZl dd@dZm dddddddAdZndddddddddBdZoddZpdCdZqdDdZrdddEdZseL dFdddd:dœdGdńZtddƄZue:dǦ dHddȜdId˄ZveL dJdKd҄ZweLdӐdӐdӐdӐdԐdԐd՜dLdZx ddMdZydS(N LazyFrameu Representation of a Lazy computation graph/query against a DataFrame. This allows for whole-query optimisation in addition to parallelism, and is the preferred (and highest-performance) mode of operation for polars. Parameters ---------- data : dict, Sequence, ndarray, Series, or pandas.DataFrame Two-dimensional data in various forms; dict input must contain Sequences, Generators, or a `range`. Sequence may contain Series or other Sequences. schema : Sequence of str, (str,DataType) pairs, or a {str:DataType,} dict The LazyFrame schema may be declared in several ways: * As a dict of {name:type} pairs; if type is None, it will be auto-inferred. * As a list of column names; in this case types are automatically inferred. * As a list of (name,type) pairs; this is equivalent to the dictionary form. If you supply a list of column names that does not match the names in the underlying data, the names given here will overwrite them. The number of names given in the schema should match the underlying data dimensions. schema_overrides : dict, default None Support type specification or override of one or more columns; note that any dtypes inferred from the schema param will be overridden. The number of entries in the schema should match the underlying data dimensions, unless a sequence of dictionaries is being passed, in which case a *partial* schema can be declared to prevent specific fields from being loaded. strict : bool, default True Throw an error if any `data` value does not exactly match the given or inferred data type for that column. If set to `False`, values that do not match the data type are cast to that data type or, if casting is not possible, set to null instead. orient : {'col', 'row'}, default None Whether to interpret two-dimensional data as columns or as rows. If None, the orientation is inferred by matching the columns and data dimensions. If this does not yield conclusive results, column orientation is used. infer_schema_length : int or None The maximum number of rows to scan for schema inference. If set to `None`, the full data may be scanned *(this can be slow)*. This parameter only applies if the input data is a sequence or generator of rows; other input is read as-is. nan_to_null : bool, default False If the data comes from one or more numpy arrays, can optionally convert input data np.nan values to null instead. This is a no-op for all other input data. Notes ----- Initialising `LazyFrame(...)` directly is equivalent to `DataFrame(...).lazy()`. Examples -------- Constructing a LazyFrame directly from a dictionary: >>> data = {"a": [1, 2], "b": [3, 4]} >>> lf = pl.LazyFrame(data) >>> lf.collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 3 │ │ 2 ┆ 4 │ └─────┴─────┘ Notice that the dtypes are automatically inferred as Polars Int64: >>> lf.collect_schema().dtypes() [Int64, Int64] To specify a more detailed/specific frame schema you can supply the `schema` parameter with a dictionary of (name,dtype) pairs... >>> data = {"col1": [0, 2], "col2": [3, 7]} >>> lf2 = pl.LazyFrame(data, schema={"col1": pl.Float32, "col2": pl.Int64}) >>> lf2.collect() shape: (2, 2) ┌──────┬──────┐ │ col1 ┆ col2 │ │ --- ┆ --- │ │ f32 ┆ i64 │ ╞══════╪══════╡ │ 0.0 ┆ 3 │ │ 2.0 ┆ 7 │ └──────┴──────┘ ...a sequence of (name,dtype) pairs... >>> data = {"col1": [1, 2], "col2": [3, 4]} >>> lf3 = pl.LazyFrame(data, schema=[("col1", pl.Float32), ("col2", pl.Int64)]) >>> lf3.collect() shape: (2, 2) ┌──────┬──────┐ │ col1 ┆ col2 │ │ --- ┆ --- │ │ f32 ┆ i64 │ ╞══════╪══════╡ │ 1.0 ┆ 3 │ │ 2.0 ┆ 4 │ └──────┴──────┘ ...or a list of typed Series. >>> data = [ ... pl.Series("col1", [1, 2], dtype=pl.Float32), ... pl.Series("col2", [3, 4], dtype=pl.Int64), ... ] >>> lf4 = pl.LazyFrame(data) >>> lf4.collect() shape: (2, 2) ┌──────┬──────┐ │ col1 ┆ col2 │ │ --- ┆ --- │ │ f32 ┆ i64 │ ╞══════╪══════╡ │ 1.0 ┆ 3 │ │ 2.0 ┆ 4 │ └──────┴──────┘ Constructing a LazyFrame from a numpy ndarray, specifying column names: >>> import numpy as np >>> data = np.array([(1, 2), (3, 4)], dtype=np.int64) >>> lf5 = pl.LazyFrame(data, schema=["a", "b"], orient="col") >>> lf5.collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 3 │ │ 2 ┆ 4 │ └─────┴─────┘ Constructing a LazyFrame from a list of lists, row orientation specified: >>> data = [[1, 2, 3], [4, 5, 6]] >>> lf6 = pl.LazyFrame(data, schema=["a", "b", "c"], orient="row") >>> lf6.collect() shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 3 │ │ 4 ┆ 5 ┆ 6 │ └─────┴─────┴─────┘ ra_ldfzClassVar[set[str]] _accessorsNTF)schema_overridesstrictorientinfer_schema_length nan_to_nulldataFrameInitTypes | NoneschemaSchemaDefinition | NonerSchemaDict | NonerrrOrientation | Noner int | NonerrNonec nddlm}||||||||j|_dS)Nr)rl)rrrrrrr)polars.dataframerllazyr) selfrrrrrrrrls r__init__zLazyFrame.__init__sZ /..... I!1$7'   TVV  rldfc>||}||_|SN)__new__r)clsrrs r _from_pyldfzLazyFrame._from_pyldfs{{3  rbytesc*|Sr) serializers r __getstate__zLazyFrame.__getstate__s~~rstatec^|t|j|_dSr) deserializer r)rrs r __setstate__zLazyFrame.__setstate__s$$$WU^^449 rrVvalidate_schema1pa.schema | SchemaDict | Callable[[], SchemaDict]scan_fnrrVrc||}t|tr=tjt |||||_nhtrEt|tj r+tj t |||||_ntj ||||_|S)Nr)r) rrrra#scan_from_python_function_pl_schemalistitemsrrRpar^&scan_from_python_function_arrow_schema)scan_from_python_function_schema_function)rrrrVrrs r_scan_python_functionzLazyFrame._scan_python_functions{{3 fg & & #GV\\^^$$ / DII   Jvry$A$A #JV gwDII$MDI rbinary)formatsourcestr | Path | IOBaserrct|tr4t|}n+t|t t frt|}|dkr tj }n'|dkr tj }nd|}t|| ||S)u Read a logical plan from a file to construct a LazyFrame. Parameters ---------- source Path to a file or a file-like object (by file-like object, we refer to objects that have a `read()` method, such as a file handler (e.g. via builtin `open` function) or `BytesIO`). format The format with which the LazyFrame was serialized. Options: - `"binary"`: Deserialize from binary format (bytes). This is the default. - `"json"`: Deserialize from JSON format (string). Warnings -------- This function uses :mod:`pickle` if the logical plan contains Python UDFs, and as such inherits the security implications. Deserializing can execute arbitrary code, so it should only be attempted on trusted data. See Also -------- LazyFrame.serialize Notes ----- Serialization is not stable across Polars versions: a LazyFrame serialized in one Polars version may not be deserializable in another Polars version. Examples -------- >>> import io >>> lf = pl.LazyFrame({"a": [1, 2, 3]}).sum() >>> bytes = lf.serialize() >>> pl.LazyFrame.deserialize(io.BytesIO(bytes)).collect() shape: (1, 1) ┌─────┐ │ a │ │ --- │ │ i64 │ ╞═════╡ │ 6 │ └─────┘ rjson0`format` must be one of {'binary', 'json'}, got ) rrr getvalueencoderrr1radeserialize_binarydeserialize_jsonrr)rrr deserializerrs rrzLazyFrame.deserializesb fh ' ' 0V__..557788FF d , , 0'//F X  &9LL v  &7LLQvQQCS// !||F33444r list[str]cztdt|S)ar Get the column names. Returns ------- list of str A list containing the name of each column in order. Warnings -------- Determining the column names of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Using :meth:`collect_schema` is the idiomatic way of resolving the schema. This property exists only for symmetry with the DataFrame class. See Also -------- collect_schema Schema.names Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ).select("foo", "bar") >>> lf.columns # doctest: +SKIP ['foo', 'bar'] zDetermining the column names of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Use `LazyFrame.collect_schema().names()` to get the column names without this warning.r)r0rWcollect_schemanamesrs rcolumnszLazyFrame.columns sFD  =(     ""$$**,,,rlist[DataType]cztdt|S)ax Get the column data types. Returns ------- list of DataType A list containing the data type of each column in order. Warnings -------- Determining the data types of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Using :meth:`collect_schema` is the idiomatic way to resolve the schema. This property exists only for symmetry with the DataFrame class. See Also -------- collect_schema Schema.dtypes Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.dtypes # doctest: +SKIP [Int64, Float64, String] zDetermining the data types of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Use `LazyFrame.collect_schema().dtypes()` to get the data types without this warning.r)r0rWrdtypesrs rrzLazyFrame.dtypes3sFD  ;(     ""$$++---rr^cVtdt|S)a Get an ordered mapping of column names to their data type. Warnings -------- Resolving the schema of a LazyFrame is a potentially expensive operation. Using :meth:`collect_schema` is the idiomatic way to resolve the schema. This property exists only for symmetry with the DataFrame class. See Also -------- collect_schema Schema Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.schema # doctest: +SKIP Schema({'foo': Int64, 'bar': Float64, 'ham': String}) zResolving the schema of a LazyFrame is a potentially expensive operation. Use `LazyFrame.collect_schema()` to get the schema without this warning.r)r0rWrrs rrzLazyFrame.schema]s88  X'    ""$$$rintcztdt|S)a Get the number of columns. Returns ------- int Warnings -------- Determining the width of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Using :meth:`collect_schema` is the idiomatic way to resolve the schema. This property exists only for symmetry with the DataFrame class. See Also -------- collect_schema Schema.len Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [4, 5, 6], ... } ... ) >>> lf.width # doctest: +SKIP 2 zdetermining the width of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Use `LazyFrame.collect_schema().len()` to get the width without this warning.r)r0rWrlenrs rwidthzLazyFrame.widthsF@  6(     ""$$((***rrc$d}t|)Nztthe truth value of a LazyFrame is ambiguous LazyFrames cannot be used in boolean context with and/or/not operators.r)rrs r__bool__zLazyFrame.__bool__s Z nnroperatorrc,d|d}t|)N"z0" comparison not supported for LazyFrame objectsr )rrrs r_comparison_errorzLazyFrame._comparison_errorsN(NNNnnrotherobjectc0|ddS)Nz==rrrs r__eq__zLazyFrame.__eq__ t$$$$$rc0|ddS)Nz!=rrs r__ne__zLazyFrame.__ne__rrc0|ddS)N>rrs r__gt__zLazyFrame.__gt__ s#####rc0|ddS)N=rrs r__ge__zLazyFrame.__ge__rrc0|ddS)Nz<=rrs r__le__zLazyFrame.__le__rrkeyc.||vSr)r)rr&s r __contains__zLazyFrame.__contains__sd))++++rc*|Srcloners r__copy__zLazyFrame.__copy__zz||rmemoc*|Srr*)rr.s r __deepcopy__zLazyFrame.__deepcopy__r-ritemint | range | slicect|tsd}t|t||S)NzZLazyFrame is not subscriptable (aside from slicing) Use `select()` or `filter()` instead.)rslicerr'apply)rr1rs r __getitem__zLazyFrame.__getitem__sH$&& !< C.. t$$**4000rc4d|dS)NzOnaive plan: (run LazyFrame.explain(optimized=True) to see the optimized plan) F optimized)explainrs r__str__zLazyFrame.__str__s) rcDd|jjdt|ddS)Nr z at 0xXr) __class____name__idrs r__repr__zLazyFrame.__repr__s)?4>*??"T((?????rc> |jd}tjgd|}d|S#t $r3|ddd}d|d cYSwxYw) NFr8)dotz -Nshape=boxz-Tsvg)inputz^

NAIVE QUERY PLAN

run LazyFrame.show_graph() to see the optimized version

 z

zqnaive plan: (run LazyFrame.explain(optimized=True) to see the optimized plan)

z
) rto_dotrT check_outputrdecode Exceptionr:replace)rrCsvginserts r _repr_html_zLazyFrame._repr_html_s )""U"33C)///#x7H7HC<-0ZZ\\<<    \\E\22::4KKF     sAA:BB.fileLiteral['binary']cdSrrrNrs rrzLazyFrame.serializes rLiteral['json']cdSrrQrRs rrzLazyFrame.serializesNQcrIOBase | str | PathcdSrrQrRs rrzLazyFrame.serializes srIOBase | str | Path | Nonebytes | str | Nonec|dkr |jj}nK|dkr1d}tj|t |jj}nd|}t |t|||S)u( Serialize the logical plan of this LazyFrame to a file or string in JSON format. Parameters ---------- file File path to which the result should be written. If set to `None` (default), the output is returned as a string instead. format The format in which to serialize. Options: - `"binary"`: Serialize to binary format (bytes). This is the default. - `"json"`: Serialize to JSON format (string) (deprecated). See Also -------- LazyFrame.deserialize Notes ----- Serialization is not stable across Polars versions: a LazyFrame serialized in one Polars version may not be deserializable in another Polars version. Examples -------- Serialize the logical plan into a binary representation. >>> lf = pl.LazyFrame({"a": [1, 2, 3]}).sum() >>> bytes = lf.serialize() The bytes can later be deserialized back into a LazyFrame. >>> import io >>> pl.LazyFrame.deserialize(io.BytesIO(bytes)).collect() shape: (1, 1) ┌─────┐ │ a │ │ --- │ │ i64 │ ╞═════╡ │ 6 │ └─────┘ rrz6'json' serialization format of LazyFrame is deprecated) stacklevelr)rserialize_binarywarningswarnr-serialize_jsonrr&)rrNr serializerrs rrzLazyFrame.serializesb X  3JJ v  JC M*,,    1JJQvQQCS// !&z4@@@rfunction&Callable[Concatenate[LazyFrame, P], T]argsP.argskwargsP.kwargsrc||g|Ri|S)u Offers a structured way to apply a sequence of user-defined functions (UDFs). Parameters ---------- function Callable; will receive the frame as the first parameter, followed by any given args/kwargs. *args Arguments to pass to the UDF. **kwargs Keyword arguments to pass to the UDF. Examples -------- >>> def cast_str_to_int(lf: pl.LazyFrame, col_name: str) -> pl.LazyFrame: ... return lf.with_columns(pl.col(col_name).cast(pl.Int64)) >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": ["10", "20", "30", "40"], ... } ... ) >>> lf.pipe(cast_str_to_int, col_name="b").collect() shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 10 │ │ 2 ┆ 20 │ │ 3 ┆ 30 │ │ 4 ┆ 40 │ └─────┴─────┘ >>> lf = pl.LazyFrame( ... { ... "b": [1, 2], ... "a": [3, 4], ... } ... ) >>> lf.collect() shape: (2, 2) ┌─────┬─────┐ │ b ┆ a │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 3 │ │ 2 ┆ 4 │ └─────┴─────┘ >>> lf.pipe(lambda lf: lf.select(sorted(lf.collect_schema()))).collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 3 ┆ 1 │ │ 4 ┆ 2 │ └─────┴─────┘ rQ)rr`rbrds rpipezLazyFrame.pipeBs%Jx.t...v...rg?g?g?nearest) interpolation percentilesSequence[float] | float | Nonerjrrlc \ ddlm}|}|sd}t|gd}t |x}r|d|D|dtdd }ttc} g} tj d } | D]\} | } | o| }tjjd tjjdg}|s | s | t&kr&tjn| }| r&tjn| }|| s&tjn| }|| s&tjn| }g}|D]}| s|r|rMtj||| n'tj||}| n| }|||d| s$| s| t<t&fvr| g||d|d|d||d| r|d| Dn|j | !tE|fdtG|"D}tItK||}|D]fd|D|<||}|&dtOj(d||S)uh Creates a summary of statistics for a LazyFrame, returning a DataFrame. Parameters ---------- percentiles One or more percentiles to include in the summary statistics. All values must be in the range `[0, 1]`. interpolation : {'nearest', 'higher', 'lower', 'midpoint', 'linear', 'equiprobable'} Interpolation method used when calculating percentiles. Returns ------- DataFrame Notes ----- The median is included by default as the 50% percentile. Warnings -------- * This method does *not* maintain the laziness of the frame, and will `collect` the final result. This could potentially be an expensive operation. * We do not guarantee the output of `describe` to be stable. It will show statistics that we deem informative, and may be updated in the future. Using `describe` programmatically (versus interactive exploration) is not recommended for this reason. Examples -------- >>> from datetime import date, time >>> lf = pl.LazyFrame( ... { ... "float": [1.0, 2.8, 3.0], ... "int": [40, 50, None], ... "bool": [True, False, True], ... "str": ["zz", "xx", "yy"], ... "date": [date(2020, 1, 1), date(2021, 7, 5), date(2022, 12, 31)], ... "time": [time(10, 20, 30), time(14, 45, 50), time(23, 15, 10)], ... } ... ) Show default frame statistics: >>> lf.describe() shape: (9, 7) ┌────────────┬──────────┬──────────┬──────────┬──────┬─────────────────────┬──────────┐ │ statistic ┆ float ┆ int ┆ bool ┆ str ┆ date ┆ time │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ f64 ┆ str ┆ str ┆ str │ ╞════════════╪══════════╪══════════╪══════════╪══════╪═════════════════════╪══════════╡ │ count ┆ 3.0 ┆ 2.0 ┆ 3.0 ┆ 3 ┆ 3 ┆ 3 │ │ null_count ┆ 0.0 ┆ 1.0 ┆ 0.0 ┆ 0 ┆ 0 ┆ 0 │ │ mean ┆ 2.266667 ┆ 45.0 ┆ 0.666667 ┆ null ┆ 2021-07-02 16:00:00 ┆ 16:07:10 │ │ std ┆ 1.101514 ┆ 7.071068 ┆ null ┆ null ┆ null ┆ null │ │ min ┆ 1.0 ┆ 40.0 ┆ 0.0 ┆ xx ┆ 2020-01-01 ┆ 10:20:30 │ │ 25% ┆ 2.8 ┆ 40.0 ┆ null ┆ null ┆ 2021-07-05 ┆ 14:45:50 │ │ 50% ┆ 2.8 ┆ 50.0 ┆ null ┆ null ┆ 2021-07-05 ┆ 14:45:50 │ │ 75% ┆ 3.0 ┆ 50.0 ┆ null ┆ null ┆ 2022-12-31 ┆ 23:15:10 │ │ max ┆ 3.0 ┆ 50.0 ┆ 1.0 ┆ zz ┆ 2022-12-31 ┆ 23:15:10 │ └────────────┴──────────┴──────────┴──────────┴──────┴─────────────────────┴──────────┘ Customize which percentiles are displayed, applying linear interpolation: >>> with pl.Config(tbl_rows=12): ... lf.describe( ... percentiles=[0.1, 0.3, 0.5, 0.7, 0.9], ... interpolation="linear", ... ) shape: (11, 7) ┌────────────┬──────────┬──────────┬──────────┬──────┬─────────────────────┬──────────┐ │ statistic ┆ float ┆ int ┆ bool ┆ str ┆ date ┆ time │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ f64 ┆ str ┆ str ┆ str │ ╞════════════╪══════════╪══════════╪══════════╪══════╪═════════════════════╪══════════╡ │ count ┆ 3.0 ┆ 2.0 ┆ 3.0 ┆ 3 ┆ 3 ┆ 3 │ │ null_count ┆ 0.0 ┆ 1.0 ┆ 0.0 ┆ 0 ┆ 0 ┆ 0 │ │ mean ┆ 2.266667 ┆ 45.0 ┆ 0.666667 ┆ null ┆ 2021-07-02 16:00:00 ┆ 16:07:10 │ │ std ┆ 1.101514 ┆ 7.071068 ┆ null ┆ null ┆ null ┆ null │ │ min ┆ 1.0 ┆ 40.0 ┆ 0.0 ┆ xx ┆ 2020-01-01 ┆ 10:20:30 │ │ 10% ┆ 1.36 ┆ 41.0 ┆ null ┆ null ┆ 2020-04-20 ┆ 11:13:34 │ │ 30% ┆ 2.08 ┆ 43.0 ┆ null ┆ null ┆ 2020-11-26 ┆ 12:59:42 │ │ 50% ┆ 2.8 ┆ 45.0 ┆ null ┆ null ┆ 2021-07-05 ┆ 14:45:50 │ │ 70% ┆ 2.88 ┆ 47.0 ┆ null ┆ null ┆ 2022-02-07 ┆ 18:09:34 │ │ 90% ┆ 2.96 ┆ 49.0 ┆ null ┆ null ┆ 2022-09-13 ┆ 21:33:18 │ │ max ┆ 3.0 ┆ 50.0 ┆ 1.0 ┆ zz ┆ 2022-12-31 ┆ 23:15:10 │ └────────────┴──────────┴──────────┴──────────┴──────┴─────────────────────┴──────────┘ r) from_dictz/cannot describe a LazyFrame that has no columns)count null_countmeanstdminc3(K|] }|dzddVdS)dg%NrQ).0qs r z%LazyFrame.describe..s.>>qa#g????>>>>>>rmaxdtrrrcn|p!|ttttt fvSr) is_nestedr:r>rFrGrN)r|s r skip_minmaxz'LazyFrame.describe..skip_minmaxs%<<>>URKtVW+U%U UrNzcount:z null_count::zmean:zstd:zmin:zmax:c3bK|]*}tj|V+dSr)Fcolsort)rxcs rrzz%LazyFrame.describe..7s2!E!Ea!%((--//!E!E!E!E!E!Erc\g|](}d|z|dzz)S)r)row)rxn df_metrics n_metricss r z&LazyFrame.describe..AsL    NN1  q9}Q)0CC D   rcg|]=}|t|trdn!vrt|nt|>Sr)rdictfloatr)rxvrhas_numeric_results rrz&LazyFrame.describe..Is`  Z4%8%8 D'(,>'>'>%(((SVV r statistic)r|rrr))polars.convertrnrrr2extendappendr setrlitr is_numeric is_temporalrronameprefixrpr9rqrrrsr{ to_physicalquantilecastaddaliasr~rF with_columnsselectcollectr rangerzip insert_columnplSeries)rrkrjrnrrmetrics quantilesr sort_cols metric_exprsnulldtyperr count_exprs mean_exprexpr_stdmin_exprmax_expr pct_exprsppct_exprcolumn_metricssummary df_summaryrrrrs @@@@rdescribezLazyFrame.describes~ -,,,,,$$&& !CCC.. @??)+66 69 ? NN>>I>>> > > >u  V V V  V ),suu%I#% uT{{ 0 0 HAu))++J(.@U->->-@-@Ka  %,,X66a##%%*11-@@K",050@0@a  *4=quQxx||~~~H-8[-?-?IquQxx||~~~TH-8[-?-?IquQxx||~~~THI = =$$'Aa,,..77=IINNuUUUU1XX..q-@@ MM!$$$$#H  1 q !;!;<<<< *U__.. *%D'?2J2J"&&q)))    OOKAKK00NN:!::..NN:!::..    NN:!::..      !!!E!E9!E!E!EEEE \ #WYY LL      6::<<((   s6>2233  A ! GAJJYw''   BIk7$C$CDDDrplainr)rr9 type_coercionpredicate_pushdownprojection_pushdownsimplify_expressionslice_pushdowncomm_subplan_elimcomm_subexpr_elimcluster_with_columnscollapse_joinsrr tree_format optimizationsrtr9rrrrrrrrrrrrsr bool | Nonerric|tdd|rd}t| } | dvrtd|r|}| dk|j_| d k|j_|j|j}|dkr| S| S|dkr|j S|j S) a Create a string representation of the query plan. Different optimizations can be turned on or off. Parameters ---------- format : {'plain', 'tree'} The format to use for displaying the logical plan. optimized Return an optimized query plan. Defaults to `True`. If this is set to `True` the subsequent optimization flags control which optimizations run. type_coercion Do type coercion optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. predicate_pushdown Do predicate pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. projection_pushdown Do projection pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. simplify_expression Run simplify expressions optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. slice_pushdown Slice pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subplan_elim Will try to cache branching subplans that occur on self-joins or unions. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subexpr_elim Common subexpressions will be cached and reused. .. deprecated:: 1.30.0 Use the `optimizations` parameters. cluster_with_columns Combine sequential independent calls to with_columns .. deprecated:: 1.30.0 Use the `optimizations` parameters. collapse_joins Collapse a join and filters into a faster join .. deprecated:: 1.30.0 Use the `optimizations` parameters. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine. If set to `"gpu"`, the GPU engine is used. Fine-grained control over the GPU engine, for example which device to use on a system with multiple devices, is possible by providing a :class:`~.GPUEngine` object with configuration options. .. note:: GPU mode is considered **unstable**. Not all queries will run successfully on the GPU, however, they should fall back transparently to the default engine if execution is not supported. Running with `POLARS_VERBOSE=1` will provide information if a query falls back (and why). .. note:: The GPU engine does not support streaming, if streaming is enabled then GPU execution is switched off. optimizations The optimization passes done during query optimization. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. tree_format Format the output as a tree. .. deprecated:: 0.20.30 Use `format="tree"` instead. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [1, 2, 3, 4, 5, 6], ... "c": [6, 5, 4, 3, 2, 1], ... } ... ) >>> lf.group_by("a", maintain_order=True).agg(pl.all().sum()).sort( ... "a" ... ).explain() # doctest: +SKIP Nzethe `tree_format` parameter for `LazyFrame.explain` is deprecated Use the `format` parameter instead.z0.20.30versiontreerr&streaming mode is considered unstable.rr) r"rr(r, _pyoptflagsr old_streamingrwith_optimizationsdescribe_optimized_plan_treedescribe_optimized_plandescribe_plan_tree describe_plan)rrr9rrrrrrrrrrrrrrs rr:zLazyFrame.explainWs@  " %7!      '' 3 3 3 "#K L L L  5)2244M28K2GM % /6<6OM % 3)..}/HIIC7799922444 V  9//11 19**,, ,r)g0@g(@ir)r9show output_path raw_outputfigsizer _type_checkrrrrrrrrr plan_stage _check_orderrrrstr | Path | Nonerrtuple[float, float]rrrr str | Nonect|}|dvrtd|}|dk|j_|dk|j_|j|j}|dkr||}nM|dkr2|dkr| |}n+||}nd|d}t|t||||| S) a Show a plot of the query plan. Note that Graphviz must be installed to render the visualization (if not already present, you can download it here: ``_). Parameters ---------- optimized Optimize the query plan. show Show the figure. output_path Write the figure to disk. raw_output Return dot syntax. This cannot be combined with `show` and/or `output_path`. figsize Passed to matplotlib if `show == True`. type_coercion Do type coercion optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. predicate_pushdown Do predicate pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. projection_pushdown Do projection pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. simplify_expression Run simplify expressions optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. slice_pushdown Slice pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subplan_elim Will try to cache branching subplans that occur on self-joins or unions. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subexpr_elim Common subexpressions will be cached and reused. .. deprecated:: 1.30.0 Use the `optimizations` parameters. cluster_with_columns Combine sequential independent calls to with_columns. .. deprecated:: 1.30.0 Use the `optimizations` parameters. collapse_joins Collapse a join and filters into a faster join. .. deprecated:: 1.30.0 Use the `optimizations` parameters. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine. If set to `"gpu"`, the GPU engine is used. Fine-grained control over the GPU engine, for example which device to use on a system with multiple devices, is possible by providing a :class:`~.GPUEngine` object with configuration options. .. note:: GPU mode is considered **unstable**. Not all queries will run successfully on the GPU, however, they should fall back transparently to the default engine if execution is not supported. Running with `POLARS_VERBOSE=1` will provide information if a query falls back (and why). .. note:: The GPU engine does not support streaming, if streaming is enabled then GPU execution is switched off. plan_stage : {'ir', 'physical'} Select the stage to display. Currently only the streaming engine has a separate physical stage, for the other engines both IR and physical are the same. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [1, 2, 3, 4, 5, 6], ... "c": [6, 5, 4, 3, 2, 1], ... } ... ) >>> lf.group_by("a", maintain_order=True).agg(pl.all().sum()).sort( ... "a" ... ).show_graph() # doctest: +SKIP rrrrrphysicalzinvalid plan stage '')rCrrrr) rr(r,rrrrrrFto_dot_streaming_physrr+)rr9rrrrrrrrrrrrrrrrrrrrC error_msgs r show_graphzLazyFrame.show_graphsD '' 3 3 3 "#K L L L%..00 28O2K !/.4 .C !+y++M,EFF   ++i((CC : % %$$00;;kk),,>> lf = pl.LazyFrame({"foo": [1, 1, -2, 3]}) >>> ( ... lf.with_columns(pl.col("foo").cum_sum().alias("bar")) ... .inspect() # print the node before the filter ... .filter(pl.col("bar") == pl.col("foo")) ... ) # doctest: +ELLIPSIS srlrcLt||Sr)printr)rrs rinspectz"LazyFrame.inspect..inspects #**Q-- HrT)rr)rrlrrl) map_batches)rrrs ` rrzLazyFrame.inspectsG$       $    r) descending nulls_lastmaintain_order multithreadedbyIntoExpr | Iterable[IntoExpr]more_byrwrbool | Sequence[bool]rrrc t|tr]|s[t|trFt|tr1||j|||||St |g|R}t|t|dd}t|t|dd}||j |||||S)u Sort the LazyFrame by the given columns. Parameters ---------- by Column(s) to sort by. Accepts expression input, including selectors. Strings are parsed as column names. *more_by Additional columns to sort by, specified as positional arguments. descending Sort in descending order. When sorting by multiple columns, can be specified per column by passing a sequence of booleans. nulls_last Place null values last; can specify a single boolean applying to all columns or a sequence of booleans for per-column control. maintain_order Whether the order should be maintained if elements are equal. Note that if `true` streaming is not possible and performance might be worse since this requires a stable search. multithreaded Sort using multiple threads. Examples -------- Pass a single column name to sort by that column. >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, None], ... "b": [6.0, 5.0, 4.0], ... "c": ["a", "c", "b"], ... } ... ) >>> lf.sort("a").collect() shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ null ┆ 4.0 ┆ b │ │ 1 ┆ 6.0 ┆ a │ │ 2 ┆ 5.0 ┆ c │ └──────┴─────┴─────┘ Sorting by expressions is also supported. >>> lf.sort(pl.col("a") + pl.col("b") * 2, nulls_last=True).collect() shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ 2 ┆ 5.0 ┆ c │ │ 1 ┆ 6.0 ┆ a │ │ null ┆ 4.0 ┆ b │ └──────┴─────┴─────┘ Sort by multiple columns by passing a list of columns. >>> lf.sort(["c", "a"], descending=True).collect() shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ 2 ┆ 5.0 ┆ c │ │ null ┆ 4.0 ┆ b │ │ 1 ┆ 6.0 ┆ a │ └──────┴─────┴─────┘ Or use positional arguments to sort by multiple columns in the same way. >>> lf.sort("c", "a", descending=[False, True]).collect() shape: (3, 3) ┌──────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞══════╪═════╪═════╡ │ 1 ┆ 6.0 ┆ a │ │ null ┆ 4.0 ┆ b │ │ 2 ┆ 5.0 ┆ c │ └──────┴─────┴─────┘ rrr) rrrrrrr%r,r  sort_by_exprs)rrrrrrrs rrzLazyFrame.sortsF r3    :t,, :t,,  ##  J   ,B 9 9 9 9 SWWlDII  SWWlDII  I # #J NM     rr) table_namequeryrcddlm}td|dd5}|r|nd}|||||cdddS#1swxYwYdS) u Execute a SQL query against the LazyFrame. .. versionadded:: 0.20.23 .. warning:: This functionality is considered **unstable**, although it is close to being considered stable. It may be changed at any point without it being considered a breaking change. Parameters ---------- query SQL query to execute. table_name Optionally provide an explicit name for the table that represents the calling frame (defaults to "self"). Notes ----- * The calling frame is automatically registered as a table in the SQL context under the name "self". If you want access to the DataFrames and LazyFrames found in the current globals, use the top-level :meth:`pl.sql `. * More control over registration and execution behaviour is available by using the :class:`SQLContext` object. See Also -------- SQLContext Examples -------- >>> lf1 = pl.LazyFrame({"a": [1, 2, 3], "b": [6, 7, 8], "c": ["z", "y", "x"]}) >>> lf2 = pl.LazyFrame({"a": [3, 2, 1], "d": [125, -654, 888]}) Query the LazyFrame using SQL: >>> lf1.sql("SELECT c, b FROM self WHERE a > 1").collect() shape: (2, 2) ┌─────┬─────┐ │ c ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ y ┆ 7 │ │ x ┆ 8 │ └─────┴─────┘ Apply SQL transforms (aliasing "self" to "frame") then filter natively (you can freely mix SQL and native operations): >>> lf1.sql( ... query=''' ... SELECT ... a, ... (a % 2 == 0) AS a_is_even, ... (b::float4 / 2) AS "b/2", ... CONCAT_WS(':', c, c, c) AS c_c_c ... FROM frame ... ORDER BY a ... ''', ... table_name="frame", ... ).filter(~pl.col("c_c_c").str.starts_with("x")).collect() shape: (2, 4) ┌─────┬───────────┬─────┬───────┐ │ a ┆ a_is_even ┆ b/2 ┆ c_c_c │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ bool ┆ f32 ┆ str │ ╞═════╪═══════════╪═════╪═══════╡ │ 1 ┆ false ┆ 3.0 ┆ z:z:z │ │ 2 ┆ true ┆ 3.5 ┆ y:y:y │ └─────┴───────────┴─────┴───────┘ r) SQLContextzS`sql` is considered **unstable** (although it is close to being considered stable).F)register_globalseagerr)rframeN) polars.sqlrr(registerexecute)rrrrctxrs rsqlz LazyFrame.sql&sT *))))) a   Ze < < < &!+7::D LLd$L / / /;;u%% & & & & & & & & & & & & & & & & & &s2A""A&)A&reversez1.0.0r)rkct|}t|t|dd}||j|||S)u' Return the `k` largest rows. Non-null elements are always preferred over null elements, regardless of the value of `reverse`. The output is not guaranteed to be in any particular order, call :func:`sort` after this function if you wish the output to be sorted. .. versionchanged:: 1.0.0 The `descending` parameter was renamed `reverse`. Parameters ---------- k Number of rows to return. by Column(s) used to determine the top rows. Accepts expression input. Strings are parsed as column names. reverse Consider the `k` smallest elements of the `by` column(s) (instead of the `k` largest). This can be specified per column by passing a sequence of booleans. See Also -------- bottom_k Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [2, 1, 1, 3, 2, 1], ... } ... ) Get the rows which contain the 4 largest values in column b. >>> lf.top_k(4, by="b").collect() shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ b ┆ 3 │ │ a ┆ 2 │ │ b ┆ 2 │ │ b ┆ 1 │ └─────┴─────┘ Get the rows which contain the 4 largest values when sorting on column b and a. >>> lf.top_k(4, by=["b", "a"]).collect() shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ b ┆ 3 │ │ b ┆ 2 │ │ a ┆ 2 │ │ c ┆ 1 │ └─────┴─────┘ rrrr)r%r,r rrtop_krrrrs rrzLazyFrame.top_kzsST,B / /gs2ww 4@@ b' J JKKKrct|}t|t|dd}||j|||S)u0 Return the `k` smallest rows. Non-null elements are always preferred over null elements, regardless of the value of `reverse`. The output is not guaranteed to be in any particular order, call :func:`sort` after this function if you wish the output to be sorted. .. versionchanged:: 1.0.0 The `descending` parameter was renamed `reverse`. Parameters ---------- k Number of rows to return. by Column(s) used to determine the bottom rows. Accepts expression input. Strings are parsed as column names. reverse Consider the `k` largest elements of the `by` column(s) (instead of the `k` smallest). This can be specified per column by passing a sequence of booleans. See Also -------- top_k Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [2, 1, 1, 3, 2, 1], ... } ... ) Get the rows which contain the 4 smallest values in column b. >>> lf.bottom_k(4, by="b").collect() shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ b ┆ 1 │ │ a ┆ 1 │ │ c ┆ 1 │ │ a ┆ 2 │ └─────┴─────┘ Get the rows which contain the 4 smallest values when sorting on column a and b. >>> lf.bottom_k(4, by=["a", "b"]).collect() shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ a ┆ 1 │ │ a ┆ 2 │ │ b ┆ 1 │ │ b ┆ 2 │ └─────┴─────┘ rrr)r%r,r rrbottom_kr s rr zLazyFrame.bottom_ksVT,B / /gs2ww 4@@ 2 21W 2 M MNNNrr))rrrrno_optimizationrrrrr show_plottruncate_nodesrrrrrrtuple[int, int]_kwargstuple[DataFrame, DataFrame]c (|D]}|dvrd|d}t|t|}|}|dk|j_|j|j}t||dkddd}|d|d}| |\}}t|t|}}| rtd d d dl m }|d | \}}|dd}|}|dkr/d}|t#jddgdz }n7|dkr/d}|t#jddgdz }nd}| d krC|t#jdjd | dz}|dd }||d|d|dz |d|d|d|d|||d|||fS) u@ Profile a LazyFrame. This will run the query and return a tuple containing the materialized DataFrame and a DataFrame that contains profiling information of each node that is executed. The units of the timings are microseconds. Parameters ---------- type_coercion Do type coercion optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. predicate_pushdown Do predicate pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. projection_pushdown Do projection pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. simplify_expression Run simplify expressions optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. no_optimization Turn off (certain) optimizations. .. deprecated:: 1.30.0 Use the `optimizations` parameters. slice_pushdown Slice pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subplan_elim Will try to cache branching subplans that occur on self-joins or unions. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subexpr_elim Common subexpressions will be cached and reused. .. deprecated:: 1.30.0 Use the `optimizations` parameters. cluster_with_columns Combine sequential independent calls to with_columns .. deprecated:: 1.30.0 Use the `optimizations` parameters. collapse_joins Collapse a join and filters into a faster join .. deprecated:: 1.30.0 Use the `optimizations` parameters. show_plot Show a gantt chart of the profiling result truncate_nodes Truncate the label lengths in the gantt chart to this number of characters. figsize matplotlib figsize of the profiling plot engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine. If set to `"gpu"`, the GPU engine is used. Fine-grained control over the GPU engine, for example which device to use on a system with multiple devices, is possible by providing a :class:`~.GPUEngine` object with configuration options. .. note:: GPU mode is considered **unstable**. Not all queries will run successfully on the GPU, however, they should fall back transparently to the default engine if execution is not supported. Running with `POLARS_VERBOSE=1` will provide information if a query falls back (and why). .. note:: The GPU engine does not support streaming, if streaming is enabled then GPU execution is switched off. optimizations The optimization passes done during query optimization. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [1, 2, 3, 4, 5, 6], ... "c": [6, 5, 4, 3, 2, 1], ... } ... ) >>> lf.group_by("a", maintain_order=True).agg(pl.all().sum()).sort( ... "a" ... ).profile() # doctest: +SKIP (shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 4 ┆ 10 │ │ b ┆ 11 ┆ 10 │ │ c ┆ 6 ┆ 1 │ └─────┴─────┴─────┘, shape: (3, 3) ┌─────────────────────────┬───────┬──────┐ │ node ┆ start ┆ end │ │ --- ┆ --- ┆ --- │ │ str ┆ u64 ┆ u64 │ ╞═════════════════════════╪═══════╪══════╡ │ optimization ┆ 0 ┆ 5 │ │ group_by_partitioned(a) ┆ 5 ┆ 470 │ │ sort(a) ┆ 475 ┆ 1964 │ └─────────────────────────┴───────┴──────┘) )post_opt_callbackz.profile() got an unexpected keyword argument 'rrFrrrrrN matplotlibz+should be installed to show profiling plots) err_suffixrr)rendgeArstarti@Bg.Amsiusnodez...)r leftzProfiling resultznode duration in [z ], total nodes)rrr,rrrrrgetprofiler5rSmatplotlib.pyplotpyplotsubplotsrrrrrr4barhtitle set_xlabel set_ylabelr)rrrrrrrrrrrrrrrrrrrrcallbackdftimingsplt_figaxmax_valtimings_unit max_in_units rr"zLazyFrame.profilesr + +ARQQQQ  ***   ''%..00 28O2K !/i**=+DEE' /     ;;* + + 7{{#677Hkk(++ G WW%5%5W #  H     , + + + + +||Aw|77HD"enR(G((H}}#007G1H1H91TUU3#007G1H1H41OPP!!#00E&MM%++A~>>F#5/!,K GG uo(99g&     II( ) ) ) MMQtQQkQ4QQ R R R MM' " " " HHJJJ7{r) rrrrrrrrrrrrr Literal[True]r[c dSrrQrrrrrrrrrrrrrrs rrzLazyFrame.collects "r) rrrrrrrrrrrrrLiteral[False]c dSrrQr6s rrzLazyFrame.collects "CrDataFrame | InProcessQueryc  `|D]}|dvrd|d}t|t| } |ddptdk}|rd} | dvrt dt | | d k| || jj }t| trd } |j | j}| r0t d t| S|d |}t|| |S)u Materialize this LazyFrame into a DataFrame. By default, all query optimizations are enabled. Individual optimizations may be disabled by setting the corresponding parameter to `False`. Parameters ---------- type_coercion Do type coercion optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. predicate_pushdown Do predicate pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. projection_pushdown Do projection pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. simplify_expression Run simplify expressions optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. slice_pushdown Slice pushdown optimization. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subplan_elim Will try to cache branching subplans that occur on self-joins or unions. .. deprecated:: 1.30.0 Use the `optimizations` parameters. comm_subexpr_elim Common subexpressions will be cached and reused. .. deprecated:: 1.30.0 Use the `optimizations` parameters. cluster_with_columns Combine sequential independent calls to with_columns .. deprecated:: 1.30.0 Use the `optimizations` parameters. collapse_joins Collapse a join and filters into a faster join .. deprecated:: 1.30.0 Use the `optimizations` parameters. no_optimization Turn off (certain) optimizations. .. deprecated:: 1.30.0 Use the `optimizations` parameters. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine. If set to `"gpu"`, the GPU engine is used. Fine-grained control over the GPU engine, for example which device to use on a system with multiple devices, is possible by providing a :class:`~.GPUEngine` object with configuration options. .. note:: GPU mode is considered **unstable**. Not all queries will run successfully on the GPU, however, they should fall back transparently to the default engine if execution is not supported. Running with `POLARS_VERBOSE=1` will provide information if a query falls back (and why). .. note:: The GPU engine does not support streaming, or running in the background. If either are enabled, then GPU execution is switched off. background Run the query in the background and get a handle to the query. This handle can be used to fetch the result or cancel the query. .. warning:: Background mode is considered **unstable**. It may be changed at any point without it being considered a breaking change. optimizations The optimization passes done during query optimization. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Returns ------- DataFrame See Also -------- explain : Print the query plan that is evaluated with collect. profile : Collect the LazyFrame and time each node in the computation graph. polars.collect_all : Collect multiple LazyFrames at the same time. polars.Config.set_streaming_chunk_size : Set the size of streaming batches. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [1, 2, 3, 4, 5, 6], ... "c": [6, 5, 4, 3, 2, 1], ... } ... ) >>> lf.group_by("a").agg(pl.all().sum()).collect() # doctest: +SKIP shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 4 ┆ 10 │ │ b ┆ 11 ┆ 10 │ │ c ┆ 6 ┆ 1 │ └─────┴─────┴─────┘ Collect in streaming mode >>> lf.group_by("a").agg(pl.all().sum()).collect( ... engine="streaming" ... ) # doctest: +SKIP shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 4 ┆ 10 │ │ b ┆ 11 ┆ 10 │ │ c ┆ 6 ┆ 1 │ └─────┴─────┴─────┘ Collect in GPU mode >>> lf.group_by("a").agg(pl.all().sum()).collect(engine="gpu") # doctest: +SKIP shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 11 ┆ 10 │ │ a ┆ 4 ┆ 10 │ │ c ┆ 6 ┆ 1 │ └─────┴─────┴─────┘ With control over the device used >>> lf.group_by("a").agg(pl.all().sum()).collect( ... engine=pl.GPUEngine(device=1) ... ) # doctest: +SKIP shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ b ┆ 11 ┆ 10 │ │ a ┆ 4 ┆ 10 │ │ c ┆ 6 ┆ 1 │ └─────┴─────┴─────┘ )rrz.collect() got an unexpected keyword argument 'rrFr)rrrrrrz'background mode is considered unstable.r)rrr!rbr(rrrrrYrrr[collect_concurrentlyr5r)rrrrrrrrrrrrrrrrrrr*rs rrzLazyFrame.collectspB + +ARQQQQ  ***  '' KK / / W3F3H3HK3W   ! F 3 3 3 "#K L L L' /!' ,2     fi ( ( Fi**=+DEE  > "#L M M M!#":":"<"<== =;;2H==s{{6844555r)rrgevent!_GeventDataFrameResult[DataFrame]cdSrrQrr<rrs r collect_asynczLazyFrame.collect_asyncs -0Cr)r<rrAwaitable[DataFrame]cdSrrQr?s rr@zLazyFrame.collect_async s  #sr8Awaitable[DataFrame] | _GeventDataFrameResult[DataFrame]ct|}|dvrtd|j|j}|rt n t }|||j|S)u~ Collect DataFrame asynchronously in thread pool. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Collects into a DataFrame (like :func:`collect`) but, instead of returning a DataFrame directly, it is scheduled to be collected inside a thread pool, while this method returns almost instantly. This can be useful if you use `gevent` or `asyncio` and want to release control to other greenlets/tasks while LazyFrames are being collected. Parameters ---------- gevent Return wrapper to `gevent.event.AsyncResult` instead of Awaitable engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars in-memory engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars in-memory engine. .. note:: The GPU engine does not support async, or running in the background. If either are enabled, then GPU execution is switched off. optimizations The optimization passes done during query optimization. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Returns ------- If `gevent=False` (default) then returns an awaitable. If `gevent=True` then returns wrapper that has a `.get(block=True, timeout=None)` method. See Also -------- polars.collect_all : Collect multiple LazyFrames at the same time. polars.collect_all_async : Collect multiple LazyFrames at the same time lazily. Notes ----- In case of error `set_exception` is used on `asyncio.Future`/`gevent.event.AsyncResult` and will be reraised by them. Examples -------- >>> import asyncio >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [1, 2, 3, 4, 5, 6], ... "c": [6, 5, 4, 3, 2, 1], ... } ... ) >>> async def main(): ... return await ( ... lf.group_by("a", maintain_order=True) ... .agg(pl.all().sum()) ... .collect_async() ... ) >>> asyncio.run(main()) shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 4 ┆ 10 │ │ b ┆ 11 ┆ 10 │ │ c ┆ 6 ┆ 1 │ └─────┴─────┴─────┘ rr) rr(rrrrrcollect_with_callback _callback)rr<rrrresults rr@zLazyFrame.collect_async st '' 3 3 3 "#K L L Li**=+DEE)/ I " $ $ $4G4I4I  !!&&*:;;; rcRt|jdS)a Resolve the schema of this LazyFrame. Examples -------- Determine the schema. >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.collect_schema() Schema({'foo': Int64, 'bar': Float64, 'ham': String}) Access various properties of the schema. >>> schema = lf.collect_schema() >>> schema["bar"] Float64 >>> schema.names() ['foo', 'bar', 'ham'] >>> schema.dtypes() [Int64, Float64, String] >>> schema.len() 3 F) check_dtypes)r^rrrs rrzLazyFrame.collect_schemaw s%<di..00uEEEErzstd) compressioncompression_level statisticsrow_group_sizedata_page_sizerstorage_optionscredential_providerretries sync_on_closemkdirrfield_overwritesrmetadatarr+str | Path | IO[bytes] | PartitioningSchemerLrMrNbool | str | dict[str, bool]rOrPrQdict[str, Any] | NonerR3CredentialProviderFunction | Literal['auto'] | NonerSrTSyncOnCloseMethod | NonerUrrVgParquetFieldOverwrites | Sequence[ParquetFieldOverwrites] | Mapping[str, ParquetFieldOverwrites] | NonerWParquetMetadata | NonecdSrrQrrrLrMrNrOrPrrQrRrSrTrUrrVrrWrs r sink_parquetzLazyFrame.sink_parquet s 4sr)rLrMrNrOrPrrQrRrSrTrUrVrrWrcdSrrQr`s rrazLazyFrame.sink_parquet s 4Cr)rLrMrNrOrPrrQrRrSrTrWrUrrVrrLazyFrame | Nonect|}| d}t|t|tr |rddddd}n't|tr|si}n |dkrddddd}ddlm}|| ||d }~ |r"t |}nd}t|}| pd || d }t| tr'| r"t | } n!d} nt| rt| } g}|ddl }dd l m}m}mt||r |g}n}t||jjr|t|}nJt||jjrfd |D}n!dt)|}t+||j||||||||| || | }|sL||j}t4|}||dSt4|S)a? Evaluate the query in streaming mode and write to a Parquet file. This allows streaming results that are larger than RAM to be written to disk. Parameters ---------- path File path to which the file should be written. compression : {'lz4', 'uncompressed', 'snappy', 'gzip', 'lzo', 'brotli', 'zstd'} Choose "zstd" for good compression performance. Choose "lz4" for fast compression/decompression. Choose "snappy" for more backwards compatibility guarantees when you deal with older parquet readers. compression_level The level of compression to use. Higher compression means smaller files on disk. - "gzip" : min-level: 0, max-level: 9. - "brotli" : min-level: 0, max-level: 11. - "zstd" : min-level: 1, max-level: 22. statistics Write statistics to the parquet headers. This is the default behavior. Possible values: - `True`: enable default set of statistics (default). Some statistics may be disabled. - `False`: disable all statistics - "full": calculate and write all available statistics. Cannot be combined with `use_pyarrow`. - `{ "statistic-key": True / False, ... }`. Cannot be combined with `use_pyarrow`. Available keys: - "min": column minimum value (default: `True`) - "max": column maximum value (default: `True`) - "distinct_count": number of unique column values (default: `False`) - "null_count": number of null values in column (default: `True`) row_group_size Size of the row groups in number of rows. If None (default), the chunks of the `DataFrame` are used. Writing in smaller chunks may reduce memory pressure and improve writing speeds. data_page_size Size limit of individual data pages. If not set defaults to 1024 * 1024 bytes maintain_order Maintain the order in which data is processed. Setting this to `False` will be slightly faster. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. sync_on_close: { None, 'data', 'all' } Sync to disk when before closing a file. * `None` does not sync. * `data` syncs the file contents. * `all` syncs the file contents and metadata. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. metadata A dictionary or callback to add key-values to the file-level Parquet metadata. .. warning:: This functionality is considered **experimental**. It may be removed or changed at any point without it being considered a breaking change. mkdir: bool Recursively create all the directories in the path. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. lazy: bool Wait to start execution until `collect` is called. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. field_overwrites Property overwrites for individual Parquet fields. This allows more control over the writing process to the granularity of a Parquet field. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars streaming engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars streaming engine. optimizations The optimization passes done during query optimization. This has no effect if `lazy` is set to `True`. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Returns ------- DataFrame Examples -------- >>> lf = pl.scan_csv("/path/to/my_larger_than_ram_file.csv") # doctest: +SKIP >>> lf.sink_parquet("out.parquet") # doctest: +SKIP Nz/`metadata` parameter is considered experimentalTF)rsr{distinct_countrpfullr!_init_credential_provider_builderranonerTrrU)r+_parquet_field_overwrites_dict_to_dict_list!_parquet_field_overwrites_to_dictc&g|] }|SrQrQ)rxrrls rrz*LazyFrame.sink_parquet.. s2***=>55a88***rz$field_overwrites got the wrong type ) targetrLrMrNrOrP cloud_optionsrRrS sink_optionsrWrVr)rr(rr,polars.io.cloud.credential_provider._builderrhrrrrrr# collections"polars.io.parquet.field_overwritesrrkrlabcrretyperrrarrrrr)rrrLrMrNrOrPrrQrRrSrTrWrUrrVrrrrhcredential_provider_builderrnrpfield_overwrites_dictsrrrrkrrls @rrazLazyFrame.sink_parquet sWN ''  CC "3 ' ' ' j$ ' ' J "'" JJ  D ) ) * JJ 6 ! !"&" J      'H&G ' ' #  #"?#8#8#:#:;;OO#O &&*4f,  h % % @  0 011  h   @5h??H8:  '               *,BCC %556FGG*&&,ko.EFF %)T)T)****&&,ko.FGG %****BR***&&VTBR=S=SUUnn$i$$#/!))) ;%3%    (()BCCC'',,C KKvK & & &4$$S)))r) rL compat_levelrrQrRrSrTrUrrrIpcCompression | NonerxCompatLevel | Nonec dSrrQ rrrLrxrrQrRrSrTrUrrrs rsink_ipczLazyFrame.sink_ipc s $sr) rLrxrrQrRrSrTrUrrc dSrrQr|s rr}zLazyFrame.sink_ipc s $Cr uncompressedc  t| } ddlm} | |||d}~|r"t|}nd}t |}|pd|| d}|d}nt |tr|j}|d}|j ||||||| }| sL| | j }t|}|| dSt|S) a Evaluate the query in streaming mode and write to an IPC file. This allows streaming results that are larger than RAM to be written to disk. Parameters ---------- path File path to which the file should be written. compression : {'uncompressed', 'lz4', 'zstd'} Choose "zstd" for good compression performance. Choose "lz4" for fast compression/decompression. compat_level Use a specific compatibility level when exporting Polars' internal data structures. maintain_order Maintain the order in which data is processed. Setting this to `False` will be slightly faster. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. sync_on_close: { None, 'data', 'all' } Sync to disk when before closing a file. * `None` does not sync. * `data` syncs the file contents. * `all` syncs the file contents and metadata. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. mkdir: bool Recursively create all the directories in the path. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. lazy: bool Wait to start execution until `collect` is called. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars streaming engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars streaming engine. .. note:: The GPU engine is currently not supported. optimizations The optimization passes done during query optimization. This has no effect if `lazy` is set to `True`. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Returns ------- DataFrame Examples -------- >>> lf = pl.scan_csv("/path/to/my_larger_than_ram_file.csv") # doctest: +SKIP >>> lf.sink_ipc("out.arrow") # doctest: +SKIP rrgr}NrirjTr)rnrLrxrorRrSrpr)rrqrhrrrrrX_versionrr}rrrrr)rrrLrxrrQrRrSrTrUrrrrhrvrnrprs rr}zLazyFrame.sink_ipc sjf ''      'H&G  ' ' #  #"?#8#8#:#:;;OO#O &&*4f,   LL  k 2 2 1'0L  (Ki  #%) ;%!   (()BCCC'',,C KKvK & & &4$$S)))r,rEri) include_bominclude_header separatorline_terminator quote_char batch_sizedatetime_format date_format time_formatfloat_scientificfloat_precision null_value quote_stylerrQrRrSrTrUrrrrrrrrrrrrrrrrrCsvQuoteStyle | NonecdSrrQrrrrrrrrrrrrrrrrrQrRrSrTrUrrrs rsink_csvzLazyFrame.sink_csv s :sr)rrrrrrrrrrrrrrrQrRrSrTrUrrcdSrrQrs rrzLazyFrame.sink_csv s :Crclddlm}|d|d|d|d| sd} t|}ddlm}||||d }~|r"t |}nd}t|}|pd ||d }|j |||t||t|||| | | | | ||||| }|sL| |j }t|}|| dSt|S)a Evaluate the query in streaming mode and write to a CSV file. This allows streaming results that are larger than RAM to be written to disk. Parameters ---------- path File path to which the file should be written. include_bom Whether to include UTF-8 BOM in the CSV output. include_header Whether to include header in the CSV output. separator Separate CSV fields with this symbol. line_terminator String used to end each row. quote_char Byte to use as quoting character. batch_size Number of rows that will be processed per thread. datetime_format A format string, with the specifiers defined by the `chrono `_ Rust crate. If no format specified, the default fractional-second precision is inferred from the maximum timeunit found in the frame's Datetime cols (if any). date_format A format string, with the specifiers defined by the `chrono `_ Rust crate. time_format A format string, with the specifiers defined by the `chrono `_ Rust crate. float_scientific Whether to use scientific form always (true), never (false), or automatically (None) for `Float32` and `Float64` datatypes. float_precision Number of decimal places to write, applied to both `Float32` and `Float64` datatypes. null_value A string representing null values (defaulting to the empty string). quote_style : {'necessary', 'always', 'non_numeric', 'never'} Determines the quoting strategy used. - necessary (default): This puts quotes around fields only when necessary. They are necessary when fields contain a quote, delimiter or record terminator. Quotes are also necessary when writing an empty record (which is indistinguishable from a record with one empty field). This is the default. - always: This puts quotes around every field. Always. - never: This never puts quotes around fields, even if that results in invalid CSV data (e.g.: by not quoting strings containing the separator). - non_numeric: This puts quotes around all fields that are non-numeric. Namely, when writing a field that does not parse as a valid float or integer, then quotes will be used even if they aren`t strictly necessary. maintain_order Maintain the order in which data is processed. Setting this to `False` will be slightly faster. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. sync_on_close: { None, 'data', 'all' } Sync to disk when before closing a file. * `None` does not sync. * `data` syncs the file contents. * `all` syncs the file contents and metadata. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. mkdir: bool Recursively create all the directories in the path. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. lazy: bool Wait to start execution until `collect` is called. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars streaming engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars streaming engine. optimizations The optimization passes done during query optimization. This has no effect if `lazy` is set to `True`. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Returns ------- DataFrame Examples -------- >>> lf = pl.scan_csv("/path/to/my_larger_than_ram_file.csv") # doctest: +SKIP >>> lf.sink_csv("out.csv") # doctest: +SKIP r)_check_arg_is_1byterF) can_be_emptyrNrgrrirj)rnrrrrrrrrrrrrrrorRrSrpr)polars.io.csv._utilsrrrqrhrrrrrordrrrrr)rrrrrrrrrrrrrrrrrQrRrSrTrUrrrrrhrvrnrprs rrzLazyFrame.sink_csv sP =<<<<<KGGGGL*5IIII J''      'H&G  ' ' #  #"?#8#8#:#:;;OO#O &&*4f,  i  #))nn+:!+##-+!#) ;%%!  * (()BCCC'',,C KKvK & & &4$$S)))r) rrQrRrSrTrUrrrc dSrrQ rrrrQrRrSrTrUrrrs r sink_ndjsonzLazyFrame.sink_ndjson s sr)rrQrRrSrTrUrrc dSrrQrs rrzLazyFrame.sink_ndjson s Crc t| } ddlm} | |||d} ~|r"t|}nd}t |} |pd||d}|j| || ||}|sL|| j }t |}| | dSt |S) aX Evaluate the query in streaming mode and write to an NDJSON file. This allows streaming results that are larger than RAM to be written to disk. Parameters ---------- path File path to which the file should be written. maintain_order Maintain the order in which data is processed. Setting this to `False` will be slightly faster. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. storage_options Options that indicate how to connect to a cloud provider. The cloud providers currently supported are AWS, GCP, and Azure. See supported keys here: * `aws `_ * `gcp `_ * `azure `_ * Hugging Face (`hf://`): Accepts an API key under the `token` parameter: `{'token': '...'}`, or by setting the `HF_TOKEN` environment variable. If `storage_options` is not provided, Polars will try to infer the information from environment variables. credential_provider Provide a function that can be called to provide cloud storage credentials. The function is expected to return a dictionary of credential keys along with an optional credential expiry time. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. retries Number of retries if accessing a cloud instance fails. sync_on_close: { None, 'data', 'all' } Sync to disk when before closing a file. * `None` does not sync. * `data` syncs the file contents. * `all` syncs the file contents and metadata. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. mkdir: bool Recursively create all the directories in the path. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. lazy: bool Wait to start execution until `collect` is called. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. engine Select the engine used to process the query, optional. At the moment, if set to `"auto"` (default), the query is run using the polars streaming engine. Polars will also attempt to use the engine set by the `POLARS_ENGINE_AFFINITY` environment variable. If it cannot run the query using the selected engine, the query is run using the polars streaming engine. optimizations The optimization passes done during query optimization. This has no effect if `lazy` is set to `True`. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Returns ------- DataFrame Examples -------- >>> lf = pl.scan_csv("/path/to/my_larger_than_ram_file.csv") # doctest: +SKIP >>> lf.sink_ndjson("out.ndjson") # doctest: +SKIP rrgrNrirj)rnrorRrSrpr) rrqrhrrrr sink_jsonrrrrr)rrrrQrRrSrTrUrrrrhrvrnrprs rrzLazyFrame.sink_ndjson s2P ''      'H&G  ' ' #  #"?#8#8#:#:;;OO#O &&*4f,  i!!) ;% "   (()BCCC'',,C KKvK & & &4$$S)))rzg`LazyFrame.fetch` is deprecated; use `LazyFrame.collect` instead, in conjunction with a call to `head`.) rrrrrrrrrrrn_rowsc D|||||||||| | | |  S)a Collect a small number of rows for debugging purposes. .. deprecated:: 1.0 Use :meth:`collect` instead, in conjunction with a call to :meth:`head`.` Notes ----- This is similar to a :func:`collect` operation, but it overwrites the number of rows read by *every* scan operation. Be aware that `fetch` does not guarantee the final number of rows in the DataFrame. Filters, join operations and fewer rows being available in the scanned data will all influence the final number of rows (joins are especially susceptible to this, and may return no data at all if `n_rows` is too small as the join keys may not be present). Warnings -------- This is strictly a utility function that can help to debug queries using a smaller number of rows, and should *not* be used in production code. ) rrrrrrrrrrrr)_fetch) rrrrrrrrrrrrrs rfetchzLazyFrame.fetch{ sDP{{'#1 3 3+)//!5)   r) rrrrrrrrrrrrc |rd}d}d}d} d} d} d} |}|j||||||| | | | dd| d}t||S)uM Collect a small number of rows for debugging purposes. Do not confuse with `collect`; this function will frequently return incorrect data (see the warning for additional details). Parameters ---------- n_rows Collect n_rows from the data sources. type_coercion Run type coercion optimization. predicate_pushdown Run predicate pushdown optimization. projection_pushdown Run projection pushdown optimization. simplify_expression Run simplify expressions optimization. no_optimization Turn off optimizations. slice_pushdown Slice pushdown optimization comm_subplan_elim Will try to cache branching subplans that occur on self-joins or unions. comm_subexpr_elim Common subexpressions will be cached and reused. cluster_with_columns Combine sequential independent calls to with_columns collapse_joins Collapse a join and filters into a faster join Notes ----- This is similar to a :func:`collect` operation, but it overwrites the number of rows read by *every* scan operation. Be aware that `fetch` does not guarantee the final number of rows in the DataFrame. Filters, join operations and fewer rows being available in the scanned data will all influence the final number of rows (joins are especially susceptible to this, and may return no data at all if `n_rows` is too small as the join keys may not be present). Warnings -------- This is strictly a utility function that can help to debug queries using a smaller number of rows, and should *not* be used in production code. Returns ------- DataFrame Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "b", "c"], ... "b": [1, 2, 3, 4, 5, 6], ... "c": [6, 5, 4, 3, 2, 1], ... } ... ) >>> lf.group_by("a", maintain_order=True).agg(pl.all().sum())._fetch(2) shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 6 │ │ b ┆ 2 ┆ 5 │ └─────┴─────┴─────┘ F)r type_checkrrrrrrrrrrrr)roptimization_toggler5r)rrrrrrrrrrrrrrrlfs rrzLazyFrame._fetch sl  #!& "' "N %  % #( "N Y * *'!1 3 3)//!5)%+   rxx''(((rc|S)aY Return lazy representation, i.e. itself. Useful for writing code that expects either a :class:`DataFrame` or :class:`LazyFrame`. On LazyFrame this is a no-op, and returns the same object. Returns ------- LazyFrame Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [None, 2, 3, 4], ... "b": [0.5, None, 2.5, 13], ... "c": [True, True, False, None], ... } ... ) >>> lf.lazy() # doctest: +ELLIPSIS rQrs rrzLazyFrame.lazy$s . rcZ||jS)z Cache the result once the execution of the physical plan hits this node. It is not recommended using this as the optimizer likely can do a better job. )rrcachers rrzLazyFrame.cache=s$   1 1222rrr`Mapping[ColumnNameOrSelector | PolarsDataType, PolarsDataType | PythonDataType] | PolarsDataTypec t|ts=t|}||j||Si}|D]\}}t|sCt|ts.t|tr(td|Drt|}t|}| t|tr||in(tt!|||||j||S)uS Cast LazyFrame column(s) to the specified dtype(s). Parameters ---------- dtypes Mapping of column names (or selector) to dtypes, or a single dtype to which all columns will be cast. strict Throw an error if a cast could not be done (for instance, due to an overflow). Examples -------- >>> from datetime import date >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": [date(2020, 1, 2), date(2021, 3, 4), date(2022, 5, 6)], ... } ... ) Cast specific frame columns to the specified dtypes: >>> lf.cast({"foo": pl.Float32, "bar": pl.UInt8}).collect() shape: (3, 3) ┌─────┬─────┬────────────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f32 ┆ u8 ┆ date │ ╞═════╪═════╪════════════╡ │ 1.0 ┆ 6 ┆ 2020-01-02 │ │ 2.0 ┆ 7 ┆ 2021-03-04 │ │ 3.0 ┆ 8 ┆ 2022-05-06 │ └─────┴─────┴────────────┘ Cast all frame columns matching one dtype (or dtype group) to another dtype: >>> lf.cast({pl.Date: pl.Datetime}).collect() shape: (3, 3) ┌─────┬─────┬─────────────────────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ datetime[μs] │ ╞═════╪═════╪═════════════════════╡ │ 1 ┆ 6.0 ┆ 2020-01-02 00:00:00 │ │ 2 ┆ 7.0 ┆ 2021-03-04 00:00:00 │ │ 3 ┆ 8.0 ┆ 2022-05-06 00:00:00 │ └─────┴─────┴─────────────────────┘ Use selectors to define the columns being cast: >>> import polars.selectors as cs >>> lf.cast({cs.numeric(): pl.UInt32, cs.temporal(): pl.String}).collect() shape: (3, 3) ┌─────┬─────┬────────────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ u32 ┆ u32 ┆ str │ ╞═════╪═════╪════════════╡ │ 1 ┆ 6 ┆ 2020-01-02 │ │ 2 ┆ 7 ┆ 2021-03-04 │ │ 3 ┆ 8 ┆ 2022-05-06 │ └─────┴─────┴────────────┘ Cast all frame columns to the specified dtype: >>> lf.cast(pl.String).collect().to_dict(as_series=False) {'foo': ['1', '2', '3'], 'bar': ['6.0', '7.0', '8.0'], 'ham': ['2020-01-02', '2021-03-04', '2022-05-06']} c34K|]}t|VdSr)rOrxxs rrzz!LazyFrame.cast..s*1P1P/!2D2D1P1P1P1P1P1Pr)rrrPrrcast_allrrOrQrallr_updaterrfromkeysr`r)rrrcast_maprrs rrzLazyFrame.castEsNh&'** H%f--F##DI$6$6vv$F$FGG G   HAu"" jM&B&B 1j)) .11P1Pa1P1P1P.P.P QKK$U++E OOa%%DE ]]?4#;#;UCC      x @ @AAArrctj||S)u Create an empty copy of the current LazyFrame, with zero to 'n' rows. Returns a copy with an identical schema but no data. Parameters ---------- n Number of (empty) rows to return in the cleared frame. See Also -------- clone : Cheap deepcopy/clone. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [None, 2, 3, 4], ... "b": [0.5, None, 2.5, 13], ... "c": [True, True, False, None], ... } ... ) >>> lf.clear().collect() shape: (0, 3) ┌─────┬─────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool │ ╞═════╪═════╪══════╡ └─────┴─────┴──────┘ >>> lf.clear(2).collect() shape: (2, 3) ┌──────┬──────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ null ┆ null ┆ null │ └──────┴──────┴──────┘ )r)rrlrclearrrrs rrzLazyFrame.clears>X|4#6#6#8#8999??BBGGIIIrcZ||jS)aH Create a copy of this LazyFrame. This is a cheap operation that does not copy data. See Also -------- clear : Create an empty copy of the current LazyFrame, with identical schema but no data. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [None, 2, 3, 4], ... "b": [0.5, None, 2.5, 13], ... "c": [True, True, False, None], ... } ... ) >>> lf.clone() # doctest: +ELLIPSIS )rrr+rs rr+zLazyFrame.clones$.  1 1222r)invert predicates`tuple[IntoExprColumn | Iterable[IntoExprColumn] | bool | list[bool] | np.ndarray[Any, Any], ...] constraintsdict[str, Any]rcg}g}|D]u}|dur|s|dur|s|dur|s|dur|s|cSt|rt|}t|dr/|t j|tt|x}rtd|DsG|st|t j sst|tr|| vsHt|t jr d|jdnt|}d|} t!| |d t%|Dw|d |D|s|sd } t!| |r)t)|d krt+j|n |d nd} |r0t+jt1t2|} | | n| | z} | ||jS|r |jjn |jj} || | jS)z"Common code for filter/remove ops.FT)include_series)rc3LK|]}t|tj V dSr)rrrnrs rrzz$LazyFrame._filter..s1>>qJq"'222>>>>>>ruSeries(…, dtype=)z invalid predicate for `filter`: c34K|]}t|VdSr)r6rs rrzz$LazyFrame._filter..#s9&&%&IaLL&&&&&&rc3jK|].\}}tj||V/dSr)rreq)rxrvalues rrzz$LazyFrame._filter..(sK  &1dEAE$KKNN5 ! !      rz5at least one predicate or constraint must be providedrrN)rr*tupler.rrrr9r/anyrrnrrrreprrrr%rr rall_horizontalrr rrrremovefilter_pyexpr) rrrrall_predicates boolean_masksris_seqerrrcombined_predicate mask_expr filter_methods r_filterzLazyFrame._filters)+   AU v 199V9T f !u**V*zz||###Q !HH $777 $$RYq%@%@%@AAAA&q>>) >>A>>>>>    #1bg..   $As++   12T5H5H5J5J0J0J"!RY//!33333a ?>>nn$%%&&*H*K*K&&&   5@5F5F5H5H       !- !ICC..   ~&&** .11#A&&   fT=99::I&- !33   %##DI.. .,2H (( 8H  .@.H I IJJJrTIntoExprColumn | Iterable[IntoExprColumn] | bool | list[bool] | np.ndarray[Any, Any]c|sd|rt|dkr|ddur|St|dkr|ddur|S|||dS)u. Filter rows in the LazyFrame based on a predicate expression. The original order of the remaining rows is preserved. Rows where the filter predicate does not evaluate to True are discarded (this includes rows where the predicate evaluates as `null`). Parameters ---------- predicates Expression that evaluates to a boolean Series. constraints Column filters; use `name = value` to filter columns using the supplied value. Each constraint behaves the same as `pl.col(name).eq(value)`, and is implicitly joined with the other filter conditions using `&`. Notes ----- If you are transitioning from Pandas, and performing filter operations based on the comparison of two or more columns, please note that in Polars any comparison involving `null` values will result in a `null` result, *not* boolean True or False. As a result, these rows will not be retained. Ensure that null values are handled appropriately to avoid unexpected behaviour (see examples below). See Also -------- remove Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3, None, 4, None, 0], ... "bar": [6, 7, 8, None, None, 9, 0], ... "ham": ["a", "b", "c", None, "d", "e", "f"], ... } ... ) Filter on one condition: >>> lf.filter(pl.col("foo") > 1).collect() shape: (3, 3) ┌─────┬──────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪══════╪═════╡ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ │ 4 ┆ null ┆ d │ └─────┴──────┴─────┘ Filter on multiple conditions: >>> lf.filter((pl.col("foo") < 3) & (pl.col("ham") == "a")).collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ Provide multiple filters using `*args` syntax: >>> lf.filter( ... pl.col("foo") == 1, ... pl.col("ham") == "a", ... ).collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ Provide multiple filters using `**kwargs` syntax: >>> lf.filter(foo=1, ham="a").collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ Filter on an OR condition: >>> lf.filter( ... (pl.col("foo") == 1) | (pl.col("ham") == "c"), ... ).collect() shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ Filter by comparing two columns against each other >>> lf.filter( ... pl.col("foo") == pl.col("bar"), ... ).collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 0 ┆ 0 ┆ f │ └─────┴─────┴─────┘ >>> lf.filter( ... pl.col("foo") != pl.col("bar"), ... ).collect() shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ Notice how the row with `None` values is filtered out; using `ne_missing` ensures that null values compare equal, and we get similar behaviour to Pandas: >>> lf.filter( ... pl.col("foo").ne_missing(pl.col("bar")), ... ).collect() shape: (5, 3) ┌──────┬──────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ │ 4 ┆ null ┆ d │ │ null ┆ 9 ┆ e │ └──────┴──────┴─────┘ rrTFrrr)r r+rrrrrs rrzLazyFrame.filterIsJ $ $#j//Q"6"6:a=D;P;Pzz||#:!## 1 (>(>zz||#||!#   rc|sd|rt|dkr|ddur|St|dkr|ddur|S|||dS)us Remove rows, dropping those that match the given predicate expression(s). The original order of the remaining rows is preserved. Rows where the filter predicate does not evaluate to True are retained (this includes rows where the predicate evaluates as `null`). Parameters ---------- predicates Expression that evaluates to a boolean Series. constraints Column filters; use `name = value` to filter columns using the supplied value. Each constraint behaves the same as `pl.col(name).eq(value)`, and is implicitly joined with the other filter conditions using `&`. Notes ----- If you are transitioning from Pandas, and performing filter operations based on the comparison of two or more columns, please note that in Polars any comparison involving `null` values will result in a `null` result, *not* boolean True or False. As a result, these rows will not be removed. Ensure that null values are handled appropriately to avoid unexpected behaviour (see examples below). See Also -------- filter Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [2, 3, None, 4, 0], ... "bar": [5, 6, None, None, 0], ... "ham": ["a", "b", None, "c", "d"], ... } ... ) Remove rows matching a condition: >>> lf.remove( ... pl.col("bar") >= 5, ... ).collect() shape: (3, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ │ 0 ┆ 0 ┆ d │ └──────┴──────┴──────┘ Discard rows based on multiple conditions, combined with and/or operators: >>> lf.remove( ... (pl.col("foo") >= 0) & (pl.col("bar") >= 0), ... ).collect() shape: (2, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ └──────┴──────┴──────┘ >>> lf.remove( ... (pl.col("foo") >= 0) | (pl.col("bar") >= 0), ... ).collect() shape: (1, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ └──────┴──────┴──────┘ Provide multiple constraints using `*args` syntax: >>> lf.remove( ... pl.col("ham").is_not_null(), ... pl.col("bar") >= 0, ... ).collect() shape: (2, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ └──────┴──────┴──────┘ Provide constraints(s) using `**kwargs` syntax: >>> lf.remove(foo=0, bar=0).collect() shape: (4, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ 2 ┆ 5 ┆ a │ │ 3 ┆ 6 ┆ b │ │ null ┆ null ┆ null │ │ 4 ┆ null ┆ c │ └──────┴──────┴──────┘ Remove rows by comparing two columns against each other; in this case, we remove rows where the two columns are not equal (using `ne_missing` to ensure that null values compare equal): >>> lf.remove( ... pl.col("foo").ne_missing(pl.col("bar")), ... ).collect() shape: (2, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞══════╪══════╪══════╡ │ null ┆ null ┆ null │ │ 0 ┆ 0 ┆ d │ └──────┴──────┴──────┘ rrTFr)r rr+rrs rrzLazyFrame.removesZ $ $#j//Q"6"6:a=D;P;Pzz||#:!## 1 (>(>zz||#||!#   rexprs named_exprsctttjdd}t |i|d|i}||j|S)u Select columns from this LazyFrame. Parameters ---------- *exprs Column(s) to select, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to select, specified as keyword arguments. The columns will be renamed to the keyword used. Examples -------- Pass the name of a column to select that column. >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.select("foo").collect() shape: (3, 1) ┌─────┐ │ foo │ │ --- │ │ i64 │ ╞═════╡ │ 1 │ │ 2 │ │ 3 │ └─────┘ Multiple columns can be selected by passing a list of column names. >>> lf.select(["foo", "bar"]).collect() shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 6 │ │ 2 ┆ 7 │ │ 3 ┆ 8 │ └─────┴─────┘ Multiple columns can also be selected using positional arguments instead of a list. Expressions are also accepted. >>> lf.select(pl.col("foo"), pl.col("bar") + 1).collect() shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 7 │ │ 2 ┆ 8 │ │ 3 ┆ 9 │ └─────┴─────┘ Use keyword arguments to easily name your expression inputs. >>> lf.select( ... threshold=pl.when(pl.col("foo") > 2).then(10).otherwise(0) ... ).collect() shape: (3, 1) ┌───────────┐ │ threshold │ │ --- │ │ i32 │ ╞═══════════╡ │ 0 │ │ 0 │ │ 10 │ └───────────┘ Expressions with multiple outputs can be automatically instantiated as Structs by enabling the setting `Config.set_auto_structify(True)`: >>> with pl.Config(auto_structify=True): ... lf.select( ... is_odd=(pl.col(pl.Int64) % 2 == 1).name.suffix("_is_odd"), ... ).collect() shape: (3, 1) ┌──────────────┐ │ is_odd │ │ --- │ │ struct[2] │ ╞══════════════╡ │ {true,false} │ │ {false,true} │ │ {true,false} │ └──────────────┘ POLARS_AUTO_STRUCTIFYr __structify) rrosenvironr!r%rrrrrr structifypyexprss rrzLazyFrame.selects}LRZ^^,CQGGHHII 0  !  /8    0 0 9 9:::rctttjdd}t |i|d|i}||j|S)a Select columns from this LazyFrame. This will run all expression sequentially instead of in parallel. Use this when the work per expression is cheap. Parameters ---------- *exprs Column(s) to select, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to select, specified as keyword arguments. The columns will be renamed to the keyword used. See Also -------- select rrr) rrrrr!r%rr select_seqrs rrzLazyFrame.select_seqs|.RZ^^,CQGGHHII 0  !  /8    4 4W = =>>>r)rnamed_byrZcD|D]V}t|ttjtjfs(dt |d|d|d}t|Wt|i|}|j ||}t|S)u Start a group by operation. Parameters ---------- *by Column(s) to group by. Accepts expression input. Strings are parsed as column names. maintain_order Ensure that the order of the groups is consistent with the input data. This is slower than a default group by. Setting this to `True` blocks the possibility to run on the streaming engine. **named_by Additional columns to group by, specified as keyword arguments. The columns will be renamed to the keyword used. Examples -------- Group by one column and call `agg` to compute the grouped sum of another column. >>> lf = pl.LazyFrame( ... { ... "a": ["a", "b", "a", "b", "c"], ... "b": [1, 2, 1, 3, 3], ... "c": [5, 4, 3, 2, 1], ... } ... ) >>> lf.group_by("a").agg(pl.col("b").sum()).collect() # doctest: +IGNORE_RESULT shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ a ┆ 2 │ │ b ┆ 5 │ │ c ┆ 3 │ └─────┴─────┘ Set `maintain_order=True` to ensure the order of the groups is consistent with the input. >>> lf.group_by("a", maintain_order=True).agg(pl.col("c")).collect() shape: (3, 2) ┌─────┬───────────┐ │ a ┆ c │ │ --- ┆ --- │ │ str ┆ list[i64] │ ╞═════╪═══════════╡ │ a ┆ [5, 3] │ │ b ┆ [4, 2] │ │ c ┆ [1] │ └─────┴───────────┘ Group by multiple columns by passing a list of column names. >>> lf.group_by(["a", "b"]).agg(pl.max("c")).collect() # doctest: +SKIP shape: (4, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 1 ┆ 5 │ │ b ┆ 2 ┆ 4 │ │ b ┆ 3 ┆ 2 │ │ c ┆ 3 ┆ 1 │ └─────┴─────┴─────┘ Or use positional arguments to group by multiple columns in the same way. Expressions are also accepted. >>> lf.group_by("a", pl.col("b") // 2).agg( ... pl.col("c").mean() ... ).collect() # doctest: +SKIP shape: (3, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ f64 │ ╞═════╪═════╪═════╡ │ a ┆ 0 ┆ 4.0 │ │ b ┆ 1 ┆ 3.0 │ │ c ┆ 1 ┆ 1.0 │ └─────┴─────┴─────┘ z=Expected Polars expression or object convertible to one, got z&. Hint: if you tried group_by(by=z;) then you probably want to use this instead: group_by(r) valuesrrrrnrrurr%rgroup_byrZ)rrrrrrrlgbs rrzLazyFrame.group_by s|__&& % %Eec27BI%>?? %/TXY^T_T_//',//%* /// nn$ %/?h??i  773rrz0.20.14right)offsetclosedr index_columnperiodstr | timedeltarstr | timedelta | Nonerrp$IntoExpr | Iterable[IntoExpr] | Nonect|}|tt|}|t|ng}t|}t|}|j|||||}t |S)u Create rolling groups based on a temporal or integer column. Different from a `group_by_dynamic` the windows are now determined by the individual values and are not of constant intervals. For constant intervals use :func:`LazyFrame.group_by_dynamic`. If you have a time series ``, then by default the windows created will be * (t_0 - period, t_0] * (t_1 - period, t_1] * ... * (t_n - period, t_n] whereas if you pass a non-default `offset`, then the windows will be * (t_0 + offset, t_0 + offset + period] * (t_1 + offset, t_1 + offset + period] * ... * (t_n + offset, t_n + offset + period] The `period` and `offset` arguments are created either from a timedelta, or by using the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) - 1i (1 index count) Or combine them: "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". .. versionchanged:: 0.20.14 The `by` parameter was renamed `group_by`. Parameters ---------- index_column Column used to group based on the time window. Often of type Date/Datetime. This column must be sorted in ascending order (or, if `group_by` is specified, then it must be sorted in ascending order within each group). In case of a rolling group by on indices, dtype needs to be one of {UInt32, UInt64, Int32, Int64}. Note that the first three get temporarily cast to Int64, so if performance matters use an Int64 column. period Length of the window - must be non-negative. offset Offset of the window. Default is `-period`. closed : {'right', 'left', 'both', 'none'} Define which sides of the temporal interval are closed (inclusive). group_by Also group by this column/these columns Returns ------- LazyGroupBy Object you can call `.agg` on to aggregate by groups, the result of which will be sorted by `index_column` (but note that if `group_by` columns are passed, it will only be sorted within each group). See Also -------- group_by_dynamic Examples -------- >>> dates = [ ... "2020-01-01 13:45:48", ... "2020-01-01 16:42:13", ... "2020-01-01 16:45:09", ... "2020-01-02 18:12:48", ... "2020-01-03 19:45:32", ... "2020-01-08 23:16:43", ... ] >>> df = pl.LazyFrame({"dt": dates, "a": [3, 7, 5, 9, 2, 1]}).with_columns( ... pl.col("dt").str.strptime(pl.Datetime).set_sorted() ... ) >>> out = ( ... df.rolling(index_column="dt", period="2d") ... .agg( ... pl.sum("a").alias("sum_a"), ... pl.min("a").alias("min_a"), ... pl.max("a").alias("max_a"), ... ) ... .collect() ... ) >>> out shape: (6, 4) ┌─────────────────────┬───────┬───────┬───────┐ │ dt ┆ sum_a ┆ min_a ┆ max_a │ │ --- ┆ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ i64 ┆ i64 ┆ i64 │ ╞═════════════════════╪═══════╪═══════╪═══════╡ │ 2020-01-01 13:45:48 ┆ 3 ┆ 3 ┆ 3 │ │ 2020-01-01 16:42:13 ┆ 10 ┆ 3 ┆ 7 │ │ 2020-01-01 16:45:09 ┆ 15 ┆ 3 ┆ 7 │ │ 2020-01-02 18:12:48 ┆ 24 ┆ 3 ┆ 9 │ │ 2020-01-03 19:45:32 ┆ 11 ┆ 2 ┆ 9 │ │ 2020-01-08 23:16:43 ┆ 1 ┆ 1 ┆ 1 │ └─────────────────────┴───────┴───────┴───────┘ )r$rrr%rrollingrZ)rrrrrr pyexprs_byrs rrzLazyFrame.rollings|-\:: >+,DV,L,LMMF9A8L *8 4 4 4RT *&11)&11i fffjQQ3rrwindow)rrinclude_boundariesrlabelrstart_byeveryrrr|rrc t|}|d}||}t|}t|}t|}|t|ng} |j|||||||| | } t | S)u= Group based on a time value (or index value of type Int32, Int64). Time windows are calculated and rows are assigned to windows. Different from a normal group by is that a row can be member of multiple groups. By default, the windows look like: - [start, start + period) - [start + every, start + every + period) - [start + 2*every, start + 2*every + period) - ... where `start` is determined by `start_by`, `offset`, `every`, and the earliest datapoint. See the `start_by` argument description for details. .. warning:: The index column must be sorted in ascending order. If `group_by` is passed, then the index column must be sorted in ascending order within each group. .. versionchanged:: 0.20.14 The `by` parameter was renamed `group_by`. Parameters ---------- index_column Column used to group based on the time window. Often of type Date/Datetime. This column must be sorted in ascending order (or, if `group_by` is specified, then it must be sorted in ascending order within each group). In case of a dynamic group by on indices, dtype needs to be one of {Int32, Int64}. Note that Int32 gets temporarily cast to Int64, so if performance matters use an Int64 column. every interval of the window period length of the window, if None it will equal 'every' offset offset of the window, does not take effect if `start_by` is 'datapoint'. Defaults to zero. include_boundaries Add the lower and upper bound of the window to the "_lower_boundary" and "_upper_boundary" columns. This will impact performance because it's harder to parallelize closed : {'left', 'right', 'both', 'none'} Define which sides of the temporal interval are closed (inclusive). label : {'left', 'right', 'datapoint'} Define which label to use for the window: - 'left': lower boundary of the window - 'right': upper boundary of the window - 'datapoint': the first value of the index column in the given window. If you don't need the label to be at one of the boundaries, choose this option for maximum performance group_by Also group by this column/these columns start_by : {'window', 'datapoint', 'monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday'} The strategy to determine the start of the first window by. * 'window': Start by taking the earliest timestamp, truncating it with `every`, and then adding `offset`. Note that weekly windows start on Monday. * 'datapoint': Start from the first encountered data point. * a day of the week (only takes effect if `every` contains `'w'`): * 'monday': Start the window on the Monday before the first data point. * 'tuesday': Start the window on the Tuesday before the first data point. * ... * 'sunday': Start the window on the Sunday before the first data point. The resulting window is then shifted back until the earliest datapoint is in or in front of it. Returns ------- LazyGroupBy Object you can call `.agg` on to aggregate by groups, the result of which will be sorted by `index_column` (but note that if `group_by` columns are passed, it will only be sorted within each group). See Also -------- rolling Notes ----- 1) If you're coming from pandas, then .. code-block:: python # polars df.group_by_dynamic("ts", every="1d").agg(pl.col("value").sum()) is equivalent to .. code-block:: python # pandas df.set_index("ts").resample("D")["value"].sum().reset_index() though note that, unlike pandas, polars doesn't add extra rows for empty windows. If you need `index_column` to be evenly spaced, then please combine with :func:`DataFrame.upsample`. 2) The `every`, `period` and `offset` arguments are created with the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) - 1i (1 index count) Or combine them: "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". In case of a group_by_dynamic on an integer column, the windows are defined by: - "1i" # length 1 - "10i" # length 10 Examples -------- >>> from datetime import datetime >>> lf = pl.LazyFrame( ... { ... "time": pl.datetime_range( ... start=datetime(2021, 12, 16), ... end=datetime(2021, 12, 16, 3), ... interval="30m", ... eager=True, ... ), ... "n": range(7), ... } ... ) >>> lf.collect() shape: (7, 2) ┌─────────────────────┬─────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ i64 │ ╞═════════════════════╪═════╡ │ 2021-12-16 00:00:00 ┆ 0 │ │ 2021-12-16 00:30:00 ┆ 1 │ │ 2021-12-16 01:00:00 ┆ 2 │ │ 2021-12-16 01:30:00 ┆ 3 │ │ 2021-12-16 02:00:00 ┆ 4 │ │ 2021-12-16 02:30:00 ┆ 5 │ │ 2021-12-16 03:00:00 ┆ 6 │ └─────────────────────┴─────┘ Group by windows of 1 hour. >>> lf.group_by_dynamic("time", every="1h", closed="right").agg( ... pl.col("n") ... ).collect() shape: (4, 2) ┌─────────────────────┬───────────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ list[i64] │ ╞═════════════════════╪═══════════╡ │ 2021-12-15 23:00:00 ┆ [0] │ │ 2021-12-16 00:00:00 ┆ [1, 2] │ │ 2021-12-16 01:00:00 ┆ [3, 4] │ │ 2021-12-16 02:00:00 ┆ [5, 6] │ └─────────────────────┴───────────┘ The window boundaries can also be added to the aggregation result >>> lf.group_by_dynamic( ... "time", every="1h", include_boundaries=True, closed="right" ... ).agg(pl.col("n").mean()).collect() shape: (4, 4) ┌─────────────────────┬─────────────────────┬─────────────────────┬─────┐ │ _lower_boundary ┆ _upper_boundary ┆ time ┆ n │ │ --- ┆ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ datetime[μs] ┆ datetime[μs] ┆ f64 │ ╞═════════════════════╪═════════════════════╪═════════════════════╪═════╡ │ 2021-12-15 23:00:00 ┆ 2021-12-16 00:00:00 ┆ 2021-12-15 23:00:00 ┆ 0.0 │ │ 2021-12-16 00:00:00 ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 00:00:00 ┆ 1.5 │ │ 2021-12-16 01:00:00 ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 01:00:00 ┆ 3.5 │ │ 2021-12-16 02:00:00 ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 02:00:00 ┆ 5.5 │ └─────────────────────┴─────────────────────┴─────────────────────┴─────┘ When closed="left", the window excludes the right end of interval: [lower_bound, upper_bound) >>> lf.group_by_dynamic("time", every="1h", closed="left").agg( ... pl.col("n") ... ).collect() shape: (4, 2) ┌─────────────────────┬───────────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ list[i64] │ ╞═════════════════════╪═══════════╡ │ 2021-12-16 00:00:00 ┆ [0, 1] │ │ 2021-12-16 01:00:00 ┆ [2, 3] │ │ 2021-12-16 02:00:00 ┆ [4, 5] │ │ 2021-12-16 03:00:00 ┆ [6] │ └─────────────────────┴───────────┘ When closed="both" the time values at the window boundaries belong to 2 groups. >>> lf.group_by_dynamic("time", every="1h", closed="both").agg( ... pl.col("n") ... ).collect() shape: (4, 2) ┌─────────────────────┬───────────┐ │ time ┆ n │ │ --- ┆ --- │ │ datetime[μs] ┆ list[i64] │ ╞═════════════════════╪═══════════╡ │ 2021-12-16 00:00:00 ┆ [0, 1, 2] │ │ 2021-12-16 01:00:00 ┆ [2, 3, 4] │ │ 2021-12-16 02:00:00 ┆ [4, 5, 6] │ │ 2021-12-16 03:00:00 ┆ [6] │ └─────────────────────┴───────────┘ Dynamic group bys can also be combined with grouping on normal keys >>> lf = lf.with_columns(groups=pl.Series(["a", "a", "a", "b", "b", "a", "a"])) >>> lf.collect() shape: (7, 3) ┌─────────────────────┬─────┬────────┐ │ time ┆ n ┆ groups │ │ --- ┆ --- ┆ --- │ │ datetime[μs] ┆ i64 ┆ str │ ╞═════════════════════╪═════╪════════╡ │ 2021-12-16 00:00:00 ┆ 0 ┆ a │ │ 2021-12-16 00:30:00 ┆ 1 ┆ a │ │ 2021-12-16 01:00:00 ┆ 2 ┆ a │ │ 2021-12-16 01:30:00 ┆ 3 ┆ b │ │ 2021-12-16 02:00:00 ┆ 4 ┆ b │ │ 2021-12-16 02:30:00 ┆ 5 ┆ a │ │ 2021-12-16 03:00:00 ┆ 6 ┆ a │ └─────────────────────┴─────┴────────┘ >>> lf.group_by_dynamic( ... "time", ... every="1h", ... closed="both", ... group_by="groups", ... include_boundaries=True, ... ).agg(pl.col("n")).collect() shape: (6, 5) ┌────────┬─────────────────────┬─────────────────────┬─────────────────────┬───────────┐ │ groups ┆ _lower_boundary ┆ _upper_boundary ┆ time ┆ n │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ datetime[μs] ┆ datetime[μs] ┆ datetime[μs] ┆ list[i64] │ ╞════════╪═════════════════════╪═════════════════════╪═════════════════════╪═══════════╡ │ a ┆ 2021-12-16 00:00:00 ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 00:00:00 ┆ [0, 1, 2] │ │ a ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 01:00:00 ┆ [2] │ │ a ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 02:00:00 ┆ [5, 6] │ │ a ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 04:00:00 ┆ 2021-12-16 03:00:00 ┆ [6] │ │ b ┆ 2021-12-16 01:00:00 ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 01:00:00 ┆ [3, 4] │ │ b ┆ 2021-12-16 02:00:00 ┆ 2021-12-16 03:00:00 ┆ 2021-12-16 02:00:00 ┆ [4] │ └────────┴─────────────────────┴─────────────────────┴─────────────────────┴───────────┘ Dynamic group by on an index column >>> lf = pl.LazyFrame( ... { ... "idx": pl.int_range(0, 6, eager=True), ... "A": ["A", "A", "B", "B", "B", "C"], ... } ... ) >>> lf.group_by_dynamic( ... "idx", ... every="2i", ... period="3i", ... include_boundaries=True, ... closed="right", ... ).agg(pl.col("A").alias("A_agg_list")).collect() shape: (4, 4) ┌─────────────────┬─────────────────┬─────┬─────────────────┐ │ _lower_boundary ┆ _upper_boundary ┆ idx ┆ A_agg_list │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ list[str] │ ╞═════════════════╪═════════════════╪═════╪═════════════════╡ │ -2 ┆ 1 ┆ -2 ┆ ["A", "A"] │ │ 0 ┆ 3 ┆ 0 ┆ ["A", "B", "B"] │ │ 2 ┆ 5 ┆ 2 ┆ ["B", "B", "C"] │ │ 4 ┆ 7 ┆ 4 ┆ ["C"] │ └─────────────────┴─────────────────┴─────┴─────────────────┘ N0ns)r$rr%rgroup_by_dynamicrZ) rrrrrrrrrrrrs rrzLazyFrame.group_by_dynamicsn -\:: >F >F)&11)&11(//9A8L *8 4 4 4RT i((            3rbackward_right)left_onright_ononby_leftby_rightrstrategysuffix toleranceallow_parallelforce_parallelcoalesceallow_exact_matchescheck_sortednessrstr | None | Exprrrrstr | Sequence[str] | Nonerrrorr $str | int | float | timedelta | Noner r r r rc t||t|ttjfr|}|}||d}t ||t|tr|gn|}|}n=||5t|tr|gn|}t|tr|gn|}nd}d}d}d}t| tr| }n't| t rt| }n| }t|tjstj |}t|tjstj |}| |j |j |j |j ||| | | |||| ||S)u/9 Perform an asof join. This is similar to a left-join except that we match on nearest key rather than equal keys. Both DataFrames must be sorted by the `on` key (within each `by` group, if specified). For each row in the left DataFrame: - A "backward" search selects the last row in the right DataFrame whose 'on' key is less than or equal to the left's key. - A "forward" search selects the first row in the right DataFrame whose 'on' key is greater than or equal to the left's key. A "nearest" search selects the last row in the right DataFrame whose value is nearest to the left's key. String keys are not currently supported for a nearest search. The default is "backward". Parameters ---------- other Lazy DataFrame to join with. left_on Join column of the left DataFrame. right_on Join column of the right DataFrame. on Join column of both DataFrames. If set, `left_on` and `right_on` should be None. by Join on these columns before doing asof join. by_left Join on these columns before doing asof join. by_right Join on these columns before doing asof join. strategy : {'backward', 'forward', 'nearest'} Join strategy. suffix Suffix to append to columns with a duplicate name. tolerance Numeric tolerance. By setting this the join will only be done if the near keys are within this distance. If an asof join is done on columns of dtype "Date", "Datetime", "Duration" or "Time", use either a datetime.timedelta object or the following string language: - 1ns (1 nanosecond) - 1us (1 microsecond) - 1ms (1 millisecond) - 1s (1 second) - 1m (1 minute) - 1h (1 hour) - 1d (1 calendar day) - 1w (1 calendar week) - 1mo (1 calendar month) - 1q (1 calendar quarter) - 1y (1 calendar year) Or combine them: "3d12h4m25s" # 3 days, 12 hours, 4 minutes, and 25 seconds By "calendar day", we mean the corresponding time on the next day (which may not be 24 hours, due to daylight savings). Similarly for "calendar week", "calendar month", "calendar quarter", and "calendar year". allow_parallel Allow the physical plan to optionally evaluate the computation of both DataFrames up to the join in parallel. force_parallel Force the physical plan to evaluate the computation of both DataFrames up to the join in parallel. coalesce Coalescing behavior (merging of `on` / `left_on` / `right_on` columns): - True: -> Always coalesce join columns. - False: -> Never coalesce join columns. Note that joining on any other expressions than `col` will turn off coalescing. allow_exact_matches Whether exact matches are valid join predicates. - If True, allow matching with the same ``on`` value (i.e. less-than-or-equal-to / greater-than-or-equal-to) - If False, don't match the same ``on`` value (i.e., strictly less-than / strictly greater-than). check_sortedness Check the sortedness of the asof keys. If the keys are not sorted Polars will error. Currently, sortedness cannot be checked if 'by' groups are provided. Examples -------- >>> from datetime import date >>> gdp = pl.LazyFrame( ... { ... "date": pl.date_range( ... date(2016, 1, 1), ... date(2020, 1, 1), ... "1y", ... eager=True, ... ), ... "gdp": [4164, 4411, 4566, 4696, 4827], ... } ... ) >>> gdp.collect() shape: (5, 2) ┌────────────┬──────┐ │ date ┆ gdp │ │ --- ┆ --- │ │ date ┆ i64 │ ╞════════════╪══════╡ │ 2016-01-01 ┆ 4164 │ │ 2017-01-01 ┆ 4411 │ │ 2018-01-01 ┆ 4566 │ │ 2019-01-01 ┆ 4696 │ │ 2020-01-01 ┆ 4827 │ └────────────┴──────┘ >>> population = pl.LazyFrame( ... { ... "date": [date(2016, 3, 1), date(2018, 8, 1), date(2019, 1, 1)], ... "population": [82.19, 82.66, 83.12], ... } ... ).sort("date") >>> population.collect() shape: (3, 2) ┌────────────┬────────────┐ │ date ┆ population │ │ --- ┆ --- │ │ date ┆ f64 │ ╞════════════╪════════════╡ │ 2016-03-01 ┆ 82.19 │ │ 2018-08-01 ┆ 82.66 │ │ 2019-01-01 ┆ 83.12 │ └────────────┴────────────┘ Note how the dates don't quite match. If we join them using `join_asof` and `strategy='backward'`, then each date from `population` which doesn't have an exact match is matched with the closest earlier date from `gdp`: >>> population.join_asof(gdp, on="date", strategy="backward").collect() shape: (3, 3) ┌────────────┬────────────┬──────┐ │ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ i64 │ ╞════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 4164 │ │ 2018-08-01 ┆ 82.66 ┆ 4566 │ │ 2019-01-01 ┆ 83.12 ┆ 4696 │ └────────────┴────────────┴──────┘ Note how: - date `2016-03-01` from `population` is matched with `2016-01-01` from `gdp`; - date `2018-08-01` from `population` is matched with `2018-01-01` from `gdp`. You can verify this by passing `coalesce=False`: >>> population.join_asof( ... gdp, on="date", strategy="backward", coalesce=False ... ).collect() shape: (3, 4) ┌────────────┬────────────┬────────────┬──────┐ │ date ┆ population ┆ date_right ┆ gdp │ │ --- ┆ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ date ┆ i64 │ ╞════════════╪════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 2016-01-01 ┆ 4164 │ │ 2018-08-01 ┆ 82.66 ┆ 2018-01-01 ┆ 4566 │ │ 2019-01-01 ┆ 83.12 ┆ 2019-01-01 ┆ 4696 │ └────────────┴────────────┴────────────┴──────┘ If we instead use `strategy='forward'`, then each date from `population` which doesn't have an exact match is matched with the closest later date from `gdp`: >>> population.join_asof(gdp, on="date", strategy="forward").collect() shape: (3, 3) ┌────────────┬────────────┬──────┐ │ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ i64 │ ╞════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 4411 │ │ 2018-08-01 ┆ 82.66 ┆ 4696 │ │ 2019-01-01 ┆ 83.12 ┆ 4696 │ └────────────┴────────────┴──────┘ Note how: - date `2016-03-01` from `population` is matched with `2017-01-01` from `gdp`; - date `2018-08-01` from `population` is matched with `2019-01-01` from `gdp`. Finally, `strategy='nearest'` gives us a mix of the two results above, as each date from `population` which doesn't have an exact match is matched with the closest date from `gdp`, regardless of whether it's earlier or later: >>> population.join_asof(gdp, on="date", strategy="nearest").collect() shape: (3, 3) ┌────────────┬────────────┬──────┐ │ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- │ │ date ┆ f64 ┆ i64 │ ╞════════════╪════════════╪══════╡ │ 2016-03-01 ┆ 82.19 ┆ 4164 │ │ 2018-08-01 ┆ 82.66 ┆ 4696 │ │ 2019-01-01 ┆ 83.12 ┆ 4696 │ └────────────┴────────────┴──────┘ Note how: - date `2016-03-01` from `population` is matched with `2016-01-01` from `gdp`; - date `2018-08-01` from `population` is matched with `2019-01-01` from `gdp`. They `by` argument allows joining on another column first, before the asof join. In this example we join by `country` first, then asof join by date, as above. >>> gdp_dates = pl.date_range( # fmt: skip ... date(2016, 1, 1), date(2020, 1, 1), "1y", eager=True ... ) >>> gdp2 = pl.LazyFrame( ... { ... "country": ["Germany"] * 5 + ["Netherlands"] * 5, ... "date": pl.concat([gdp_dates, gdp_dates]), ... "gdp": [4164, 4411, 4566, 4696, 4827, 784, 833, 914, 910, 909], ... } ... ).sort("country", "date") >>> >>> gdp2.collect() shape: (10, 3) ┌─────────────┬────────────┬──────┐ │ country ┆ date ┆ gdp │ │ --- ┆ --- ┆ --- │ │ str ┆ date ┆ i64 │ ╞═════════════╪════════════╪══════╡ │ Germany ┆ 2016-01-01 ┆ 4164 │ │ Germany ┆ 2017-01-01 ┆ 4411 │ │ Germany ┆ 2018-01-01 ┆ 4566 │ │ Germany ┆ 2019-01-01 ┆ 4696 │ │ Germany ┆ 2020-01-01 ┆ 4827 │ │ Netherlands ┆ 2016-01-01 ┆ 784 │ │ Netherlands ┆ 2017-01-01 ┆ 833 │ │ Netherlands ┆ 2018-01-01 ┆ 914 │ │ Netherlands ┆ 2019-01-01 ┆ 910 │ │ Netherlands ┆ 2020-01-01 ┆ 909 │ └─────────────┴────────────┴──────┘ >>> pop2 = pl.LazyFrame( ... { ... "country": ["Germany"] * 3 + ["Netherlands"] * 3, ... "date": [ ... date(2016, 3, 1), ... date(2018, 8, 1), ... date(2019, 1, 1), ... date(2016, 3, 1), ... date(2018, 8, 1), ... date(2019, 1, 1), ... ], ... "population": [82.19, 82.66, 83.12, 17.11, 17.32, 17.40], ... } ... ).sort("country", "date") >>> >>> pop2.collect() shape: (6, 3) ┌─────────────┬────────────┬────────────┐ │ country ┆ date ┆ population │ │ --- ┆ --- ┆ --- │ │ str ┆ date ┆ f64 │ ╞═════════════╪════════════╪════════════╡ │ Germany ┆ 2016-03-01 ┆ 82.19 │ │ Germany ┆ 2018-08-01 ┆ 82.66 │ │ Germany ┆ 2019-01-01 ┆ 83.12 │ │ Netherlands ┆ 2016-03-01 ┆ 17.11 │ │ Netherlands ┆ 2018-08-01 ┆ 17.32 │ │ Netherlands ┆ 2019-01-01 ┆ 17.4 │ └─────────────┴────────────┴────────────┘ >>> pop2.join_asof(gdp2, by="country", on="date", strategy="nearest").collect() shape: (6, 4) ┌─────────────┬────────────┬────────────┬──────┐ │ country ┆ date ┆ population ┆ gdp │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ date ┆ f64 ┆ i64 │ ╞═════════════╪════════════╪════════════╪══════╡ │ Germany ┆ 2016-03-01 ┆ 82.19 ┆ 4164 │ │ Germany ┆ 2018-08-01 ┆ 82.66 ┆ 4696 │ │ Germany ┆ 2019-01-01 ┆ 83.12 ┆ 4696 │ │ Netherlands ┆ 2016-03-01 ┆ 17.11 ┆ 784 │ │ Netherlands ┆ 2018-08-01 ┆ 17.32 ┆ 910 │ │ Netherlands ┆ 2019-01-01 ┆ 17.4 ┆ 910 │ └─────────────┴────────────┴────────────┴──────┘ Nz4you should pass the column to join on as an argument)r allow_eqr)r4rrrrnrr rrrrr join_asofr)rrrrrrrrrrr r r r r rrby_left_ by_right_ tolerance_str tolerance_nums rrzLazyFrame.join_asofisx $&&& b3. ) ) GH ?h.HCS// ! >)"c22:ttH II!x';$.w$<$<Iyy'H&03&?&?M XIIHI$( ,0 i % % &%MM  9 - - &4Y??MM%M'27++ %eGnnG(BG,, 'uXH I    !,!1      r join_nulls nulls_equalz1.24innerzm:m) rrrvalidaterr rr r (str | Expr | Sequence[str | Expr] | Nonehowrzrr{MaintainOrderJoin | Nonec t||| d} |du} |du}|du}|p|}| r|rd}t|||krd}t||dkrd}tdd nr|d krd } d}td d nV|d krP| s|rd}t|||j|jgg| | ||||| S| rt |}|}|}n2|rt |}t |}nd}t|||j|j||| | ||||| | S)u" Add a join operation to the Logical Plan. .. versionchanged:: 1.24 The `join_nulls` parameter was renamed `nulls_equal`. Parameters ---------- other Lazy DataFrame to join with. on Name(s) of the join columns in both DataFrames. If set, `left_on` and `right_on` should be None. This should not be specified if `how='cross'`. how : {'inner','left', 'right', 'full', 'semi', 'anti', 'cross'} Join strategy. .. list-table :: :header-rows: 0 * - **inner** - *(Default)* Returns rows that have matching values in both tables. * - **left** - Returns all rows from the left table, and the matched rows from the right table. * - **full** - Returns all rows when there is a match in either left or right. * - **cross** - Returns the Cartesian product of rows from both tables * - **semi** - Returns rows from the left table that have a match in the right table. * - **anti** - Returns rows from the left table that have no match in the right table. left_on Join column of the left DataFrame. right_on Join column of the right DataFrame. suffix Suffix to append to columns with a duplicate name. validate: {'m:m', 'm:1', '1:m', '1:1'} Checks if join is of specified type. .. list-table :: :header-rows: 0 * - **m:m** - *(Default)* Many-to-many. Does not result in checks. * - **1:1** - One-to-one. Checks if join keys are unique in both left and right datasets. * - **1:m** - One-to-many. Checks if join keys are unique in left dataset. * - **m:1** - Many-to-one. Check if join keys are unique in right dataset. .. note:: This is currently not supported by the streaming engine. nulls_equal Join on null values. By default null values will never produce matches. coalesce Coalescing behavior (merging of join columns). .. list-table :: :header-rows: 0 * - **None** - *(Default)* Coalesce unless `how='full'` is specified. * - **True** - Always coalesce join columns. * - **False** - Never coalesce join columns. .. note:: Joining on any other expressions than `col` will turn off coalescing. maintain_order : {'none', 'left', 'right', 'left_right', 'right_left'} Which DataFrame row order to preserve, if any. Do not rely on any observed ordering without explicitly setting this parameter, as your code may break in a future release. Not specifying any ordering can improve performance. Supported for inner, left, right and full joins .. list-table :: :header-rows: 0 * - **none** - *(Default)* No specific ordering is desired. The ordering might differ across Polars versions or even between different runs. * - **left** - Preserves the order of the left DataFrame. * - **right** - Preserves the order of the right DataFrame. * - **left_right** - First preserves the order of the left DataFrame, then the right. * - **right_left** - First preserves the order of the right DataFrame, then the left. allow_parallel Allow the physical plan to optionally evaluate the computation of both DataFrames up to the join in parallel. force_parallel Force the physical plan to evaluate the computation of both DataFrames up to the join in parallel. See Also -------- join_asof Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> other_lf = pl.LazyFrame( ... { ... "apple": ["x", "y", "z"], ... "ham": ["a", "b", "d"], ... } ... ) >>> lf.join(other_lf, on="ham").collect() shape: (2, 4) ┌─────┬─────┬─────┬───────┐ │ foo ┆ bar ┆ ham ┆ apple │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str │ ╞═════╪═════╪═════╪═══════╡ │ 1 ┆ 6.0 ┆ a ┆ x │ │ 2 ┆ 7.0 ┆ b ┆ y │ └─────┴─────┴─────┴───────┘ >>> lf.join(other_lf, on="ham", how="full").collect() shape: (4, 5) ┌──────┬──────┬──────┬───────┬───────────┐ │ foo ┆ bar ┆ ham ┆ apple ┆ ham_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str ┆ str │ ╞══════╪══════╪══════╪═══════╪═══════════╡ │ 1 ┆ 6.0 ┆ a ┆ x ┆ a │ │ 2 ┆ 7.0 ┆ b ┆ y ┆ b │ │ null ┆ null ┆ null ┆ z ┆ d │ │ 3 ┆ 8.0 ┆ c ┆ null ┆ null │ └──────┴──────┴──────┴───────┴───────────┘ >>> lf.join(other_lf, on="ham", how="left", coalesce=True).collect() shape: (3, 4) ┌─────┬─────┬─────┬───────┐ │ foo ┆ bar ┆ ham ┆ apple │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str │ ╞═════╪═════╪═════╪═══════╡ │ 1 ┆ 6.0 ┆ a ┆ x │ │ 2 ┆ 7.0 ┆ b ┆ y │ │ 3 ┆ 8.0 ┆ c ┆ null │ └─────┴─────┴─────┴───────┘ >>> lf.join(other_lf, on="ham", how="semi").collect() shape: (2, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6.0 ┆ a │ │ 2 ┆ 7.0 ┆ b │ └─────┴─────┴─────┘ >>> lf.join(other_lf, on="ham", how="anti").collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str │ ╞═════╪═════╪═════╡ │ 3 ┆ 8.0 ┆ c │ └─────┴─────┴─────┘ >>> lf.join(other_lf, how="cross").collect() shape: (9, 5) ┌─────┬─────┬─────┬───────┬───────────┐ │ foo ┆ bar ┆ ham ┆ apple ┆ ham_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ str ┆ str ┆ str │ ╞═════╪═════╪═════╪═══════╪═══════════╡ │ 1 ┆ 6.0 ┆ a ┆ x ┆ a │ │ 1 ┆ 6.0 ┆ a ┆ y ┆ b │ │ 1 ┆ 6.0 ┆ a ┆ z ┆ d │ │ 2 ┆ 7.0 ┆ b ┆ x ┆ a │ │ 2 ┆ 7.0 ┆ b ┆ y ┆ b │ │ 2 ┆ 7.0 ┆ b ┆ z ┆ d │ │ 3 ┆ 8.0 ┆ c ┆ x ┆ a │ │ 3 ┆ 8.0 ┆ c ┆ y ┆ b │ │ 3 ┆ 8.0 ┆ c ┆ z ┆ d │ └─────┴─────┴─────┴───────┴───────────┘ Nriz;cannot use 'on' in conjunction with 'left_on' or 'right_on'z+'left_on' requires corresponding 'right_on'outerrf:use of `how='outer'` should be replaced with `how='full'`.0.20.29router_coalesceTzRuse of `how='outer_coalesce'` should be replaced with `how='full', coalesce=True`.crossz$cross join should not pass join keysz-must specify `on` OR `left_on` and `right_on`)r4rr"rrjoinr%)rrrrrrrrrr rr r uses_on uses_left_on uses_right_on uses_lr_onrr pyexprs_left pyexprs_rights rr&zLazyFrame.joins j $&&&  !#ND.d*  , !2]  "z "OCS// ! ] * *?CS// ! '>>C %L!     $ $ $HC %d!     G^^ &* &< oo%## J"""       "4R88G"L#MM  "9'BBL:8DDMMACS// ! INN      r)rExpr | Iterable[Expr]ct||t|}||j|j||S)u Perform a join based on one or multiple (in)equality predicates. This performs an inner join, so only rows where all predicates are true are included in the result, and a row from either DataFrame may be included multiple times in the result. .. note:: The row order of the input DataFrames is not preserved. .. warning:: This functionality is experimental. It may be changed at any point without it being considered a breaking change. Parameters ---------- other DataFrame to join with. *predicates (In)Equality condition to join the two tables on. When a column name occurs in both tables, the proper suffix must be applied in the predicate. suffix Suffix to append to columns with a duplicate name. Examples -------- >>> east = pl.LazyFrame( ... { ... "id": [100, 101, 102], ... "dur": [120, 140, 160], ... "rev": [12, 14, 16], ... "cores": [2, 8, 4], ... } ... ) >>> west = pl.LazyFrame( ... { ... "t_id": [404, 498, 676, 742], ... "time": [90, 130, 150, 170], ... "cost": [9, 13, 15, 16], ... "cores": [4, 2, 1, 4], ... } ... ) >>> east.join_where( ... west, ... pl.col("dur") < pl.col("time"), ... pl.col("rev") < pl.col("cost"), ... ).collect() shape: (5, 8) ┌─────┬─────┬─────┬───────┬──────┬──────┬──────┬─────────────┐ │ id ┆ dur ┆ rev ┆ cores ┆ t_id ┆ time ┆ cost ┆ cores_right │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═══════╪══════╪══════╪══════╪═════════════╡ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 498 ┆ 130 ┆ 13 ┆ 2 │ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 676 ┆ 150 ┆ 15 ┆ 1 │ │ 100 ┆ 120 ┆ 12 ┆ 2 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ │ 101 ┆ 140 ┆ 14 ┆ 8 ┆ 676 ┆ 150 ┆ 15 ┆ 1 │ │ 101 ┆ 140 ┆ 14 ┆ 8 ┆ 742 ┆ 170 ┆ 16 ┆ 4 │ └─────┴─────┴─────┴───────┴──────┴──────┴──────┴─────────────┘ )r4r%rr join_where)rrrrrs rr/zLazyFrame.join_wheresWH $&&&0*= I       rctttjdd}t |i|d|i}||j|S)u Add columns to this LazyFrame. Added columns will replace existing columns with the same name. Parameters ---------- *exprs Column(s) to add, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to add, specified as keyword arguments. The columns will be renamed to the keyword used. Returns ------- LazyFrame A new LazyFrame with the columns added. Notes ----- Creating a new LazyFrame using this method does not create a new copy of existing data. Examples -------- Pass an expression to add it as a new column. >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [0.5, 4, 10, 13], ... "c": [True, True, False, True], ... } ... ) >>> lf.with_columns((pl.col("a") ** 2).alias("a^2")).collect() shape: (4, 4) ┌─────┬──────┬───────┬─────┐ │ a ┆ b ┆ c ┆ a^2 │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ i64 │ ╞═════╪══════╪═══════╪═════╡ │ 1 ┆ 0.5 ┆ true ┆ 1 │ │ 2 ┆ 4.0 ┆ true ┆ 4 │ │ 3 ┆ 10.0 ┆ false ┆ 9 │ │ 4 ┆ 13.0 ┆ true ┆ 16 │ └─────┴──────┴───────┴─────┘ Added columns will replace existing columns with the same name. >>> lf.with_columns(pl.col("a").cast(pl.Float64)).collect() shape: (4, 3) ┌─────┬──────┬───────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╡ │ 1.0 ┆ 0.5 ┆ true │ │ 2.0 ┆ 4.0 ┆ true │ │ 3.0 ┆ 10.0 ┆ false │ │ 4.0 ┆ 13.0 ┆ true │ └─────┴──────┴───────┘ Multiple columns can be added using positional arguments. >>> lf.with_columns( ... (pl.col("a") ** 2).alias("a^2"), ... (pl.col("b") / 2).alias("b/2"), ... (pl.col("c").not_()).alias("not c"), ... ).collect() shape: (4, 6) ┌─────┬──────┬───────┬─────┬──────┬───────┐ │ a ┆ b ┆ c ┆ a^2 ┆ b/2 ┆ not c │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ i64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╪═════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true ┆ 1 ┆ 0.25 ┆ false │ │ 2 ┆ 4.0 ┆ true ┆ 4 ┆ 2.0 ┆ false │ │ 3 ┆ 10.0 ┆ false ┆ 9 ┆ 5.0 ┆ true │ │ 4 ┆ 13.0 ┆ true ┆ 16 ┆ 6.5 ┆ false │ └─────┴──────┴───────┴─────┴──────┴───────┘ Multiple columns can also be added by passing a list of expressions. >>> lf.with_columns( ... [ ... (pl.col("a") ** 2).alias("a^2"), ... (pl.col("b") / 2).alias("b/2"), ... (pl.col("c").not_()).alias("not c"), ... ] ... ).collect() shape: (4, 6) ┌─────┬──────┬───────┬─────┬──────┬───────┐ │ a ┆ b ┆ c ┆ a^2 ┆ b/2 ┆ not c │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ i64 ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╪═════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true ┆ 1 ┆ 0.25 ┆ false │ │ 2 ┆ 4.0 ┆ true ┆ 4 ┆ 2.0 ┆ false │ │ 3 ┆ 10.0 ┆ false ┆ 9 ┆ 5.0 ┆ true │ │ 4 ┆ 13.0 ┆ true ┆ 16 ┆ 6.5 ┆ false │ └─────┴──────┴───────┴─────┴──────┴───────┘ Use keyword arguments to easily name your expression inputs. >>> lf.with_columns( ... ab=pl.col("a") * pl.col("b"), ... not_c=pl.col("c").not_(), ... ).collect() shape: (4, 5) ┌─────┬──────┬───────┬──────┬───────┐ │ a ┆ b ┆ c ┆ ab ┆ not_c │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ bool ┆ f64 ┆ bool │ ╞═════╪══════╪═══════╪══════╪═══════╡ │ 1 ┆ 0.5 ┆ true ┆ 0.5 ┆ false │ │ 2 ┆ 4.0 ┆ true ┆ 8.0 ┆ false │ │ 3 ┆ 10.0 ┆ false ┆ 30.0 ┆ true │ │ 4 ┆ 13.0 ┆ true ┆ 52.0 ┆ false │ └─────┴──────┴───────┴──────┴───────┘ Expressions with multiple outputs can automatically be instantiated as Structs by enabling the experimental setting `Config.set_auto_structify(True)`: >>> with pl.Config(auto_structify=True): ... lf.drop("c").with_columns( ... diffs=pl.col(["a", "b"]).diff().name.suffix("_diff"), ... ).collect() shape: (4, 3) ┌─────┬──────┬─────────────┐ │ a ┆ b ┆ diffs │ │ --- ┆ --- ┆ --- │ │ i64 ┆ f64 ┆ struct[2] │ ╞═════╪══════╪═════════════╡ │ 1 ┆ 0.5 ┆ {null,null} │ │ 2 ┆ 4.0 ┆ {1,3.5} │ │ 3 ┆ 10.0 ┆ {1,6.0} │ │ 4 ┆ 13.0 ┆ {1,3.0} │ └─────┴──────┴─────────────┘ rrr) rrrrr!r%rrrrs rrzLazyFrame.with_columnsLs}dRZ^^,CQGGHHII 0  !  /8    6 6w ? ?@@@rctttjdd}t |i|d|i}||j|S)a6 Add columns to this LazyFrame. Added columns will replace existing columns with the same name. This will run all expression sequentially instead of in parallel. Use this when the work per expression is cheap. Parameters ---------- *exprs Column(s) to add, specified as positional arguments. Accepts expression input. Strings are parsed as column names, other non-expression inputs are parsed as literals. **named_exprs Additional columns to add, specified as keyword arguments. The columns will be renamed to the keyword used. Returns ------- LazyFrame A new LazyFrame with the columns added. See Also -------- with_columns rrr) rrrrr!r%rrwith_columns_seqrs rr2zLazyFrame.with_columns_seqs}@RZ^^,CQGGHHII 0  !  /8    : :7 C CDDDrzW`LazyFrame.with_context` is deprecated; use `pl.concat(..., how='horizontal')` instead.Self | list[Self]ct|ts|g}||jd|DS)u Add an external context to the computation graph. .. deprecated:: 1.0.0 Use :func:`concat` instead, with `how='horizontal'` This allows expressions to also access columns from DataFrames that are not part of this one. Parameters ---------- other Lazy DataFrame to join with. Examples -------- >>> lf = pl.LazyFrame({"a": [1, 2, 3], "b": ["a", "c", None]}) >>> lf_other = pl.LazyFrame({"c": ["foo", "ham"]}) >>> lf.with_context(lf_other).select( # doctest: +SKIP ... pl.col("b") + pl.col("c").first() ... ).collect() shape: (3, 1) ┌──────┐ │ b │ │ --- │ │ str │ ╞══════╡ │ afoo │ │ cfoo │ │ null │ └──────┘ Fill nulls with the median from another DataFrame: >>> train_lf = pl.LazyFrame( ... {"feature_0": [-1.0, 0, 1], "feature_1": [-1.0, 0, 1]} ... ) >>> test_lf = pl.LazyFrame( ... {"feature_0": [-1.0, None, 1], "feature_1": [-1.0, 0, 1]} ... ) >>> test_lf.with_context( # doctest: +SKIP ... train_lf.select(pl.all().name.suffix("_train")) ... ).select( ... pl.col("feature_0").fill_null(pl.col("feature_0_train").median()) ... ).collect() shape: (3, 1) ┌───────────┐ │ feature_0 │ │ --- │ │ f64 │ ╞═══════════╡ │ -1.0 │ │ 0.0 │ │ 1.0 │ └───────────┘ cg|] }|j SrQ)r)rxrs rrz*LazyFrame.with_context..Ls7P7P7PB7P7P7Pr)rrrr with_contextrs rr6zLazyFrame.with_context sQz%&& GE 6 67P7P%7P7P7P Q QRRRrr5ColumnNameOrSelector | Iterable[ColumnNameOrSelector]crt|}||j||S)u Remove columns from the DataFrame. Parameters ---------- *columns Names of the columns that should be removed from the dataframe. Accepts column selector input. strict Validate that all column names exist in the current schema, and throw an exception if any do not. Examples -------- Drop a single column by passing the name of that column. >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6.0, 7.0, 8.0], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.drop("ham").collect() shape: (3, 2) ┌─────┬─────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪═════╡ │ 1 ┆ 6.0 │ │ 2 ┆ 7.0 │ │ 3 ┆ 8.0 │ └─────┴─────┘ Drop multiple columns by passing a selector. >>> import polars.selectors as cs >>> lf.drop(cs.numeric()).collect() shape: (3, 1) ┌─────┐ │ ham │ │ --- │ │ str │ ╞═════╡ │ a │ │ b │ │ c │ └─────┘ Use positional arguments to drop multiple columns. >>> lf.drop("foo", "ham").collect() shape: (3, 1) ┌─────┐ │ bar │ │ --- │ │ f64 │ ╞═════╡ │ 6.0 │ │ 7.0 │ │ 8.0 │ └─────┘ r)r%rrdrop)rrr drop_colss rr9zLazyFrame.dropNs5J3G<  y H HIIIrmapping(Mapping[str, str] | Callable[[str], str]c~t|r>|tjj|St |}t |}| |j |||S)u Rename column names. Parameters ---------- mapping Key value pairs that map from old name to new name, or a function that takes the old name as input and returns the new name. strict Validate that all column names exist in the current schema, and throw an exception if any do not. (Note that this parameter is a no-op when passing a function to `mapping`). Notes ----- If existing names are swapped (e.g. 'A' points to 'B' and 'B' points to 'A'), polars will block projection and predicate pushdowns at this node. Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, 7, 8], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.rename({"foo": "apple"}).collect() shape: (3, 3) ┌───────┬─────┬─────┐ │ apple ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═══════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └───────┴─────┴─────┘ >>> lf.rename(lambda column_name: "c" + column_name[1:]).collect() shape: (3, 3) ┌─────┬─────┬─────┐ │ coo ┆ car ┆ cam │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ │ 2 ┆ 7 ┆ b │ │ 3 ┆ 8 ┆ c │ └─────┴─────┴─────┘ ) rrrrrmaprkeysrrrrename)rr;rexistingnews rr@zLazyFrame.renamesj G   M;;quww|//8899 9GLLNN++Hw~~''((C##DI$4$4XsF$K$KLL LrcZ||jS)u_ Reverse the DataFrame. Examples -------- >>> lf = pl.LazyFrame( ... { ... "key": ["a", "b", "c"], ... "val": [1, 2, 3], ... } ... ) >>> lf.reverse().collect() shape: (3, 2) ┌─────┬─────┐ │ key ┆ val │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════╪═════╡ │ c ┆ 3 │ │ b ┆ 2 │ │ a ┆ 1 │ └─────┴─────┘ )rrrrs rrzLazyFrame.reverses&0 1 1 3 3444rr) fill_valueint | IntoExprColumnrDIntoExpr | Nonec|t|d}t|}||j||S)u Shift values by the given number of indices. Parameters ---------- n Number of indices to shift forward. If a negative value is passed, values are shifted in the opposite direction instead. fill_value Fill the resulting null values with this value. Accepts scalar expression input. Non-expression inputs are parsed as literals. Notes ----- This method is similar to the `LAG` operation in SQL when the value for `n` is positive. With a negative value for `n`, it is similar to `LEAD`. Examples -------- By default, values are shifted forward by one index. >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [5, 6, 7, 8], ... } ... ) >>> lf.shift().collect() shape: (4, 2) ┌──────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞══════╪══════╡ │ null ┆ null │ │ 1 ┆ 5 │ │ 2 ┆ 6 │ │ 3 ┆ 7 │ └──────┴──────┘ Pass a negative value to shift in the opposite direction instead. >>> lf.shift(-2).collect() shape: (4, 2) ┌──────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞══════╪══════╡ │ 3 ┆ 7 │ │ 4 ┆ 8 │ │ null ┆ null │ │ null ┆ null │ └──────┴──────┘ Specify `fill_value` to fill the resulting null values. >>> lf.shift(-2, fill_value=100).collect() shape: (4, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 3 ┆ 7 │ │ 4 ┆ 8 │ │ 100 ┆ 100 │ │ 100 ┆ 100 │ └─────┴─────┘ NT) str_as_lit)r$rrshift)rrrDs rrIzLazyFrame.shiftsNR  !.zdKKKJ !! $ $ : > >???rlengthc|r|dkrd|d}t|||j||S)u Get a slice of this DataFrame. Parameters ---------- offset Start index. Negative indexing is supported. length Length of the slice. If set to `None`, all rows starting at the offset will be selected. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": ["x", "y", "z"], ... "b": [1, 3, 5], ... "c": [2, 4, 6], ... } ... ) >>> lf.slice(1, 2).collect() shape: (2, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ y ┆ 3 ┆ 4 │ │ z ┆ 5 ┆ 6 │ └─────┴─────┴─────┘ rznegative slice lengths (z) are invalid for LazyFrame)rrrr4)rrrJrs rr4zLazyFrame.slice:sU@  "fqjjRVRRRCS// !  ? ?@@@rc,||S)uw Get the first `n` rows. Alias for :func:`LazyFrame.head`. Parameters ---------- n Number of rows to return. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4, 5, 6], ... "b": [7, 8, 9, 10, 11, 12], ... } ... ) >>> lf.limit().collect() shape: (5, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 7 │ │ 2 ┆ 8 │ │ 3 ┆ 9 │ │ 4 ┆ 10 │ │ 5 ┆ 11 │ └─────┴─────┘ >>> lf.limit(2).collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 7 │ │ 2 ┆ 8 │ └─────┴─────┘ )headrs rlimitzLazyFrame.limit_sVyy||rc.|d|S)uJ Get the first `n` rows. Parameters ---------- n Number of rows to return. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4, 5, 6], ... "b": [7, 8, 9, 10, 11, 12], ... } ... ) >>> lf.head().collect() shape: (5, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 7 │ │ 2 ┆ 8 │ │ 3 ┆ 9 │ │ 4 ┆ 10 │ │ 5 ┆ 11 │ └─────┴─────┘ >>> lf.head(2).collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 7 │ │ 2 ┆ 8 │ └─────┴─────┘ rr4rs rrNzLazyFrame.headsRzz!Qrc\||j|S)uI Get the last `n` rows. Parameters ---------- n Number of rows to return. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4, 5, 6], ... "b": [7, 8, 9, 10, 11, 12], ... } ... ) >>> lf.tail().collect() shape: (5, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 2 ┆ 8 │ │ 3 ┆ 9 │ │ 4 ┆ 10 │ │ 5 ┆ 11 │ │ 6 ┆ 12 │ └─────┴─────┘ >>> lf.tail(2).collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 5 ┆ 11 │ │ 6 ┆ 12 │ └─────┴─────┘ )rrtailrs rrSzLazyFrame.tails'R q 1 1222rc,|dS)u& Get the last row of the DataFrame. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 5, 3], ... "b": [2, 4, 6], ... } ... ) >>> lf.last().collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 3 ┆ 6 │ └─────┴─────┘ r)rSrs rlastzLazyFrame.lasts,yy||rc.|ddS)u( Get the first row of the DataFrame. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> lf.first().collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 2 │ └─────┴─────┘ rrrQrs rfirstzLazyFrame.firsts,zz!Qrz\`LazyFrame.approx_n_unique` is deprecated; use `select(pl.all().approx_n_unique())` instead.cr|tjS)u Approximate count of unique values. .. deprecated:: 0.20.11 Use `select(pl.all().approx_n_unique())` instead. This is done using the HyperLogLog++ algorithm for cardinality estimation. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.approx_n_unique().collect() # doctest: +SKIP shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ u32 ┆ u32 │ ╞═════╪═════╡ │ 4 ┆ 2 │ └─────┴─────┘ )rrrapprox_n_uniquers rrYzLazyFrame.approx_n_uniques(>{{15772244555rindexrc ||j||S#t$r#|dkrdnd}d|d|}t |dwxYw)u Add a row index as the first column in the LazyFrame. Parameters ---------- name Name of the index column. offset Start the index at this offset. Cannot be negative. Warnings -------- Using this function can have a negative effect on query performance. This may, for instance, block predicate pushdown optimization. Notes ----- The resulting column does not have any special properties. It is a regular column of type `UInt32` (or `UInt64` in `polars-u64-idx`). Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> lf.with_row_index().collect() shape: (3, 3) ┌───────┬─────┬─────┐ │ index ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞═══════╪═════╪═════╡ │ 0 ┆ 1 ┆ 2 │ │ 1 ┆ 3 ┆ 4 │ │ 2 ┆ 5 ┆ 6 │ └───────┴─────┴─────┘ >>> lf.with_row_index("id", offset=1000).collect() shape: (3, 3) ┌──────┬─────┬─────┐ │ id ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞══════╪═════╪═════╡ │ 1000 ┆ 1 ┆ 2 │ │ 1001 ┆ 3 ┆ 4 │ │ 1002 ┆ 5 ┆ 6 │ └──────┴─────┴─────┘ An index column can also be created using the expressions :func:`int_range` and :func:`len`. >>> lf.select( ... pl.int_range(pl.len(), dtype=pl.UInt32).alias("index"), ... pl.all(), ... ).collect() shape: (3, 3) ┌───────┬─────┬─────┐ │ index ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞═══════╪═════╪═════╡ │ 0 ┆ 1 ┆ 2 │ │ 1 ┆ 3 ┆ 4 │ │ 2 ┆ 5 ┆ 6 │ └───────┴─────┴─────┘ rnegativez$greater than the maximum index valuez.`offset` input for `with_row_index` cannot be z, got N)rrwith_row_index OverflowErrorr)rrrissuers rr]zLazyFrame.with_row_index3sL ,##DI$<$>> lf = pl.LazyFrame( ... { ... "a": [1, 3, 5], ... "b": [2, 4, 6], ... } ... ) >>> lf.with_row_count().collect() # doctest: +SKIP shape: (3, 3) ┌────────┬─────┬─────┐ │ row_nr ┆ a ┆ b │ │ --- ┆ --- ┆ --- │ │ u32 ┆ i64 ┆ i64 │ ╞════════╪═════╪═════╡ │ 0 ┆ 1 ┆ 2 │ │ 1 ┆ 3 ┆ 4 │ │ 2 ┆ 5 ┆ 6 │ └────────┴─────┴─────┘ )r])rrrs rwith_row_countzLazyFrame.with_row_countsX""4000rcx|tjd||S)uW Take every nth row in the LazyFrame and return as a new LazyFrame. Parameters ---------- n Gather every *n*-th row. offset Starting index. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [5, 6, 7, 8], ... } ... ) >>> lf.gather_every(2).collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 5 │ │ 3 ┆ 7 │ └─────┴─────┘ >>> lf.gather_every(2, offset=1).collect() shape: (2, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 2 ┆ 6 │ │ 4 ┆ 8 │ └─────┴─────┘ *)rrr gather_every)rrrs rrezLazyFrame.gather_everys/P{{15::221f==>>>r)matches_supertyperAny | Expr | NoneFillNullStrategy | NonerOrfc Dddlm}|t|tjrd}nt|t r t g}nb|rbt|ttfrFtttttttt t"t$t&|g }nt|tr tg}nt|tr t&g}nt|t(rt*gdt,Dz}nt|t.rt0gdt,Dz}nbt|t2r t4g}nDt|t6r t8g}n&t|t:rt<t>g}nd}|r<| tCj"|#|||S|$tCj%#|||S)uB Fill null values using the specified value or strategy. Parameters ---------- value Value used to fill null values. strategy : {None, 'forward', 'backward', 'min', 'max', 'mean', 'zero', 'one'} Strategy used to fill null values. limit Number of consecutive null values to fill when using the 'forward' or 'backward' strategy. matches_supertype Fill all matching supertypes of the fill `value` literal. See Also -------- fill_nan Notes ----- A null value is not the same as a NaN value. To fill NaN values, use :func:`fill_nan`. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, None, 4], ... "b": [0.5, 4, None, 13], ... } ... ) >>> lf.fill_null(99).collect() shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 99 ┆ 99.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ >>> lf.fill_null(strategy="forward").collect() shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 2 ┆ 4.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ >>> lf.fill_null(strategy="max").collect() shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 4 ┆ 13.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ >>> lf.fill_null(strategy="zero").collect() shape: (4, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪══════╡ │ 1 ┆ 0.5 │ │ 2 ┆ 4.0 │ │ 0 ┆ 0.0 │ │ 4 ┆ 13.0 │ └─────┴──────┘ r)DecimalNc,g|]}t|SrQ)r<rxus rrz'LazyFrame.fill_null..P&Q&Q&Qqx{{&Q&Q&Qrc,g|]}t|SrQ)r=rls rrz'LazyFrame.fill_null..Rrnr)&polarsrjrrrnrr9rrrArBrCrDrErJrKrLrMr?r@rr<r7r r=rr;rrIrrHr:rrr fill_nullrr)rrrrOrfrjrs rrqzLazyFrame.fill_nullsv #"""""  %))# E4((! !" z%#u'F'F  E3'' E5)) !E8,, "&Q&Q>> lf = pl.LazyFrame( ... { ... "a": [1.5, 2, float("nan"), 4], ... "b": [0.5, 4, float("nan"), 13], ... } ... ) >>> lf.fill_nan(99).collect() shape: (4, 2) ┌──────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞══════╪══════╡ │ 1.5 ┆ 0.5 │ │ 2.0 ┆ 4.0 │ │ 99.0 ┆ 99.0 │ │ 4.0 ┆ 13.0 │ └──────┴──────┘ ) rrrnrrrrfill_nanr)rrs rrtzLazyFrame.fill_nandsLN%)) !E%LLE 2 25= A ABBBrddofc\||j|S)u: Aggregate the columns in the LazyFrame to their standard deviation value. Parameters ---------- ddof “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.std().collect() shape: (1, 2) ┌──────────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞══════════╪═════╡ │ 1.290994 ┆ 0.5 │ └──────────┴─────┘ >>> lf.std(ddof=0).collect() shape: (1, 2) ┌──────────┬──────────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞══════════╪══════════╡ │ 1.118034 ┆ 0.433013 │ └──────────┴──────────┘ )rrrrrrus rrrz LazyFrame.std'L d 3 3444rc\||j|S)u Aggregate the columns in the LazyFrame to their variance value. Parameters ---------- ddof “Delta Degrees of Freedom”: the divisor used in the calculation is N - ddof, where N represents the number of elements. By default ddof is 1. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.var().collect() shape: (1, 2) ┌──────────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞══════════╪══════╡ │ 1.666667 ┆ 0.25 │ └──────────┴──────┘ >>> lf.var(ddof=0).collect() shape: (1, 2) ┌──────┬────────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞══════╪════════╡ │ 1.25 ┆ 0.1875 │ └──────┴────────┘ )rrvarrws rrzz LazyFrame.varrxrcZ||jS)uG Aggregate the columns in the LazyFrame to their maximum value. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.max().collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 4 ┆ 2 │ └─────┴─────┘ )rrr{rs rr{z LazyFrame.max", 000rcZ||jS)uG Aggregate the columns in the LazyFrame to their minimum value. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.min().collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 1 │ └─────┴─────┘ )rrrsrs rrsz LazyFrame.minr|rcZ||jS)uC Aggregate the columns in the LazyFrame to their sum value. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.sum().collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 10 ┆ 5 │ └─────┴─────┘ )rrsumrs rrz LazyFrame.sumr|rcZ||jS)uR Aggregate the columns in the LazyFrame to their mean value. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.mean().collect() shape: (1, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞═════╪══════╡ │ 2.5 ┆ 1.25 │ └─────┴──────┘ )rrrqrs rrqzLazyFrame.mean's$,  0 0111rcZ||jS)uI Aggregate the columns in the LazyFrame to their median value. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.median().collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞═════╪═════╡ │ 2.5 ┆ 1.0 │ └─────┴─────┘ )rrmedianrs rrzLazyFrame.median?s&, 0 0 2 2333rcZ||jS)u Aggregate the columns in the LazyFrame as the sum of their null value count. Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, None, 3], ... "bar": [6, 7, None], ... "ham": ["a", "b", "c"], ... } ... ) >>> lf.null_count().collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ u32 ┆ u32 ┆ u32 │ ╞═════╪═════╪═════╡ │ 1 ┆ 1 ┆ 0 │ └─────┴─────┴─────┘ )rrrprs rrpzLazyFrame.null_countWs&. 4 4 6 6777rr float | Exprc|t|}||j||S)u1 Aggregate the columns in the LazyFrame to their quantile value. Parameters ---------- quantile Quantile between 0.0 and 1.0. interpolation : {'nearest', 'higher', 'lower', 'midpoint', 'linear', 'equiprobable'} Interpolation method. Examples -------- >>> lf = pl.LazyFrame( ... { ... "a": [1, 2, 3, 4], ... "b": [1, 2, 1, 1], ... } ... ) >>> lf.quantile(0.7).collect() shape: (1, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ f64 ┆ f64 │ ╞═════╪═════╡ │ 3.0 ┆ 1.0 │ └─────┴─────┘ )r$rrr)rrrjs rrzLazyFrame.quantileps8B)22 2 28] K KLLLr!str | Expr | Sequence[str | Expr] more_columns str | Exprcvt|g|R}||j|S)u, Explode the DataFrame to long format by exploding the given columns. Parameters ---------- columns Column names, expressions, or a selector defining them. The underlying columns being exploded must be of the `List` or `Array` data type. *more_columns Additional names of columns to explode, specified as positional arguments. Examples -------- >>> lf = pl.LazyFrame( ... { ... "letters": ["a", "a", "b", "c"], ... "numbers": [[1], [2, 3], [4, 5], [6, 7, 8]], ... } ... ) >>> lf.explode("numbers").collect() shape: (8, 2) ┌─────────┬─────────┐ │ letters ┆ numbers │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═════════╪═════════╡ │ a ┆ 1 │ │ a ┆ 2 │ │ a ┆ 3 │ │ b ┆ 4 │ │ b ┆ 5 │ │ c ┆ 6 │ │ c ┆ 7 │ │ c ┆ 8 │ └─────────┴─────────┘ )r%rrexploderrrs rrzLazyFrame.explodes>R1H<HHH 1 1' : :;;;rr)keeprsubset>ColumnNameOrSelector | Collection[ColumnNameOrSelector] | Nonerrc|t|}||j|||S)uN Drop duplicate rows from this DataFrame. Parameters ---------- subset Column name(s) or selector(s), to consider when identifying duplicate rows. If set to `None` (default), use all columns. keep : {'first', 'last', 'any', 'none'} Which of the duplicate rows to keep. * 'any': Does not give any guarantee of which row is kept. This allows more optimizations. * 'none': Don't keep duplicate rows. * 'first': Keep first unique row. * 'last': Keep last unique row. maintain_order Keep the same order as the original DataFrame. This is more expensive to compute. Settings this to `True` blocks the possibility to run on the streaming engine. Returns ------- LazyFrame LazyFrame with unique rows. Warnings -------- This method will fail if there is a column of type `List` in the DataFrame or subset. Notes ----- If you're coming from pandas, this is similar to `pandas.DataFrame.drop_duplicates`. Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3, 1], ... "bar": ["a", "a", "a", "a"], ... "ham": ["b", "b", "b", "b"], ... } ... ) >>> lf.unique(maintain_order=True).collect() shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ a ┆ b │ │ 2 ┆ a ┆ b │ │ 3 ┆ a ┆ b │ └─────┴─────┴─────┘ >>> lf.unique(subset=["bar", "ham"], maintain_order=True).collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ a ┆ b │ └─────┴─────┴─────┘ >>> lf.unique(keep="last", maintain_order=True).collect() shape: (3, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ str ┆ str │ ╞═════╪═════╪═════╡ │ 2 ┆ a ┆ b │ │ 3 ┆ a ┆ b │ │ 1 ┆ a ┆ b │ └─────┴─────┴─────┘ )r%rrunique)rrrrs rrzLazyFrame.uniques@j  3F;;F 0 0 N NOOOrc~|t|}||j|S)u Drop all rows that contain one or more NaN values. The original order of the remaining rows is preserved. Parameters ---------- subset Column name(s) for which NaN values are considered; if set to `None` (default), use all columns (note that only floating-point columns can contain NaNs). See Also -------- drop_nulls Notes ----- A NaN value is not the same as a null value. To drop null values, use :func:`drop_nulls`. Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [-20.5, float("nan"), 80.0], ... "bar": [float("nan"), 110.0, 25.5], ... "ham": ["xxx", "yyy", None], ... } ... ) The default behavior of this method is to drop rows where any single value in the row is NaN: >>> lf.drop_nans().collect() shape: (1, 3) ┌──────┬──────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞══════╪══════╪══════╡ │ 80.0 ┆ 25.5 ┆ null │ └──────┴──────┴──────┘ This behaviour can be constrained to consider only a subset of columns, as defined by name, or with a selector. For example, dropping rows only if there is a NaN in the "bar" column: >>> lf.drop_nans(subset=["bar"]).collect() shape: (2, 3) ┌──────┬───────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ str │ ╞══════╪═══════╪══════╡ │ NaN ┆ 110.0 ┆ yyy │ │ 80.0 ┆ 25.5 ┆ null │ └──────┴───────┴──────┘ Dropping a row only if *all* values are NaN requires a different formulation: >>> lf = pl.LazyFrame( ... { ... "a": [float("nan"), float("nan"), float("nan"), float("nan")], ... "b": [10.0, 2.5, float("nan"), 5.25], ... "c": [65.75, float("nan"), float("nan"), 10.5], ... } ... ) >>> lf.filter(~pl.all_horizontal(pl.all().is_nan())).collect() shape: (3, 3) ┌─────┬──────┬───────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 │ ╞═════╪══════╪═══════╡ │ NaN ┆ 10.0 ┆ 65.75 │ │ NaN ┆ 2.5 ┆ NaN │ │ NaN ┆ 5.25 ┆ 10.5 │ └─────┴──────┴───────┘ )r%rr drop_nansrrs rrzLazyFrame.drop_nanss<h  3F;;F 3 3F ; ;<<>> lf = pl.LazyFrame( ... { ... "foo": [1, 2, 3], ... "bar": [6, None, 8], ... "ham": ["a", "b", None], ... } ... ) The default behavior of this method is to drop rows where any single value in the row is null: >>> lf.drop_nulls().collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪═════╡ │ 1 ┆ 6 ┆ a │ └─────┴─────┴─────┘ This behaviour can be constrained to consider only a subset of columns, as defined by name or with a selector. For example, dropping rows if there is a null in any of the integer columns: >>> import polars.selectors as cs >>> lf.drop_nulls(subset=cs.integer()).collect() shape: (2, 3) ┌─────┬─────┬──────┐ │ foo ┆ bar ┆ ham │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ str │ ╞═════╪═════╪══════╡ │ 1 ┆ 6 ┆ a │ │ 3 ┆ 8 ┆ null │ └─────┴─────┴──────┘ Dropping a row only if *all* values are null requires a different formulation: >>> lf = pl.LazyFrame( ... { ... "a": [None, None, None, None], ... "b": [1, 2, None, 1], ... "c": [1, None, None, 1], ... } ... ) >>> lf.filter(~pl.all_horizontal(pl.all().is_null())).collect() shape: (3, 3) ┌──────┬─────┬──────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ null ┆ i64 ┆ i64 │ ╞══════╪═════╪══════╡ │ null ┆ 1 ┆ 1 │ │ null ┆ 2 ┆ null │ │ null ┆ 1 ┆ 1 │ └──────┴─────┴──────┘ )r%rr drop_nullsrs rrzLazyFrame.drop_nullsqs<^  3F;;F 4 4V < <===r)rZ variable_name value_name streamable>> lf = pl.LazyFrame( ... { ... "a": ["x", "y", "z"], ... "b": [1, 3, 5], ... "c": [2, 4, 6], ... } ... ) >>> import polars.selectors as cs >>> lf.unpivot(cs.numeric(), index="a").collect() shape: (6, 3) ┌─────┬──────────┬───────┐ │ a ┆ variable ┆ value │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞═════╪══════════╪═══════╡ │ x ┆ b ┆ 1 │ │ y ┆ b ┆ 3 │ │ z ┆ b ┆ 5 │ │ x ┆ c ┆ 2 │ │ y ┆ c ┆ 4 │ │ z ┆ c ┆ 6 │ └─────┴──────────┴───────┘ z\the `streamable` parameter for `LazyFrame.unpivot` is deprecatedThis parameter has no effectz1.5.0r)r"r%rrunpivot)rrrZrrrs rrzLazyFrame.unpivots@  %/     :RR#A"#E#Em)G)N)N 1 1"eZ W WXXXr)rrrno_optimizationsrvalidate_output_schemar Callable[[DataFrame], DataFrame]rNone | SchemaDictrc z|rd}d}d}||j|||||||S)u/ Apply a custom function. It is important that the function returns a Polars DataFrame. Parameters ---------- function Lambda/ function to apply. predicate_pushdown Allow predicate pushdown optimization to pass this node. projection_pushdown Allow projection pushdown optimization to pass this node. slice_pushdown Allow slice pushdown optimization to pass this node. no_optimizations Turn off all optimizations past this point. schema Output schema of the function, if set to `None` we assume that the schema will remain unchanged by the applied function. validate_output_schema It is paramount that polars' schema is correct. This flag will ensure that the output schema of this function will be checked with the expected schema. Setting this to `False` will not do this check, but may lead to hard to debug bugs. streamable Whether the function that is given is eligible to be running with the streaming engine. That means that the function must produce the same result when it is executed in batches or when it is be executed on the full dataset. Warnings -------- The `schema` of a `LazyFrame` must always be correct. It is up to the caller of this function to ensure that this invariant is upheld. It is important that the optimization flags are correct. If the custom function for instance does an aggregation of a column, `predicate_pushdown` should not be allowed, as this prunes rows and will influence your aggregation results. Examples -------- >>> lf = ( # doctest: +SKIP ... pl.LazyFrame( ... { ... "a": pl.int_range(-100_000, 0, eager=True), ... "b": pl.int_range(0, 100_000, eager=True), ... } ... ) ... .map_batches(lambda x: 2 * x, streamable=True) ... .collect(engine="streaming") ... ) shape: (100_000, 2) ┌─────────┬────────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════════╪════════╡ │ -200000 ┆ 0 │ │ -199998 ┆ 2 │ │ -199996 ┆ 4 │ │ -199994 ┆ 6 │ │ … ┆ … │ │ -8 ┆ 199992 │ │ -6 ┆ 199994 │ │ -4 ┆ 199996 │ │ -2 ┆ 199998 │ └─────────┴────────┘ F)rrvalidate_output)rrr) rr`rrrrrrrs rrzLazyFrame.map_batchesseb  #!& "' "N I ! !"#% 6 "     rct|tjdS)u Interpolate intermediate values. The interpolation method is linear. Nulls at the beginning and end of the series remain null. Examples -------- >>> lf = pl.LazyFrame( ... { ... "foo": [1, None, 9, 10], ... "bar": [6, 7, 9, None], ... "baz": [1, None, None, 9], ... } ... ) >>> lf.interpolate().collect() shape: (4, 3) ┌──────┬──────┬──────────┐ │ foo ┆ bar ┆ baz │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 │ ╞══════╪══════╪══════════╡ │ 1.0 ┆ 6.0 ┆ 1.0 │ │ 5.0 ┆ 7.0 ┆ 3.666667 │ │ 9.0 ┆ 9.0 ┆ 6.333333 │ │ 10.0 ┆ null ┆ 9.0 │ └──────┴──────┴──────────┘ rd)rrr interpolaters rrzLazyFrame.interpolaters*8{{15::1133444r7ColumnNameOrSelector | Collection[ColumnNameOrSelector]rqcvt|g|R}||j|S)u Decompose struct columns into separate columns for each of their fields. The new columns will be inserted into the DataFrame at the location of the struct column. Parameters ---------- columns Name of the struct column(s) that should be unnested. *more_columns Additional columns to unnest, specified as positional arguments. Examples -------- >>> df = pl.LazyFrame( ... { ... "before": ["foo", "bar"], ... "t_a": [1, 2], ... "t_b": ["a", "b"], ... "t_c": [True, None], ... "t_d": [[1, 2], [3]], ... "after": ["baz", "womp"], ... } ... ).select("before", pl.struct(pl.col("^t_.$")).alias("t_struct"), "after") >>> df.collect() shape: (2, 3) ┌────────┬─────────────────────┬───────┐ │ before ┆ t_struct ┆ after │ │ --- ┆ --- ┆ --- │ │ str ┆ struct[4] ┆ str │ ╞════════╪═════════════════════╪═══════╡ │ foo ┆ {1,"a",true,[1, 2]} ┆ baz │ │ bar ┆ {2,"b",null,[3]} ┆ womp │ └────────┴─────────────────────┴───────┘ >>> df.unnest("t_struct").collect() shape: (2, 6) ┌────────┬─────┬─────┬──────┬───────────┬───────┐ │ before ┆ t_a ┆ t_b ┆ t_c ┆ t_d ┆ after │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ str ┆ bool ┆ list[i64] ┆ str │ ╞════════╪═════╪═════╪══════╪═══════════╪═══════╡ │ foo ┆ 1 ┆ a ┆ true ┆ [1, 2] ┆ baz │ │ bar ┆ 2 ┆ b ┆ null ┆ [3] ┆ womp │ └────────┴─────┴─────┴──────┴───────────┴───────┘ )r%rrunnestrs rrzLazyFrame.unnests>f1H<HHH 0 0 9 9:::rct||||j|j|S)uR Take two sorted DataFrames and merge them by the sorted key. The output of this operation will also be sorted. It is the callers responsibility that the frames are sorted in ascending order by that key otherwise the output will not make sense. The schemas of both LazyFrames must be equal. Parameters ---------- other Other DataFrame that must be merged key Key that is sorted. Examples -------- >>> df0 = pl.LazyFrame( ... {"name": ["steve", "elise", "bob"], "age": [42, 44, 18]} ... ).sort("age") >>> df0.collect() shape: (3, 2) ┌───────┬─────┐ │ name ┆ age │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═══════╪═════╡ │ bob ┆ 18 │ │ steve ┆ 42 │ │ elise ┆ 44 │ └───────┴─────┘ >>> df1 = pl.LazyFrame( ... {"name": ["anna", "megan", "steve", "thomas"], "age": [21, 33, 42, 20]} ... ).sort("age") >>> df1.collect() shape: (4, 2) ┌────────┬─────┐ │ name ┆ age │ │ --- ┆ --- │ │ str ┆ i64 │ ╞════════╪═════╡ │ thomas ┆ 20 │ │ anna ┆ 21 │ │ megan ┆ 33 │ │ steve ┆ 42 │ └────────┴─────┘ >>> df0.merge_sorted(df1, key="age").collect() shape: (7, 2) ┌────────┬─────┐ │ name ┆ age │ │ --- ┆ --- │ │ str ┆ i64 │ ╞════════╪═════╡ │ bob ┆ 18 │ │ thomas ┆ 20 │ │ anna ┆ 21 │ │ megan ┆ 33 │ │ steve ┆ 42 │ │ steve ┆ 42 │ │ elise ┆ 44 │ └────────┴─────┘ Notes ----- No guarantee is given over the output row order when the key is equal between the both dataframes. The key must be sorted in ascending order. )r4rr merge_sorted)rrr&s rrzLazyFrame.merge_sorteds=P $&&& 6 6uz3 G GHHHrrcolumnct|tsd}t||t j||S)a Flag a column as sorted. This can speed up future operations. Parameters ---------- column Column that is sorted descending Whether the column is sorted in descending order. Warnings -------- This can lead to incorrect results if the data is NOT sorted!! Use with care! z6expected a 'str' for argument 'column' in 'set_sorted'r)rrrrrr set_sorted)rrrrs rrzLazyFrame.set_sortedsT4&#&& !JCC..   v!9!9Z!9!P!PQQQr)rr include_nullsr Literal['left', 'inner', 'full']rc 4t|||dvrd}tdd|dvrd|}t|d} |_|6|4d } d } || }|| }| gx}}n+|d }t||d }t|n|x}}t |t r|g}t |t r|g}|} |D]} | | vrd| d}t||} |D]} | | vrd| d}t||dkr9t| t|kr| r|| S|St|  | t|z }rCd| tj d d}nddgfd|D}||jg||R|||d | fd|D|}| r|| }||jS)u Update the values in this `LazyFrame` with the values in `other`. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Parameters ---------- other LazyFrame that will be used to update the values on Column names that will be joined on. If set to `None` (default), the implicit row index of each frame is used as a join key. how : {'left', 'inner', 'full'} * 'left' will keep all rows from the left table; rows may be duplicated if multiple rows in the right frame match the left row's key. * 'inner' keeps only those rows where the key exists in both frames. * 'full' will update existing rows where the key matches while also adding any new rows contained in the given frame. left_on Join column(s) of the left DataFrame. right_on Join column(s) of the right DataFrame. include_nulls Overwrite values in the left frame with null values from the right frame. If set to `False` (default), null values in the right frame are ignored. maintain_order : {'none', 'left', 'right', 'left_right', 'right_left'} Which order of rows from the inputs to preserve. See :func:`~LazyFrame.join` for details. Unlike `join` this function preserves the left order by default. Notes ----- This is syntactic sugar for a left/inner join that preserves the order of the left `DataFrame` by default, with an optional coalesce when `include_nulls = False`. Examples -------- >>> lf = pl.LazyFrame( ... { ... "A": [1, 2, 3, 4], ... "B": [400, 500, 600, 700], ... } ... ) >>> lf.collect() shape: (4, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 400 │ │ 2 ┆ 500 │ │ 3 ┆ 600 │ │ 4 ┆ 700 │ └─────┴─────┘ >>> new_lf = pl.LazyFrame( ... { ... "B": [-66, None, -99], ... "C": [5, 3, 1], ... } ... ) Update `df` values with the non-null values in `new_df`, by row index: >>> lf.update(new_lf).collect() shape: (4, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ -66 │ │ 2 ┆ 500 │ │ 3 ┆ -99 │ │ 4 ┆ 700 │ └─────┴─────┘ Update `df` values with the non-null values in `new_df`, by row index, but only keeping those rows that are common to both frames: >>> lf.update(new_lf, how="inner").collect() shape: (3, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ -66 │ │ 2 ┆ 500 │ │ 3 ┆ -99 │ └─────┴─────┘ Update `df` values with the non-null values in `new_df`, using a full outer join strategy that defines explicit join columns in each frame: >>> lf.update(new_lf, left_on=["A"], right_on=["C"], how="full").collect() shape: (5, 2) ┌─────┬─────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ -99 │ │ 2 ┆ 500 │ │ 3 ┆ 600 │ │ 4 ┆ 700 │ │ 5 ┆ -66 │ └─────┴─────┘ Update `df` values including null values in `new_df`, using a full outer join strategy that defines explicit join columns in each frame: >>> lf.update( ... new_lf, left_on="A", right_on="C", how="full", include_nulls=True ... ).collect() shape: (5, 2) ┌─────┬──────┐ │ A ┆ B │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪══════╡ │ 1 ┆ -99 │ │ 2 ┆ 500 │ │ 3 ┆ null │ │ 4 ┆ 700 │ │ 5 ┆ -66 │ └─────┴──────┘ )r!r$rfr"r#r)rrrfz6`how` must be one of {'left', 'inner', 'full'}; found FNT__POLARS_ROW_INDEXz#missing join columns for left framez$missing join columns for right framezleft join column z not foundzright join column )__POLARS_VALIDITYrrQ__POLARS_RIGHTc3$K|] }|V dSrrQ)rxrtmp_names rrzz#LazyFrame.update..s.FF$T-8--FFFFFFr)rrrrr rc3K|]}rtjtjtj|tj|n+tj|tj|g|VdSr)rwhenris_nullthen otherwiser r)rxrrrvaliditys rrzz#LazyFrame.update..s  %HAF15??224455T!%++&&Yqu%8h%8%899:::%8h%8%8!%++$FGG%++      r)r4r"rr]rrrr r9r intersectionrrrrr&rrr)rrrrrrrrrrow_index_usedrow_index_name left_schemar right_schema right_other drop_columnsrGrrs ` @@rrzLazyFrame.update0sq^ $&&& - - -C %L!     / / /TSTTCS// ! :8#3!%!5**>::,,^<<&4%55((??C$S//)#@C$S//)$ "$ #Gh gs # # iG h $ $ " zH))++  & &D;&&<$<<< oo%'++--  & &D<''=4=== oo%( &==S..#h--?? 1yy000K,''44[AACMMQ  -H&&quT{{'8'8!'E'EFFEEH#RFFFF+FFFRR II @h@@x@@@!-   \      (     T,  + .  1[[00F ,,,rcZ||jS)u Return the number of non-null elements for each column. Examples -------- >>> lf = pl.LazyFrame( ... {"a": [1, 2, 3, 4], "b": [1, 2, 1, None], "c": [None, None, None, None]} ... ) >>> lf.count().collect() shape: (1, 3) ┌─────┬─────┬─────┐ │ a ┆ b ┆ c │ │ --- ┆ --- ┆ --- │ │ u32 ┆ u32 ┆ u32 │ ╞═════╪═════╪═════╡ │ 4 ┆ 3 ┆ 0 │ └─────┴─────┴─────┘ )rrrors rrozLazyFrame.count!s$&  1 1222rz`LazyFrame.melt` is deprecated; use `LazyFrame.unpivot` instead, with `index` instead of `id_vars` and `on` instead of `value_vars`)rid_vars value_varsc6||||||S)a Unpivot a DataFrame from wide to long format. Optionally leaves identifiers set. This function is useful to massage a DataFrame into a format where one or more columns are identifier variables (id_vars) while all other columns, considered measured variables (value_vars), are "unpivoted" to the row axis leaving just two non-identifier columns, 'variable' and 'value'. .. deprecated:: 1.0.0 Use the :meth:`.unpivot` method instead. Parameters ---------- id_vars Column(s) or selector(s) to use as identifier variables. value_vars Column(s) or selector(s) to use as values variables; if `value_vars` is empty all columns that are not in `id_vars` will be used. variable_name Name to give to the `variable` column. Defaults to "variable" value_name Name to give to the `value` column. Defaults to "value" streamable Allow this node to run in the streaming engine. If this runs in streaming, the output of the unpivot operation will not have a stable ordering. )rZrrrr)r)rrrrrrs rmeltzLazyFrame.melt6s/T||'!!    rrCcontextpc.ComputeContext | None plan_typepc._typing.PlanTypePreferencepc.LazyFrameExtc0tj|||S)u Run a query remotely on Polars Cloud. This allows you to run Polars remotely on one or more workers via several strategies for distributed compute. Read more in the `Announcement post `_ Parameters ---------- context Compute context in which queries are executed. If none given, it will take the default context. plan_type Whether to give a dot diagram of a plain text version of logical plan. Examples -------- Run a query on a cloud instance. >>> lf = pl.LazyFrame([1, 2, 3]).sum() >>> in_progress = lf.remote().collect() # doctest: +SKIP >>> # do some other work >>> in_progress.await_result() # doctest: +SKIP shape: (1, 1) ┌──────────┐ │ column_0 │ │ --- │ │ i64 │ ╞══════════╡ │ 6 │ └──────────┘ Run a query distributed. >>> lf = ( ... pl.scan_parquet("s3://my_bucket/").group_by("key").agg(pl.sum("values")) ... ) >>> in_progress = lf.remote().distributed().collect() # doctest: +SKIP >>> in_progress.await_result() # doctest: +SKIP shape: (1, 1) ┌──────────┐ │ column_0 │ │ --- │ │ i64 │ ╞══════════╡ │ 6 │ └──────────┘ )rrr)pc LazyFrameExt)rrrs rremotezLazyFrame.remotehst$9MMMMrraiseforbid)missing_columnsmissing_struct_fields extra_columnsextra_struct_fields integer_cast float_castSchemaDict | SchemarLLiteral['insert', 'raise'] | Mapping[str, Literal['insert', 'raise'] | Expr]rELiteral['insert', 'raise'] | Mapping[str, Literal['insert', 'raise']]rLiteral['ignore', 'raise']rELiteral['ignore', 'raise'] | Mapping[str, Literal['ignore', 'raise']]rGLiteral['upcast', 'forbid'] | Mapping[str, Literal['upcast', 'forbid']]rc  ddlm d fd t|trt |}n|}t|tr! fd|D} nt| r  |} n|} t |j || ||||| S) u{ Match or evolve the schema of a LazyFrame into a specific schema. By default, match_to_schema returns an error if the input schema does not exactly match the target schema. It also allows columns to be freely reordered, with additional coercion rules available through optional parameters. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. Parameters ---------- schema Target schema to match or evolve to. missing_columns Raise of insert missing columns from the input with respect to the `schema`. This can also be an expression per column with what to insert if it is missing. missing_struct_fields Raise of insert missing struct fields from the input with respect to the `schema`. extra_columns Raise of ignore extra columns from the input with respect to the `schema`. extra_struct_fields Raise of ignore extra struct fields from the input with respect to the `schema`. integer_cast Forbid of upcast for integer columns from the input to the respective column in `schema`. float_cast Forbid of upcast for float columns from the input to the respective column in `schema`. Examples -------- Ensuring the schema matches >>> lf = pl.LazyFrame({"a": [1, 2, 3], "b": ["A", "B", "C"]}) >>> lf.match_to_schema({"a": pl.Int64, "b": pl.String}).collect() shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═════╡ │ 1 ┆ A │ │ 2 ┆ B │ │ 3 ┆ C │ └─────┴─────┘ >>> (lf.match_to_schema({"a": pl.Int64}).collect()) # doctest: +SKIP polars.exceptions.SchemaError: extra columns in `match_to_schema`: "b" Adding missing columns >>> ( ... pl.LazyFrame({"a": [1, 2, 3]}) ... .match_to_schema( ... {"a": pl.Int64, "b": pl.String}, ... missing_columns="insert", ... ) ... .collect() ... ) shape: (3, 2) ┌─────┬──────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪══════╡ │ 1 ┆ null │ │ 2 ┆ null │ │ 3 ┆ null │ └─────┴──────┘ >>> ( ... pl.LazyFrame({"a": [1, 2, 3]}) ... .match_to_schema( ... {"a": pl.Int64, "b": pl.String}, ... missing_columns={"b": pl.col.a.cast(pl.String)}, ... ) ... .collect() ... ) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ str │ ╞═════╪═════╡ │ 1 ┆ 1 │ │ 2 ┆ 2 │ │ 3 ┆ 3 │ └─────┴─────┘ Removing extra columns >>> ( ... pl.LazyFrame({"a": [1, 2, 3], "b": ["A", "B", "C"]}) ... .match_to_schema( ... {"a": pl.Int64}, ... extra_columns="ignore", ... ) ... .collect() ... ) shape: (3, 1) ┌─────┐ │ a │ │ --- │ │ i64 │ ╞═════╡ │ 1 │ │ 2 │ │ 3 │ └─────┘ Upcasting integers and floats >>> ( ... pl.LazyFrame( ... {"a": [1, 2, 3], "b": [1.0, 2.0, 3.0]}, ... schema={"a": pl.Int32, "b": pl.Float32}, ... ) ... .match_to_schema( ... {"a": pl.Int64, "b": pl.Float64}, ... integer_cast="upcast", ... float_cast="upcast", ... ) ... .collect() ... ) shape: (3, 2) ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ f64 │ ╞═════╪═════╡ │ 1 ┆ 1.0 │ │ 2 ┆ 2.0 │ │ 3 ┆ 3.0 │ └─────┴─────┘ r)rnr!Literal['insert', 'raise'] | Exprr#Literal['insert', 'raise'] | PyExprc6t|r|jS|Sr)rr)rrns rprepare_missing_columnsz:LazyFrame.match_to_schema..prepare_missing_columnsB s$%&& %}$Lrc.i|]\}}||SrQrQ)rxr&rrs r z-LazyFrame.match_to_schema..S s;&&&C,,U33&&&r)rrrrrrr)rrrr) rprnrrr^rrrrmatch_to_schema) rrrrrrrr schema_prepmissing_columns_pyexprrnrs @@rrzLazyFrame.match_to_schemasx        fg & & ! ..KK K ow / / 5&&&&"1"7"7"9"9&&& " " . . 5%<%<_%M%M " "%4 "$$ I % %" 6&;+$7)% &     rNone | str | list[str]statsc|}|-t|tr|g}||}||S)zz Get all runtime metadata for each column. This is unstable and is meant for debugging purposes. N)r)rrrr _to_metadata)rrrrs rrzLazyFrame._to_metadatah sX  '3'' $")7##Bzz||((u(555r)NN)rrrrrrrrrrrrrrrr)rrarr)rr)rrrr) rrrrrVrrrrr)rrrrrr)rr)rr)rr^)rr)rr)rrrr)rrrr)rrrr)r&rrr)rrr)r.rrr)r1r2rr)rr).)rNrrrOrr)rNrrrSrr)rNrUrrrr)rNrWrrrrX)r`rarbrcrdrerr)rh)rkrlrjrrrl) rrtr9rrrrrrrrrrrrrrrrrrrrrrrsrrrrirr)(r9rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrsrrrrrrirr)r)rrrr)rrrrwrrrrrrrrrr)rrrrrr)rrrrrrrr)"rrrrrrrrrrrrrrrrrrrrrrrrrrrrsrrirrrr)rrrrrrrrrrrrrrrrrrrrrrsrr4rrirr[)rrrrrrrrrrrrrrrrrrrrrrsrr7rrirrl)rrrrrrrrrrrrrrrrrrrrrrsrrrrirrrr9)r<r4rrsrrirr=)r<r7rrsrrirrA)r<rrrsrrirrC)$rrXrLrrMrrNrYrOrrPrrrrQrZrRr[rSrrTr\rUrrr7rVr]rrsrWr^rrirr)$rrXrLrrMrrNrYrOrrPrrrrQrZrRr[rSrrTr\rUrrr4rVr]rrsrWr^rrirr)$rrXrLrrMrrNrYrOrrPrrrrQrZrRr[rSrrTr\rWr^rUrrrrVr]rrsrrirrc)rrXrLryrxrzrrrQrZrRr[rSrrTr\rUrrr7rrsrrirr)rrXrLryrxrzrrrQrZrRr[rSrrTr\rUrrr4rrsrrirr)rrXrLryrxrzrrrQrZrRr[rSrrTr\rUrrrrrsrrirrc)0rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrQrZrRr[rSrrTr\rUrrr7rrsrrirr)0rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrQrZrRr[rSrrTr\rUrrr4rrsrrirr)0rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrQrZrRr[rSrrTr\rUrrrrrsrrirrc)rrrrrQrZrRr[rSrrTr\rUrrr7rrsrrirr)rrrrrQrZrRr[rSrrTr\rUrrr4rrsrrirr)rrrrrQrZrRr[rSrrTr\rUrrrrrsrrirrc)r)rrrrrrrrrrrrrrrrrrrrrrrrrrl)rrrrrrrrrrrrrrrrrrrrrrrrrrrrl)rrrrrr)r)rrrr)rrrrrrrr)rrrrrr)rrrrwrr)rrrrrrwrrZ) rrwrrrrrrprrrrZ)rrwrrrrrrrrrrprr|rrrrrrZ) rrrrrrrrrrrrrrrrorrr rr rr rr rr rrrrr)Nr)rrrrrrzrrrrrrrr{rrr rrrr rr rrr)rrrr-rrrr)rr3rr)rr7rrrr)r;r<rrrr)r)rrErDrFrr)rrrJrrr)rL)rZr)rrrrrr)r`r)rrrrrr)NNN) rrgrrhrOrrfrrr)rrrrr)rurrr)ri)rrrjrrr)rrrrrr)rrrrrrrr)rrrr) rrrZrrrrrrrrr)r`rrrrrrrrrrrrrrrrr)rrrrqrr)rrr&rrr)rrrrrr)Nr)rrrrrrrrrrrrrrrr)NNNN) rrrrrrrrrrrr)NrC)rrrrrr)rrrrrrrrrrrrrrrr)rrrrrrl)zr? __module__ __qualname____doc____annotations__rrr8r classmethodrrrrrpropertyrrrr r rrrrr!r#r%r(r,r0r6r;rArMrrrgrr r]r\r:rrrrrrr r"rr@rrar}rrr!rrrrrrr+rrrrrrrrrr&r)r/rr2r6r9r@rrIr4rOrNrSrUrWrYr]rbrerqrtrrrzr{rsrrqrrprrrrrrrrrrrrrorrrrrQrrrrsVVp%(SUUJ****'+*. /3%)*9!      6[     ::::  % [4KS=5=5=5=5=5[=5~'-'-'-X'-R'.'.'.X'.R % % %X %D%+%+%+X%+N%%%%%%%%$$$$$$$$%%%%%%%%,,,,1111@@@@&?BXQQQQXQ JMX ,0>A'/ >A>A>A>A>A>A@E/E/E/E/R7IL)2 LLLLLL\#"$$!("#'$($(#"&"&%)###''>#Z-Z-Z-Z-Z-%$Z-x#"$$)- '3" #'$($(#"&"&%)## $!'>+[ [ [ [ [ %$[ z     <-2,1$"v v v v v v p4:R&R&R&R&R&R&h! y'JJJ */ KLKLKLKLKLKJKLZ! y'JJJ */ KOKOKOKOKOKJKOZ##'$($( %#"&"&%)##*#'>#VVVVVVp##'$($(#"&"&%)# %#'>X$##'$($(#"&"&%)# %#%*'>X$#"$$##'$($(#"&"&%)# %# '>e6e6e6e6e6%$e6N $'> 00000X0"'#'> #####X##"$$#'> ddddd%$dLFFFF@ "(,37%)%)#1526"#+/'>1X6 "(,37%)%)#1526 #+/'>1X>"(,37%)%)#1526+/#'>1K*K*K*K*K*K*Z .4+/#1526"#'>!X& .4+/#1526#'>!X..<+/#1526#'>!b*b*b*b*b*b*H "##&*"&"&(,&*!%,0#1526"#'>7X< "##&*"&"&(,&*!%,0#1526#'>7XD"##&*"&"&(,&*!%,0#1526#'>7`*`*`*`*`*`*D $1526"#'>X" $1526#'>X* $1526#'>M*M*M*M*M*M*^Z 9 1 # #'$($( %#"&"&%)#1 1 1 1 1 1 jp)# #'$($( %#"&"&%)#!p)p)p)p)p)p)d23333"fBfBfBfBfBfBP,J,J,J,J,J\3333JSKSKSKSKSKSKjp p p p dX X X X tk;k;k;k;Z????B %j j j j j j X! z9EEE *.!(9=H H H H H FEH T! z9EEE *.)-#(!'9=$O O O O O FEO j &*&* $.2/3)-%/:>#$$(!%#q q q q q q f ! }fMMM8<# ] =A=A#(! $37#$] ] ] ] ] NM] ~XZZ  M M M M M ZM ^WAWAWAWAr%E%E%E%ENZ :<S<S<S <SBFJFJFJFJFJFJRTX:M:M:M:M:M:Mx55556)*L@LPL@L@L@L@L@L@\#A#A#A#A#AJ+++++Z) ) ) ) ) V)3)3)3)3)3V0    0Z <666 6:K,K,K,K,K,ZZ S(1(1(1(1 (1T(?(?(?(?(?X$(,0 JF #' JFJFJFJFJFJFX)C)C)C)CV&5&5&5&5&5P&5&5&5&5&5P111101111011110222204444088888)2"M"M"M"M"MH*<*<*<*<\RVWP$)$ WPWPWPWPWPWPvRVV=V=V=V=V=tRVQ>Q>Q>Q>Q>jLPJYOS$(!%JYJYJYJYJYJY`$($(#!&$('+ ` ` ` ` ` ` D5555<4;4;4;4;lIIIIIIII^! RRRRRR>XZZ*.06 n- /3/3#39n-n-n-n-n-Zn-`3333*Z H QUSW$(!% ,  , , , , , , \XZZ-1389N9N9N9NZ9NvXZZ =D5<4;5<6>6>A A A A A ZA J+/(,6666666rr)rrsrrs)rrrr) rrsrrrrrrrrrr) __future__r contextlibrrr\collections.abcrrrrrr functoolsr r r r rrrpathlibrtypingrrrrrrrpolars._reexport _reexportrrprrpolars._typingrrpolars._utils.async_rrpolars._utils.convertrrpolars._utils.deprecationrr r!r"polars._utils.parquetr#polars._utils.parser$r%polars._utils.serder&polars._utils.slicer'polars._utils.unstabler(r)polars._utils.variousr*r+r,r-r.r/r0r1r2r3r4polars._utils.wrapr5r6polars.datatypesr7r8r9r:r;r<r=r>r?r@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPpolars.datatypes.grouprQpolars.dependenciesrRrSrTrUrrVrpolars.exceptionsrWpolars.interchange.protocolrXpolars.lazyframe.engine_configrYpolars.lazyframe.group_byrZpolars.lazyframe.in_processr[polars.lazyframe.opt_flagsr\r] polars.schemar^polars.selectorsr_r`suppress ImportError polars.polarsrarbsysrcrdrerfrgrhrirjrkrlrmrnrorprqrrrsrtrurvrwrxryrzr{r|r}r~rrrrrrrrrrrnppolars.io.cloudrpolars.io.parquetr version_inforrtyping_extensionsrrrrrrrrQrrrsY """""" ////////4444444444440000000000 !!!!!!MLLLLLLLRRRRRRRR A@@@@@877777//////CCCCCCCC                          211111118100000 322222------000000333333444444111111666666UUUUUUUU 66666666Z%%??>>>>>>>>???????????????=JJJ==========""""""""888888  [ ) )99888888889999999999999991000000000:0/////::::::888888 7""111111111<<<<<<<< 7""****** 7""'''''''000000 A #AAAAA"(A(A(A(AVO~6O~6O~6O~6O~6O~6O~6O~6O~6O~6s$' E<<FF< GGG