0Ph* dZddlZddlmZmZddlmZddlmZddl m Z m Z ddl Z ddlmZdd lmZmZmZdd lmZmZmZdd lmZdd lmZdd lmZddlmZm Z m!Z!ddl"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*m+Z+m,Z,ddl-m.Z.ddl/m0Z0m1Z1m2Z2m3Z3m4Z4ddl5m6Z6ddl7m8Z8m9Z9ddl:m;Z;mZ>m?Z?m@Z@mAZAmBZBgdZCdZDGddee6ZEdZFdZGdZHdddd d!d!d"d#ZIGd$d%ZJGd&d'eZKd(ZLd)eMd*eMd+eMd,eMfd-ZNdS).z The :mod:`sklearn.compose._column_transformer` module implements utilities to work with heterogeneous data and to apply different transformers to different columns. N)CounterUserList)partial)chain)IntegralReal)sparse)TransformerMixin _fit_contextclone)_fit_transform_one_name_estimators_transform_one)FunctionTransformer)Bunch) _VisualBlock)_determine_key_type_get_column_indices_safe_indexing)METHODS) HasMethodsHiddenInterval StrOptions)_get_container_adapter_get_output_config_safe_set_output)get_tags)MetadataRouter MethodMapping_raise_for_params_routing_enabledprocess_routing)_BaseComposition)Paralleldelayed)_check_feature_names_check_feature_names_in_check_n_features_get_feature_names _is_pandas_df _num_samples check_arraycheck_is_fitted)ColumnTransformermake_column_transformermake_column_selectorz1D data passed to a transformer that expects 2D data. Try to specify the column selection as a list of one item instead of a scalar.c eZdZUdZeeegeddheddgeddgge e ddd ge d ge d gd gd e egd gdZe ed<ddd d dddddZedZejdZd dfd Zd4dZdZdZdZdZdZdZd Zed!Zd"Zd5d#Z d$Z!d%Z"d&Z#d'Z$d(Z%d)Z&d5d*Z'e(d+d5d,Z)d-Z*d.Z+d/Z,d0Z-d1Z.d2Z/fd3Z0xZ1S)6r0aT+Applies transformers to columns of an array or pandas DataFrame. This estimator allows different columns or column subsets of the input to be transformed separately and the features generated by each transformer will be concatenated to form a single feature space. This is useful for heterogeneous or columnar data, to combine several feature extraction mechanisms or transformations into a single transformer. Read more in the :ref:`User Guide `. .. versionadded:: 0.20 Parameters ---------- transformers : list of tuples List of (name, transformer, columns) tuples specifying the transformer objects to be applied to subsets of the data. name : str Like in Pipeline and FeatureUnion, this allows the transformer and its parameters to be set using ``set_params`` and searched in grid search. transformer : {'drop', 'passthrough'} or estimator Estimator must support :term:`fit` and :term:`transform`. Special-cased strings 'drop' and 'passthrough' are accepted as well, to indicate to drop the columns or to pass them through untransformed, respectively. columns : str, array-like of str, int, array-like of int, array-like of bool, slice or callable Indexes the data on its second axis. Integers are interpreted as positional columns, while strings can reference DataFrame columns by name. A scalar string or int should be used where ``transformer`` expects X to be a 1d array-like (vector), otherwise a 2d array will be passed to the transformer. A callable is passed the input data `X` and can return any of the above. To select multiple columns by name or dtype, you can use :obj:`make_column_selector`. remainder : {'drop', 'passthrough'} or estimator, default='drop' By default, only the specified columns in `transformers` are transformed and combined in the output, and the non-specified columns are dropped. (default of ``'drop'``). By specifying ``remainder='passthrough'``, all remaining columns that were not specified in `transformers`, but present in the data passed to `fit` will be automatically passed through. This subset of columns is concatenated with the output of the transformers. For dataframes, extra columns not seen during `fit` will be excluded from the output of `transform`. By setting ``remainder`` to be an estimator, the remaining non-specified columns will use the ``remainder`` estimator. The estimator must support :term:`fit` and :term:`transform`. Note that using this feature requires that the DataFrame columns input at :term:`fit` and :term:`transform` have identical order. sparse_threshold : float, default=0.3 If the output of the different transformers contains sparse matrices, these will be stacked as a sparse matrix if the overall density is lower than this value. Use ``sparse_threshold=0`` to always return dense. When the transformed output consists of all dense data, the stacked result will be dense, and this keyword will be ignored. n_jobs : int, default=None Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. transformer_weights : dict, default=None Multiplicative weights for features per transformer. The output of the transformer is multiplied by these weights. Keys are transformer names, values the weights. verbose : bool, default=False If True, the time elapsed while fitting each transformer will be printed as it is completed. verbose_feature_names_out : bool, str or Callable[[str, str], str], default=True - If True, :meth:`ColumnTransformer.get_feature_names_out` will prefix all feature names with the name of the transformer that generated that feature. It is equivalent to setting `verbose_feature_names_out="{transformer_name}__{feature_name}"`. - If False, :meth:`ColumnTransformer.get_feature_names_out` will not prefix any feature names and will error if feature names are not unique. - If ``Callable[[str, str], str]``, :meth:`ColumnTransformer.get_feature_names_out` will rename all the features using the name of the transformer. The first argument of the callable is the transformer name and the second argument is the feature name. The returned string will be the new feature name. - If ``str``, it must be a string ready for formatting. The given string will be formatted using two field names: ``transformer_name`` and ``feature_name``. e.g. ``"{feature_name}__{transformer_name}"``. See :meth:`str.format` method from the standard library for more info. .. versionadded:: 1.0 .. versionchanged:: 1.6 `verbose_feature_names_out` can be a callable or a string to be formatted. force_int_remainder_cols : bool, default=True Force the columns of the last entry of `transformers_`, which corresponds to the "remainder" transformer, to always be stored as indices (int) rather than column names (str). See description of the `transformers_` attribute for details. .. note:: If you do not access the list of columns for the remainder columns in the `transformers_` fitted attribute, you do not need to set this parameter. .. versionadded:: 1.5 .. versionchanged:: 1.7 The default value for `force_int_remainder_cols` will change from `True` to `False` in version 1.7. Attributes ---------- transformers_ : list The collection of fitted transformers as tuples of (name, fitted_transformer, column). `fitted_transformer` can be an estimator, or `'drop'`; `'passthrough'` is replaced with an equivalent :class:`~sklearn.preprocessing.FunctionTransformer`. In case there were no columns selected, this will be the unfitted transformer. If there are remaining columns, the final element is a tuple of the form: ('remainder', transformer, remaining_columns) corresponding to the ``remainder`` parameter. If there are remaining columns, then ``len(transformers_)==len(transformers)+1``, otherwise ``len(transformers_)==len(transformers)``. .. versionchanged:: 1.5 If there are remaining columns and `force_int_remainder_cols` is True, the remaining columns are always represented by their positional indices in the input `X` (as in older versions). If `force_int_remainder_cols` is False, the format attempts to match that of the other transformers: if all columns were provided as column names (`str`), the remaining columns are stored as column names; if all columns were provided as mask arrays (`bool`), so are the remaining columns; in all other cases the remaining columns are stored as indices (`int`). named_transformers_ : :class:`~sklearn.utils.Bunch` Read-only attribute to access any transformer by given name. Keys are transformer names and values are the fitted transformer objects. sparse_output_ : bool Boolean flag indicating whether the output of ``transform`` is a sparse matrix or a dense numpy array, which depends on the output of the individual transformers and the `sparse_threshold` keyword. output_indices_ : dict A dictionary from each transformer name to a slice, where the slice corresponds to indices in the transformed output. This is useful to inspect which transformer is responsible for which transformed feature(s). .. versionadded:: 1.0 n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying transformers expose such an attribute when fit. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- make_column_transformer : Convenience function for combining the outputs of multiple transformer objects applied to column subsets of the original feature space. make_column_selector : Convenience function for selecting columns based on datatype or the columns name with a regex pattern. Notes ----- The order of the columns in the transformed feature matrix follows the order of how the columns are specified in the `transformers` list. Columns of the original feature matrix that are not specified are dropped from the resulting transformed feature matrix, unless specified in the `passthrough` keyword. Those columns specified with `passthrough` are added at the right to the output of the transformers. Examples -------- >>> import numpy as np >>> from sklearn.compose import ColumnTransformer >>> from sklearn.preprocessing import Normalizer >>> ct = ColumnTransformer( ... [("norm1", Normalizer(norm='l1'), [0, 1]), ... ("norm2", Normalizer(norm='l1'), slice(2, 4))]) >>> X = np.array([[0., 1., 2., 2.], ... [1., 1., 0., 1.]]) >>> # Normalizer scales each row of X to unit norm. A separate scaling >>> # is applied for the two first and two last elements of each >>> # row independently. >>> ct.fit_transform(X) array([[0. , 1. , 0.5, 0.5], [0.5, 0.5, 0. , 1. ]]) :class:`ColumnTransformer` can be configured with a transformer that requires a 1d array by setting the column to a string: >>> from sklearn.feature_extraction.text import CountVectorizer >>> from sklearn.preprocessing import MinMaxScaler >>> import pandas as pd # doctest: +SKIP >>> X = pd.DataFrame({ ... "documents": ["First item", "second one here", "Is this the last?"], ... "width": [3, 4, 5], ... }) # doctest: +SKIP >>> # "documents" is a string which configures ColumnTransformer to >>> # pass the documents column as a 1d array to the CountVectorizer >>> ct = ColumnTransformer( ... [("text_preprocess", CountVectorizer(), "documents"), ... ("num_preprocess", MinMaxScaler(), ["width"])]) >>> X_trans = ct.fit_transform(X) # doctest: +SKIP For a more detailed example of usage, see :ref:`sphx_glr_auto_examples_compose_plot_column_transformer_mixed_types.py`. drop passthroughfit transform fit_transformrboth)closedNverboseboolean transformers remaindersparse_thresholdn_jobstransformer_weightsr<verbose_feature_names_outforce_int_remainder_cols_parameter_constraints333333?FT)r@rArBrCr<rDrEcv||_||_||_||_||_||_||_||_dSNr>) selfr?r@rArBrCr<rDrEs c/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/sklearn/compose/_column_transformer.py__init__zColumnTransformer.__init__2sG)" 0 #6  )B&(@%%%cb d|jDS#ttf$r |jcYSwxYw)aS Internal list of transformer only containing the name and transformers, dropping the columns. DO NOT USE: This is for the implementation of get_params via BaseComposition._get_params which expects lists of tuples of len 2. To iterate through the transformers, use ``self._iter`` instead. cg|] \}}}||f SrP.0nametrans_s rK z3ColumnTransformer._transformers..Ss"JJJndE1T5MJJJrM)r? TypeError ValueErrorrJs rK _transformerszColumnTransformer._transformersGsM %JJ8IJJJ J:& % % %$ $ $ $ %s ..c dt||jD|_dS#ttf$r ||_YdSwxYw)zDO NOT USE: This is for the implementation of set_params via BaseComposition._get_params which gives lists of tuples of len 2. c,g|]\\}}\}}}|||fSrPrP)rRrSrTrUcols rKrVz3ColumnTransformer._transformers..]s=!!!0]dEKQ3uc"!!!rMN)zipr?rWrX)rJvalues rKrZzColumnTransformer._transformersWsn  &!!47t?P4Q4Q!!!D   :& & & & %D     &s$(AAr7c t|dt|jt |dgD}|D]}t |||jdvrt |j||S)aSet the output container when `"transform"` and `"fit_transform"` are called. Calling `set_output` will set the output of all estimators in `transformers` and `transformers_`. Parameters ---------- transform : {"default", "pandas", "polars"}, default=None Configure output of `transform` and `fit_transform`. - `"default"`: Default output format of a transformer - `"pandas"`: DataFrame output - `"polars"`: Polars output - `None`: Transform configuration is unchanged .. versionadded:: 1.4 `"polars"` option was added. Returns ------- self : estimator instance Estimator instance. r`c3*K|]\}}}|dv |VdSr4r5NrP)rRrUrTs rK z/ColumnTransformer.set_output..~sD  5!333 4333   rM transformers_rd)super set_outputrr?getattrrr@)rJr7r?rT __class__s rKrhzColumnTransformer.set_outputds0 Y///  $!74"#E#E     " 9 9E Ui 8 8 8 8 8 >!8 8 8 T^y A A A A rMc0|d|S)aGet parameters for this estimator. Returns the parameters given in the constructor as well as the estimators contained within the `transformers` of the `ColumnTransformer`. Parameters ---------- deep : bool, default=True If True, will return the parameters for this estimator and contained subobjects that are estimators. Returns ------- params : dict Parameter names mapped to their values. rZ)deep) _get_params)rJrls rK get_paramszColumnTransformer.get_paramss$d;;;rMc |jdi||S)aSet the parameters of this estimator. Valid parameter keys can be listed with ``get_params()``. Note that you can directly set the parameters of the estimators contained in `transformers` of `ColumnTransformer`. Parameters ---------- **kwargs : dict Estimator parameters. Returns ------- self : ColumnTransformer This estimator. rZ)rZ) _set_params)rJkwargss rK set_paramszColumnTransformer.set_paramss"" 33F333 rMc#K|r|j}nGdt|j|jD}|jdrt ||jg}t d|}|jpij}|D]l\}}} |r|dkr|rt| r!|r8tj | } |j |} |j | } | r| d} ||| ||fVmdS)a% Generate (name, trans, columns, weight) tuples. Parameters ---------- fitted : bool If True, use the fitted transformers (``self.transformers_``) to iterate through transformers, else use the transformers passed by the user (``self.transformers``). column_as_labels : bool If True, columns are returned as string labels. If False, columns are returned as they were given by the user. This can only be True if the ``ColumnTransformer`` is already fitted. skip_drop : bool If True, 'drop' transformers are filtered out. skip_empty_columns : bool If True, transformers with empty selected columns are filtered out. Yields ------ A generator of tuples containing: - name : the name of the transformer - transformer : the transformer object - columns : the columns for that transformer - weight : the weight of the transformer c&g|]\\}}}}|||fSrPrP)rRrSrTrUcolumns rKrVz+ColumnTransformer._iter..s:,$T5!fuf%rMr Fr4rN)rfr^r?_columns _remainderr"_with_dtype_warning_enabled_set_torCget_is_empty_column_selectionnpisscalar_transformer_to_input_indicesfeature_names_in_) rJfittedcolumn_as_labels skip_dropskip_empty_columnsr? get_weightrSrTcolumnscolumns_is_scalarindicess rK_iterzColumnTransformer._itersB>  F-LL03D4Et}0U0UL q! F$\DO3DEE :%NN .4"9 $0 ; ; D% Uf__! &@&I&I  )$&K$8$8!?QQQI    rMcg}i}|jD]H\}}}t|r ||}||t||||<I||_||_dS)a7 Converts callable column specifications. This stores a dictionary of the form `{step_name: column_indices}` and calls the `columns` on `X` if `columns` is a callable for a given transformer. The results are then stored in `self._transformer_to_input_indices`. N)r?callableappendrrvr})rJX all_columnstransformer_to_input_indicesrSrUrs rK_validate_column_callablesz,ColumnTransformer._validate_column_callabless ')$ $ 1 Q Q D!W   %!'!**   w ' ' '1DQ1P1P ( . .# -I***rMctt|j}t tt |j|z }||jd<||}d|j|f|_ dS)zm Validates ``remainder`` and defines ``_remainder`` targeting the remaining columns. r@N) setrr}valuessortedrangen_features_in__get_remainder_colsr@rw)rJrcols remainingremainder_colss rK_validate_remainderz%ColumnTransformer._validate_remainder-s~ 5$<CCEEFGG3uT%899::TABB :C*;711)<<&GrMc d|jD}t|dkrtt|Sn#t$rYdSwxYwdS)Nc2h|]^}}t|SrP)r)rRrUcs rK z>ColumnTransformer._get_remainder_cols_dtype..:s%RRRWq!-a00RRRrMr9int)r?lennextiterrX)rJ all_dtypess rK_get_remainder_cols_dtypez+ColumnTransformer._get_remainder_cols_dtype8st RR@QRRRJ:!##D,,---$   55 us?A AAc|}|jr|dkrt|S|dkrt|jS|dkr fdt |jDSS)Nr) future_dtypestrboolcg|]}|vSrPrP)rRirs rKrVz9ColumnTransformer._get_remainder_cols..JsEEEQALEEErM)rrE_RemainderColsListlistr~rr)rJrdtypes ` rKrz%ColumnTransformer._get_remainder_colsCs..00  ( CUe^^%gEBBB B E>>.w788 8 F??EEEE%0C*D*DEEE ErMc8tdid|jDS)zAccess the fitted transformer by name. Read-only attribute to access any transformer by given name. Keys are transformer names and values are the fitted transformer objects. ci|] \}}}|| SrPrPrQs rK z9ColumnTransformer.named_transformers_..Vs MMMeQeMMMrMrP)rrfrYs rKnamed_transformers_z%ColumnTransformer.named_transformers_Ms*NNMM$:LMMMNNNrMc|j|}||}t|ds(td|dt|jd||S)zGets feature names of transformer. Used in conjunction with self._iter(fitted=True) in get_feature_names_out. get_feature_names_out Transformer z (type z)) does not provide get_feature_names_out.)r}rAttributeErrorr__name__r)rJrSrTfeature_names_incolumn_indicesrs rK%_get_feature_name_out_for_transformerz7ColumnTransformer._get_feature_name_out_for_transformerXs ;DA 0u566  5t55DKK,@555 **5111rMcHt|t||}g}|ddddD]6^}}}||||}||||f7|st jgtS||S)aGet output feature names for transformation. Parameters ---------- input_features : array-like of str or None, default=None Input features. - If `input_features` is `None`, then `feature_names_in_` is used as feature names in. If `feature_names_in_` is not defined, then the following input feature names are generated: `["x0", "x1", ..., "x(n_features_in_ - 1)"]`. - If `input_features` is an array-like, then `input_features` must match `feature_names_in_` if `feature_names_in_` is defined. Returns ------- feature_names_out : ndarray of str objects Transformed feature names. TF)rrrrNr) r/r)rrrr{arrayobject!_add_prefix_for_feature_names_out)rJinput_features"transformer_with_feature_names_outrSrTrUfeature_names_outs rKrz'ColumnTransformer.get_feature_names_outgs( 0~FF.0*#zz"# *  Q QOD%! !% J Je^!! !( . 5 5t=N6O P P P P1 .8Bf--- -55 .   rMc*dt|jr|jnUt|jtrt t |jn|jdurt t dHt tjfd|D}tj |tSttjd|D}d| d D}||rXt|d kr#t|dd dd d z}nt|}t!d |dtjd|DS)aAdd prefix for feature names out that includes the transformer names. Parameters ---------- transformer_with_feature_names_out : list of tuples of (str, array-like of str) The tuple consistent of the transformer's name and its feature names out. Returns ------- feature_names_out : ndarray of shape (n_features,), dtype=str Transformed feature names. N) str_formatTz"{transformer_name}__{feature_name}c3>K|]\}fd|DVdS)c30K|]}|VdSrIrP)rRrfeature_names_out_callablerSs rKrezPColumnTransformer._add_prefix_for_feature_names_out...s1TTQ//a88TTTTTTrMNrP)rRrrSrs @rKrezFColumnTransformer._add_prefix_for_feature_names_out..sV$$//UTTTTBSTTT$$$$$$rMrc3 K|] \}}|V dSrIrP)rRrUss rKrezFColumnTransformer._add_prefix_for_feature_names_out..s&QQdaQQQQQQrMc$g|] \}}|dk |S)r9rPrRrScounts rKrVzGColumnTransformer._add_prefix_for_feature_names_out..s)    T5%RS))D)))rMz, ...]zOutput feature names: z[ are not unique. Please set verbose_feature_names_out=True to add prefixes to feature namescg|]\}}|SrPrP)rRrUrSs rKrVzGColumnTransformer._add_prefix_for_feature_names_out..s D D DgaT D D DrM)rrD isinstancerr"_feature_names_out_with_str_formatrr from_iterabler{asarrayrr most_commonsortrrX concatenate)rJrrfeature_names_count top_6_overlap names_reprrs @rKrz3ColumnTransformer._add_prefix_for_feature_names_outs&*" D2 3 3 )-)G & & 6 < < )029*** & & +t 3 3)02?*** & & 1#$$$$3U$$$E :e6222 2&  QQ.PQQQ Q Q    $7$C$CA$F$F      =!!Q&&!rr!233CRC88C  // RRRR  ~ D D!C D D D   rMc:t|}g}|ddddD]I\}}}}|dkrd}n!t|r|}nt|}||||fJt |rJt d||_dS)aSet self.transformers_ from given transformers. Parameters ---------- transformers : list of estimators The fitted estimators as the output of `self._call_func_on_transformers(func=_fit_transform_one, ...)`. That function doesn't include 'drop' or transformers for which no column is selected. 'drop' is kept as is, and for the no-column transformers the unfitted transformer is put in `self.transformers_`. Frrrrr4TN)rrrzrrrrxrf) rJr?fitted_transformersrfrSoldrurUrTs rK_update_fitted_transformersz-ColumnTransformer._update_fitted_transformerss#<00 $(JJ"$ %/% % 8 8 D#vq f}}+F33 2011  $v!6 7 7 7 7+,,,,,?mTTrMcd|ddddD}t||D]L\}}t|dddks2t|ds"t d |Mt d |d d krd S dd l}n#t$rYd SwxYwt||D]\}}t|s|j D]]\}}t|dd |j ur|j ||jvr3|jj}t d|d|d|d|d|d d S)z Ensure that the output of each transformer is 2D. Otherwise hstack can raise an error or produce incorrect results. cg|] \}}}}| SrPrP)rRrSrUs rKrVz6ColumnTransformer._validate_output..s0   aA    rMTFrndimrr __dataframe__z^The output of the '{0}' transformer should be 2D (numpy array, scipy sparse array, dataframe).r7densepandasNna_valuezThe output of the 'z' transformer for column 'z ' has dtype z and uses pandas.NA to represent null values. Storing this output in a numpy array can cause errors in downstream scikit-learn estimators, and inefficiencies. To avoid this problem you can (i) store the output in a pandas DataFrame by using zF.set_output(transform='pandas') or (ii) modify the input data or the 'z`' transformer to avoid the presence of pandas.NA (for example by using pandas.DataFrame.astype).)rr^rirrXformatrr ImportErrorr,dtypesto_dictitemsNArrjr) rJresultrXsrSpdcol_namer class_names rK_validate_outputz"ColumnTransformer._validate_outputs   !%!&#' ",""   FE**  HB2vq))Q..wr?7S7S. 66#>  %5*d3325@@58 333!^4   1$ 1 1! 1 1/4 1 1 # 1 104 1 1 1       sB## B10B1cVd}i|_t|ddddD]=\}\}}}}||jd}t |||z|j|<||z }>d|jDdgz}|D]#}||jvrt dd|j|<$dS) zA Record which transformer produced which column. rTFrr9cg|] }|d SrrPrRrs rKrVz.<s555aQqT555rMr@N)output_indices_ enumeratershapeslicer?)rJridxtransformer_idxrSrU n_columns all_namess rK_record_output_indicesz(ColumnTransformer._record_output_indices&s!09 JJ!&#'   1 1   ,O_dAq!?+1!4I).sC)O)D)DD  & 9 CC 654#4555 E  9 9D4///-21a[[$T* 9 9rMc$|jsdSd|||fzS)Nz(%d of %d) Processing %s)r<)rJrSrtotals rK _log_messagezColumnTransformer._log_messageAs#| 4)S%,>>>rMc |turd}nd}t|||dd} g}t|dD]\} \} } } } |turv| dkr$IE%LLLE(G!<<<%  %  -T2     084;///55 5   8CFFBB !233:  sDE F'E<<Fc Ht||d|j|fd|i||S)aFit all transformers using X. Parameters ---------- X : {array-like, dataframe} of shape (n_samples, n_features) Input data, of which specified subsets are used to fit the transformers. y : array-like of shape (n_samples,...), default=None Targets for supervised learning. **params : dict, default=None Parameters to be passed to the underlying transformers' ``fit`` and ``transform`` methods. You can only pass this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.4 Returns ------- self : ColumnTransformer This estimator. r6r)r"r8)rJrrrs rKr6zColumnTransformer.fits=4 &$... 1,,,V,,, rM)prefer_skip_nested_validationc t||dt||dt|}t||d|t |}||||trt|dfi|}n| }| ||td|}|s+| gtj|dfSt!|\}}t#d|DrHt%d|D} t%d |D} | | z } | |jk|_nd|_| ||||||t1|| S) acFit all transformers, transform the data and concatenate results. Parameters ---------- X : {array-like, dataframe} of shape (n_samples, n_features) Input data, of which specified subsets are used to fit the transformers. y : array-like of shape (n_samples,), default=None Targets for supervised learning. **params : dict, default=None Parameters to be passed to the underlying transformers' ``fit`` and ``transform`` methods. You can only pass this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.4 Returns ------- X_t : {array-like, sparse matrix} of shape (n_samples, sum_n_components) Horizontally stacked results of transformers. sum_n_components is the sum of n_components (output dimension) over transformers. If any result is a sparse matrix, everything will be converted to sparse matrices. r8TresetFrrrc3>K|]}tj|VdSrI)r issparserRrs rKrez2ColumnTransformer.fit_transform..s,..avq!!......rMc3ZK|]&}tj|r|jn|jV'dSrI)r r#nnzsizer$s rKrez2ColumnTransformer.fit_transform..s9JJ!vq11=aeeqvJJJJJJrMc3K|]:}tj|r|jd|jdzn|jV;dS)rr9N)r r#rr'r$s rKrez2ColumnTransformer.fit_transform..sZNO6?1+=+=I QWQZ''16rM n_samples)r"r(_check_Xr*rr-rrr#r$_get_empty_routingrrrr{zerosr^anysumrAsparse_output_rr_hstackr) rJrrrr*rrrr?r&rdensitys rKr8zColumnTransformer.fit_transformsD &$888T1D1111 QKK$.... ##%%% OO  ''***   ###    6+D/LLVLLMM 3355M00 "' 1   ,  , ,R 0 0 08YN++ +<L ..2... . . (JJrJJJJJCSUEEkG")D,A"AD  "'D  ((666 b!!! ##B'''||DHH |:::rMc p t|dtt|}tdot |pt|d}t |}t |}|rj fdj D}tt|}tfd|D}|t|z } | rtd| nt|dtrtdfi|} n} |d t$|| } | | st)j|d fSt/| | S) aTransform X separately by each transformer, concatenate results. Parameters ---------- X : {array-like, dataframe} of shape (n_samples, n_features) The data to be transformed by subset. **params : dict, default=None Parameters to be passed to the underlying transformers' ``transform`` method. You can only pass this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.4 Returns ------- X_t : {array-like, sparse matrix} of shape (n_samples, sum_n_components) Horizontally stacked results of transformers. sum_n_components is the sum of n_components (output dimension) over transformers. If any result is a sparse matrix, everything will be converted to sparse matrices. r7r~rc:g|]\}}|v |dk|S)r4rP)rRrSindnamed_transformerss rKrVz/ColumnTransformer.transform..7sC###D#---2DT2Jf2T2T2T2T2TrMc32K|]}j|VdSrI)r~)rRr5rJs rKrez.ColumnTransformer.transform..>s+OOCD237OOOOOOrMzcolumns are missing: FrNr!rr))r"r/r+rr,r-r+rr}rrrrXr*r#r$r,rrrr{r-r1r) rJrr%fit_dataframe_and_transform_dataframer* column_namesnon_dropped_indices all_indicesrdiffrrr6s ` @rKr7zColumnTransformer.transform s4 &$ 444 QKK18>Q0R0R1 !   ;? ; ; .!OO )!,, 0 4!%!9 ####!%!C!I!I!K!K###  e%89::KOOOO;OOOOOIs<000D A !?!?!?@@@ A dAU 3 3 3 3    6+D+HHHHMM 3355M  , ,  B' -   b!!! ,8YN++ +||DHH |:::rMcv|jrV d|D}n"#t$r}td|d}~wwxYwtj|Sd|D}t d|r)t fd|Dr d|dd dd D}d |D}|jr2| tt||}nttj |}t|}td |Drt#d |D}d|d} t||D]f\} } | jddkrt#t)| j|} t/| r | d| d| dz } gt| dzd} |D]R} | jddkr|| | | jdz}| || | jdz } S|}|jd}||krtd|St3j|S)aStacks Xs horizontally. This allows subclasses to control the stacking behavior, while reusing everything else from ColumnTransformer. Parameters ---------- Xs : list of {array-like, sparse matrix, dataframe} The container to concatenate. n_samples : int The number of samples in the input data to checking the transformation consistency. c2g|]}t|ddS)TF)r ensure_all_finite)r.r$s rKrVz-ColumnTransformer._hstack..os7    OOO   rMzQFor a sparse output, all columns should be a numeric or convertible to a numeric.Ncbg|],}tj|r|n|-SrP)r r#toarray)rRfs rKrVz-ColumnTransformer._hstack..{s3GGGq!3!3:!))+++GGGrMr7c3BK|]}|VdSrI)is_supported_container)rRradapters rKrez,ColumnTransformer._hstack..}s1MMQw==a@@MMMMMMrMcg|] }|d SrrPrs rKrVz-ColumnTransformer._hstack..s,%%%aD%%%rMTFrc>g|]}|jddk|jS)r9r)rrr$s rKrVz-ColumnTransformer._hstack..s%%O%O%OAqwqzQairMc3"K|] }|dkV dSr9NrP)rRrs rKrez,ColumnTransformer._hstack..s&OO519OOOOOOrMc3,K|]\}}|dk |VdSrIrPrs rKrez,ColumnTransformer._hstack..s8:: +e$qyy!(yyy::rMzUDuplicated feature names found before concatenating the outputs of the transformers: z. r9rrz has conflicting columns names: zEither make sure that the transformers named above do not generate columns with conflicting names or set verbose_feature_names_out=True to automatically prefix to the output feature names with the name of the transformer to prevent any conflicting names.aConcatenating DataFrames from the transformer's output lead to an inconsistent number of samples. The output may have Pandas Indexes that do not match, or that transformers are returning number of samples which are not the same as the number input samples.)r0rXr hstacktocsrrallrrDrrr^rrrr.rrrrrr intersectionrrename_columnsr{)rJrr* converted_Xsrtransformer_namesfeature_names_outsrduplicated_feature_nameserr_msgtransformer_namerdup_cols_in_transformer names_idx names_outoutputoutput_samplesrEs @rKr1zColumnTransformer._hstack\s  _ !           @  =..4466 6GGBGGGB,[$??GJ 3MMMM"MMMMMJ %%!ZZ#)."&+/ (%%%!&P%O%O%O%O"1(*.)O)OS!24FGGHH**&& *.e.ABT.U.U)V)V&*12D*E*E'OO2E2L2L2N2NOOOOO39::/B/H/H/J/J:::440 > 8>>> 477H"3M3M " "/,a wqzQ (6< #AI ; ;.s,NNSJsC00NNNNNNrMr@parallel)r name_details) rr@rr?rrwrMr~tolistrr^r)rJr?remainder_columnsrr`s rK_sk_visual_block_z#ColumnTransformer._sk_visual_block_s" dnc * * Yt~/G/G,LL T< ( ( Y $ 2 122 W% WNNSU5V4WXXL,/,>)|\  E     rMc |j|S#t$r}td|d}~wt$r}td|d|d}~wwxYw)Nz5ColumnTransformer is subscriptable after it is fitted'z!' is not a valid transformer name)rrrWKeyError)rJkeyrs rK __getitem__zColumnTransformer.__getitem__s N+C0 0   G  N N NEsEEEFFA M Ns  A) AA  Ac \tdid|ddddDS)zReturn empty routing. Used while routing can be disabled. TODO: Remove when ``set_config(enable_metadata_routing=False)`` is no more an option. c Pi|]#\}}}}|tdidtD$S)ci|]}|iSrPrP)rRmethods rKrzCColumnTransformer._get_empty_routing...s@@@fvr@@@rMrP)rr)rRrSsteprUs rKrz8ColumnTransformer._get_empty_routing..sO$D$1eAA@@@@@AArMFTrrP)rrrYs rKr,z$ColumnTransformer._get_empty_routings[   (,  %*"'+ )3))   rMct|jj}t|jd|jdfg}|D]\}}}t }t|dr-|ddddnV|dddddddd|dd|jd d|i||i|S) ajGet metadata routing of this object. Please check :ref:`User Guide ` on how the routing mechanism works. .. versionadded:: 1.4 Returns ------- routing : MetadataRouter A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating routing information. )ownerr@Nr8r6)callercalleer7method_mappingrP) r rjrrr?r@r!radd)rJrouterr?rSrmrUrrs rKget_metadata_routingz&ColumnTransformer.get_metadata_routings6 dn&=>>> T.+t~t1T0UVV ) F FMD$*__Nt_-- "&&eO&LLPP.Q #&&eE&BBSkS::SS>>S SDDD   k+  F F F FJ E En Et E E E E rMct} td|jD|j_n#t $rYnwxYw|S)Nc3XK|]%\}}}|dv t|jjV&dSrc)r input_tagsr rQs rKrez5ColumnTransformer.__sklearn_tags__..*sL))"D% 777*17777))rM)rg__sklearn_tags__rMr?rxr Exception)rJtagsrjs rKryz"ColumnTransformer.__sklearn_tags__'s~ww'')) %())&*&7)))&&DO " "     D    s(A AA)TrI)2r __module__ __qualname____doc__rrtuplerrrrrrrrrF__annotations__rLpropertyrZsetterrhrnrrrrrrrrrrrrrrrrrr6r r8r7r1rcrhr,rury __classcell__rjs@rKr0r0?slaaHvve}}- J . / / J{+ , , J5 6 6 &XdAq@@@AT" $d|;&/h%?%.K $ $D   & "&!%AAAAA* % %X % & & &'+'''''''R<<<<((E;E;E;N8JJJ* H H H   OOXO 2 2 2, , , , \< < < |!U!U!UF...`9996??? NNN`@\&+M;M;M; M;^Q;Q;Q;fm!m!m!^   ,NNN   (&&&P         rMr0ct|drt|ds$t|dstj|r|St|dtS)zMUse check_array only when necessary, e.g. on lists and other non-array-likes. __array__rrz allow-nan)r?r)rr r#r.r)rs rKr+r+7sj K %,Q%8%8 1o & & ?1    qKv F F FFrMc0t|dr9tj|jtjr| St|dr.Ms,;;S:c4((;;;;;;rMF)rr{ issubdtyperbool_r.rrM)rus rKrzrzBs vw BM&,$I$I ::<<  # # KK1  ;;F;;;;; KK urMct|\}}tt|\}}tt|||}|S)z; Construct (name, trans, column) tuples from list )r^rr) estimatorsr?rrrUtransformer_lists rK_get_transformer_listrTsJ  ,L'$\223HE1C|W==>> rMr4rGFT)r@rArBr<rDrEc Lt|}t|||||||S)a`Construct a ColumnTransformer from the given transformers. This is a shorthand for the ColumnTransformer constructor; it does not require, and does not permit, naming the transformers. Instead, they will be given names automatically based on their types. It also does not allow weighting with ``transformer_weights``. Read more in the :ref:`User Guide `. Parameters ---------- *transformers : tuples Tuples of the form (transformer, columns) specifying the transformer objects to be applied to subsets of the data. transformer : {'drop', 'passthrough'} or estimator Estimator must support :term:`fit` and :term:`transform`. Special-cased strings 'drop' and 'passthrough' are accepted as well, to indicate to drop the columns or to pass them through untransformed, respectively. columns : str, array-like of str, int, array-like of int, slice, array-like of bool or callable Indexes the data on its second axis. Integers are interpreted as positional columns, while strings can reference DataFrame columns by name. A scalar string or int should be used where ``transformer`` expects X to be a 1d array-like (vector), otherwise a 2d array will be passed to the transformer. A callable is passed the input data `X` and can return any of the above. To select multiple columns by name or dtype, you can use :obj:`make_column_selector`. remainder : {'drop', 'passthrough'} or estimator, default='drop' By default, only the specified columns in `transformers` are transformed and combined in the output, and the non-specified columns are dropped. (default of ``'drop'``). By specifying ``remainder='passthrough'``, all remaining columns that were not specified in `transformers` will be automatically passed through. This subset of columns is concatenated with the output of the transformers. By setting ``remainder`` to be an estimator, the remaining non-specified columns will use the ``remainder`` estimator. The estimator must support :term:`fit` and :term:`transform`. sparse_threshold : float, default=0.3 If the transformed output consists of a mix of sparse and dense data, it will be stacked as a sparse matrix if the density is lower than this value. Use ``sparse_threshold=0`` to always return dense. When the transformed output consists of all sparse or all dense data, the stacked result will be sparse or dense, respectively, and this keyword will be ignored. n_jobs : int, default=None Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. verbose : bool, default=False If True, the time elapsed while fitting each transformer will be printed as it is completed. verbose_feature_names_out : bool, default=True If True, :meth:`ColumnTransformer.get_feature_names_out` will prefix all feature names with the name of the transformer that generated that feature. If False, :meth:`ColumnTransformer.get_feature_names_out` will not prefix any feature names and will error if feature names are not unique. .. versionadded:: 1.0 force_int_remainder_cols : bool, default=True Force the columns of the last entry of `transformers_`, which corresponds to the "remainder" transformer, to always be stored as indices (int) rather than column names (str). See description of the :attr:`ColumnTransformer.transformers_` attribute for details. .. note:: If you do not access the list of columns for the remainder columns in the :attr:`ColumnTransformer.transformers_` fitted attribute, you do not need to set this parameter. .. versionadded:: 1.5 .. versionchanged:: 1.7 The default value for `force_int_remainder_cols` will change from `True` to `False` in version 1.7. Returns ------- ct : ColumnTransformer Returns a :class:`ColumnTransformer` object. See Also -------- ColumnTransformer : Class that allows combining the outputs of multiple transformer objects used on column subsets of the data into a single feature space. Examples -------- >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder >>> from sklearn.compose import make_column_transformer >>> make_column_transformer( ... (StandardScaler(), ['numerical_column']), ... (OneHotEncoder(), ['categorical_column'])) ColumnTransformer(transformers=[('standardscaler', StandardScaler(...), ['numerical_column']), ('onehotencoder', OneHotEncoder(...), ['categorical_column'])]) )rBr@rAr<rDrE)rr0)r@rArBr<rDrEr?rs rKr1r1bs?t-\:: )";!9   rMc(eZdZdZdddddZdZdS)r2a Create a callable to select columns to be used with :class:`ColumnTransformer`. :func:`make_column_selector` can select columns based on datatype or the columns name with a regex. When using multiple selection criteria, **all** criteria must match for a column to be selected. For an example of how to use :func:`make_column_selector` within a :class:`ColumnTransformer` to select columns based on data type (i.e. `dtype`), refer to :ref:`sphx_glr_auto_examples_compose_plot_column_transformer_mixed_types.py`. Parameters ---------- pattern : str, default=None Name of columns containing this regex pattern will be included. If None, column selection will not be selected based on pattern. dtype_include : column dtype or list of column dtypes, default=None A selection of dtypes to include. For more details, see :meth:`pandas.DataFrame.select_dtypes`. dtype_exclude : column dtype or list of column dtypes, default=None A selection of dtypes to exclude. For more details, see :meth:`pandas.DataFrame.select_dtypes`. Returns ------- selector : callable Callable for column selection to be used by a :class:`ColumnTransformer`. See Also -------- ColumnTransformer : Class that allows combining the outputs of multiple transformer objects used on column subsets of the data into a single feature space. Examples -------- >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder >>> from sklearn.compose import make_column_transformer >>> from sklearn.compose import make_column_selector >>> import numpy as np >>> import pandas as pd # doctest: +SKIP >>> X = pd.DataFrame({'city': ['London', 'London', 'Paris', 'Sallisaw'], ... 'rating': [5, 3, 4, 5]}) # doctest: +SKIP >>> ct = make_column_transformer( ... (StandardScaler(), ... make_column_selector(dtype_include=np.number)), # rating ... (OneHotEncoder(), ... make_column_selector(dtype_include=object))) # city >>> ct.fit_transform(X) # doctest: +SKIP array([[ 0.90453403, 1. , 0. , 0. ], [-1.50755672, 1. , 0. , 0. ], [-0.30151134, 0. , 1. , 0. ], [ 0.90453403, 0. , 0. , 1. ]]) N) dtype_include dtype_excludec0||_||_||_dSrI)patternrr)rJrrrs rKrLzmake_column_selector.__init__$s **rMcNt|dstd|jdd}|j|j!||j|j}|j}|j'||j |jd}| S)zCallable for column selection to be used by a :class:`ColumnTransformer`. Parameters ---------- df : dataframe of shape (n_features, n_samples) DataFrame to select columns from. ilocz=make_column_selector can only be applied to pandas dataframesNr9)includeexcludeT)regex) rrXrrr select_dtypesrrrcontainsra)rJdfdf_rowrs rK__call__zmake_column_selector.__call__)sr6"" O !   )T-?-K))*D4F*F~ < #))$,d)CCDD{{}}rMrI)rr|r}r~rLrrPrMrKr2r2sP99v+d$+++++ rMr2cBeZdZdZddddfd ZfdZdZd ZxZS) raA list that raises a warning whenever items are accessed. It is used to store the columns handled by the "remainder" entry of ``ColumnTransformer.transformers_``, ie ``transformers_[-1][-1]``. For some values of the ``ColumnTransformer`` ``transformers`` parameter, this list of indices will be replaced by either a list of column names or a boolean mask; in those cases we emit a ``FutureWarning`` the first time an element is accessed. Parameters ---------- columns : list of int The remainder columns. future_dtype : {'str', 'bool'}, default=None The dtype that will be used by a ColumnTransformer with the same inputs in a future release. There is a default value because providing a constructor that takes a single argument is a requirement for subclasses of UserList, but we do not use it in practice. It would only be used if a user called methods that return a new list such are copying or concatenating `_RemainderColsList`. warning_was_emitted : bool, default=False Whether the warning for that particular list was already shown, so we only emit it once. warning_enabled : bool, default=True When False, the list never emits the warning nor updates `warning_was_emitted``. This is used to obtain a quiet copy of the list for use by the `ColumnTransformer` itself, so that the warning is only shown when a user accesses it directly. NFTrwarning_was_emittedwarning_enabledctt|||_||_||_dSrI)rgrLrrr)rJrrrrrjs rKrLz_RemainderColsList.__init__ds; !!!(#6 .rMcn|t|SrI)_show_remainder_cols_warningrgrh)rJindexrjs rKrhz_RemainderColsList.__getitem__qs- ))+++ww""5)))rMc|js|jsdSd|_dddd|j|j}t jd|dt dS) NTzcolumn names (of type str)za mask array (of type bool)z:a different type depending on the ColumnTransformer inputs)rrNaD The format of the columns of the 'remainder' transformer in ColumnTransformer.transformers_ will change in version 1.7 to match the format of the other transformers. At the moment the remainder columns are stored as indices (of type int). With the same ColumnTransformer configuration, in the future they will be stored as zp. To use the new behavior now and suppress this warning, use ColumnTransformer(force_int_remainder_cols=False). )category)rrryrwarningswarn FutureWarning)rJfuture_dtype_descriptions rKrz/_RemainderColsList._show_remainder_cols_warningus  # 4+?  F#' /1O $ $ #d!2 3 3 !  H 0 HHH# rMcT|t|jdS)zGOverride display in ipython console, otherwise the class name is shown.N)textreprdata)rJprinterrUs rK _repr_pretty_z _RemainderColsList._repr_pretty_s" T$)__%%%%%rM) rr|r}r~rLrhrrrrs@rKrrAs  L! / / / / / / /*****   8&&&&&&&rMrcg}|D]U\}}}t|tr"t|j|j|j|}||||fV|S)Nr)rrrrrr)rr?rrSrTrs rKrxrxs{ F ,..eW g1 2 2 ( $1$+$? / G  tUG,---- MrMrU feature_namerreturnc0|||S)N)rUr)r)rUrrs rKrrs&   )    rM)Or~r collectionsrr functoolsr itertoolsrnumbersrrnumpyr{scipyr baser r r pipelinerrr preprocessingrutilsrutils._estimator_html_reprrutils._indexingrrrutils._metadata_requestsrutils._param_validationrrrrutils._set_outputrrr utils._tagsrutils.metadata_routingr r!r"r#r$utils.metaestimatorsr%utils.parallelr&r'utils.validationr(r)r*r+r,r-r.r/__all__rr0r+rzrr1r2rrxrrrPrMrKrs))))))))""""""""8888888888KKKKKKKKKK//////555555VVVVVVVVVV......NNNNNNNNNNNN #"""""433333........                     S R R uuuuu(*:uuup'GGG$      "!CCCCCLVVVVVVVVrR&R&R&R&R&R&R&R&j   ),:=rM