0Phv dZddlZddlmZddlmZddlmZmZddl Z ddl m Z ddl mZddlmZd d lmZd d lmZmZmZmZmZmZd d lmZd d lmZd dlmZm Z m!Z!d dl"m#Z#m$Z$d dl%m&Z&d dl'm(Z(m)Z)m*Z*m+Z+d dl,m-Z-m.Z.m/Z/m0Z0m1Z1d dl2m3Z3m4Z4d dl5m6Z6m7Z7d dl8m9Z9m:Z:m;Z;mZ>d dl?m@Z@mAZAd dlBmCZCmDZDmEZEmFZFmGZGmHZHmIZIGddeeeZJ d1dZKd2dZLGddZM d3d ZNGd!d"eeZOe1d#gd#geePd$dge/ed dd%&ge0d'd(hgd)d*+dd,d'd-d.ZQGd/d0e3ZRdS)4z0Methods for calibrating predicted probabilities.N) signature)log)IntegralReal)minimize)expit)Bunch)HalfBinomialLoss) BaseEstimatorClassifierMixinMetaEstimatorMixinRegressorMixin _fit_contextclone)FrozenEstimator)IsotonicRegression) LeaveOneOutcheck_cvcross_val_predict) LabelEncoderlabel_binarize) LinearSVC)_safe_indexing column_or_1dget_tags indexable) HasMethodsHiddenInterval StrOptionsvalidate_params)"_BinaryClassifierCurveDisplayMixin_validate_style_kwargs)_get_response_values_process_predict_proba)MetadataRouter MethodMapping_routing_enabledprocess_routing)check_classification_targets)Paralleldelayed)_check_method_params_check_pos_label_consistency_check_response_method_check_sample_weight _num_samplescheck_consistent_lengthcheck_is_fittedc(eZdZUdZeddgeddgdgeddhgdeed hgedgd ed hgd Ze e d < ddddd ddZ dZ e dddZdZdZdZfdZxZS)CalibratedClassifierCVa Probability calibration with isotonic regression or logistic regression. This class uses cross-validation to both estimate the parameters of a classifier and subsequently calibrate a classifier. With default `ensemble=True`, for each cv split it fits a copy of the base estimator to the training subset, and calibrates it using the testing subset. For prediction, predicted probabilities are averaged across these individual calibrated classifiers. When `ensemble=False`, cross-validation is used to obtain unbiased predictions, via :func:`~sklearn.model_selection.cross_val_predict`, which are then used for calibration. For prediction, the base estimator, trained using all the data, is used. This is the prediction method implemented when `probabilities=True` for :class:`~sklearn.svm.SVC` and :class:`~sklearn.svm.NuSVC` estimators (see :ref:`User Guide ` for details). Already fitted classifiers can be calibrated by wrapping the model in a :class:`~sklearn.frozen.FrozenEstimator`. In this case all provided data is used for calibration. The user has to take care manually that data for model fitting and calibration are disjoint. The calibration is based on the :term:`decision_function` method of the `estimator` if it exists, else on :term:`predict_proba`. Read more in the :ref:`User Guide `. In order to learn more on the CalibratedClassifierCV class, see the following calibration examples: :ref:`sphx_glr_auto_examples_calibration_plot_calibration.py`, :ref:`sphx_glr_auto_examples_calibration_plot_calibration_curve.py`, and :ref:`sphx_glr_auto_examples_calibration_plot_calibration_multiclass.py`. Parameters ---------- estimator : estimator instance, default=None The classifier whose output need to be calibrated to provide more accurate `predict_proba` outputs. The default classifier is a :class:`~sklearn.svm.LinearSVC`. .. versionadded:: 1.2 method : {'sigmoid', 'isotonic'}, default='sigmoid' The method to use for calibration. Can be 'sigmoid' which corresponds to Platt's method (i.e. a logistic regression model) or 'isotonic' which is a non-parametric approach. It is not advised to use isotonic calibration with too few calibration samples ``(<<1000)`` since it tends to overfit. cv : int, cross-validation generator, or iterable, default=None Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross-validation, - integer, to specify the number of folds. - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, if ``y`` is binary or multiclass, :class:`~sklearn.model_selection.StratifiedKFold` is used. If ``y`` is neither binary nor multiclass, :class:`~sklearn.model_selection.KFold` is used. Refer to the :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. versionchanged:: 0.22 ``cv`` default value if None changed from 3-fold to 5-fold. .. versionchanged:: 1.6 `"prefit"` is deprecated. Use :class:`~sklearn.frozen.FrozenEstimator` instead. n_jobs : int, default=None Number of jobs to run in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. Base estimator clones are fitted in parallel across cross-validation iterations. Therefore parallelism happens only when `cv != "prefit"`. See :term:`Glossary ` for more details. .. versionadded:: 0.24 ensemble : bool, or "auto", default="auto" Determines how the calibrator is fitted. "auto" will use `False` if the `estimator` is a :class:`~sklearn.frozen.FrozenEstimator`, and `True` otherwise. If `True`, the `estimator` is fitted using training data, and calibrated using testing data, for each `cv` fold. The final estimator is an ensemble of `n_cv` fitted classifier and calibrator pairs, where `n_cv` is the number of cross-validation folds. The output is the average predicted probabilities of all pairs. If `False`, `cv` is used to compute unbiased predictions, via :func:`~sklearn.model_selection.cross_val_predict`, which are then used for calibration. At prediction time, the classifier used is the `estimator` trained on all the data. Note that this method is also internally implemented in :mod:`sklearn.svm` estimators with the `probabilities=True` parameter. .. versionadded:: 0.24 .. versionchanged:: 1.6 `"auto"` option is added and is the default. Attributes ---------- classes_ : ndarray of shape (n_classes,) The class labels. n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. .. versionadded:: 1.0 calibrated_classifiers_ : list (len() equal to cv or 1 if `ensemble=False`) The list of classifier and calibrator pairs. - When `ensemble=True`, `n_cv` fitted `estimator` and calibrator pairs. `n_cv` is the number of cross-validation folds. - When `ensemble=False`, the `estimator`, fitted on all the data, and fitted calibrator. .. versionchanged:: 0.24 Single calibrated classifier case when `ensemble=False`. See Also -------- calibration_curve : Compute true and predicted probabilities for a calibration curve. References ---------- .. [1] Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers, B. Zadrozny & C. Elkan, ICML 2001 .. [2] Transforming Classifier Scores into Accurate Multiclass Probability Estimates, B. Zadrozny & C. Elkan, (KDD 2002) .. [3] Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods, J. Platt, (1999) .. [4] Predicting Good Probabilities with Supervised Learning, A. Niculescu-Mizil & R. Caruana, ICML 2005 Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.naive_bayes import GaussianNB >>> from sklearn.calibration import CalibratedClassifierCV >>> X, y = make_classification(n_samples=100, n_features=2, ... n_redundant=0, random_state=42) >>> base_clf = GaussianNB() >>> calibrated_clf = CalibratedClassifierCV(base_clf, cv=3) >>> calibrated_clf.fit(X, y) CalibratedClassifierCV(...) >>> len(calibrated_clf.calibrated_classifiers_) 3 >>> calibrated_clf.predict_proba(X)[:5, :] array([[0.110..., 0.889...], [0.072..., 0.927...], [0.928..., 0.071...], [0.928..., 0.071...], [0.071..., 0.928...]]) >>> from sklearn.model_selection import train_test_split >>> X, y = make_classification(n_samples=100, n_features=2, ... n_redundant=0, random_state=42) >>> X_train, X_calib, y_train, y_calib = train_test_split( ... X, y, random_state=42 ... ) >>> base_clf = GaussianNB() >>> base_clf.fit(X_train, y_train) GaussianNB() >>> from sklearn.frozen import FrozenEstimator >>> calibrated_clf = CalibratedClassifierCV(FrozenEstimator(base_clf)) >>> calibrated_clf.fit(X_calib, y_calib) CalibratedClassifierCV(...) >>> len(calibrated_clf.calibrated_classifiers_) 1 >>> calibrated_clf.predict_proba([[-0.5, 0.5]]) array([[0.936..., 0.063...]]) fit predict_probadecision_functionNisotonicsigmoid cv_objectprefitbooleanauto estimatormethodcvn_jobsensemble_parameter_constraints)rBrCrDrEcL||_||_||_||_||_dSNr@)selfrArBrCrDrEs S/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/sklearn/calibration.py__init__zCalibratedClassifierCV.__init__s+#    c|j5td}tr|dn|j}|S)z8Resolve which estimator to return (default is LinearSVC)Nr) random_stateT sample_weight)rArr)set_fit_request)rIrAs rJ_get_estimatorz%CalibratedClassifierCV._get_estimatorsO > !"q111I!! >)))===IrLFprefer_skip_nested_validationc tt\tj}|dkrt t  }g_jdkrtj dtj dgj j _ tddg \}}|jd kr|d d }t#|j j}j|nt)} | j _ t-rt/d fd i|nt1jj} d | v} .| s,t5j} tj d| dt9t9i_t9|_ | rj jd <t jt<rj} n$t?jdr jj } nd} | rHtCj"tCj#dd | krtId| d| dt jtJrtIdtMjd}|rNtOj(}|fd|j)fijj)D_ntU}tW|ddgj}tY|||j(j j}t[j dkr?|dkr#t]|dj j d }|d d }|jfij jt#||j j}j|jd j }t?|d!r |j/_/t?|d"r |j0_0S)#aOFit the calibrated model. Parameters ---------- X : array-like of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) Target values. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. **fit_params : dict Parameters to pass to the `fit` method of the underlying classifier. Returns ------- self : object Returns an instance of self. Nr?r=zThe `cv='prefit'` option is deprecated in 1.6 and will be removed in 1.8. You can use CalibratedClassifierCV(FrozenEstimator(estimator)) instead.classes_) attributesr9r8response_methodr r7rPzSince aW does not appear to accept sample_weight, sample weights will only be used for the calibration itself. This can be caused by a limitation of the current scikit-learn API. See the following issue for more details: https://github.com/scikit-learn/scikit-learn/issues/21134. Be warned that the result of the calibration is likely to be incorrect.)split)r7n_splitsT) return_countsz Requesting z.-fold cross-validation but provided less than z! examples for at least one class.zLeaveOneOut cross-validation does not allowall classes to be present in test splits. Please use a cross-validation generator that allows all classes to appear in every test and train split.) classifier)rDc 3K|]N\}}ttt||jjjj VOdS))traintestrBclassesrP fit_paramsN)r-_fit_classifier_calibrator_pairrrBrVrAr7) .0r`raXrA routed_paramsrPrIys rJ z-CalibratedClassifierCV.fit..s 8 8$t=G;<<i((#!#{ $ &3#0#:#>    8 8 8 8 8 8rL)rArfrhrCrBrDparamsbinary)y_pred target_typerb pos_labelrn_features_in_feature_names_in_)1r+rr1rRrE isinstancercalibrated_classifiers_rCwarningswarnr4rArVr%ndimreshape_fit_calibratorrBappendrr7r)r*r parameterstype__name__r splitterinthasattrr\npanyunique ValueErrorrrr,rDr[rr0rlenr&rprq)rIrfrhrPrc _ensemble predictions_calibrated_classifierlabel_encoder_fit_parameters supports_swestimator_namen_foldsrCparallelthis_estimator method_name first_clfrArgs```` @@rJr7zCalibratedClassifierCV.fit$sc6 %Q'''A1  $0BBM'')) M   &y/BBBI')$ 7h   M    DN | D D D D N3DM1!4o FNK 1$$)11"a88 $3   %% !  ( / /0E F F F F*^^//22N*3DM!! Q /!!#0!! !! "+9=!9!9!D-?  ,[,%))__%=NM&&&&   !& ).R &*/J*?*?*? ' ,,CPM+/@$'3'' '*-- '* 26")AT"B"B"B1"E"OPP  B'BBBBB $';//  K $'1666B6 K#4;777/7x 8 8 8 8 8 8 8 8 8(0rx1'U'U 8N8T'U'U 8 8 8 0 0,,"'y!1!14"(/:0,&;(26 t}%%**"o55&<#.(0$(M&*mA&6 ''' #."5"5b!"<"` on how the routing mechanism works. Returns ------- routing : MetadataRouter A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating routing information. )ownerr7)callercallee)rAmethod_mappingr[)r}r)r' __class__r|add_self_requestaddrRr(rC)rIrouters rJget_metadata_routingz+CalibratedClassifierCV.get_metadata_routings !8 9 9 9  d # # S--//,22%2NNS,22%2PP  rLct}t|jj|j_|SrH)super__sklearn_tags__rrR input_tagssparse)rItagsrs rJrz'CalibratedClassifierCV.__sklearn_tags__)sAww''))!)$*=*=*?*?!@!@!K!R rLrH)r| __module__ __qualname____doc__rr!rrrFdict__annotations__rKrRrr7r8rrr __classcell__)rs@rJr6r6<s}}B J/ 0 0 J23 4 4  :z95667FF::xj#9#9::;T" F8 4 45 $ $D    !  ! ! ! ! !   \&+vvv vp8GGG&4rLr6c t|||} t||t||} } t||t||} } |j| | fi| t|| ddg\}}|jdkr|dd}|dnt||}t ||| |||}|S) a-Fit a classifier/calibration pair on a given train/test split. Fit the classifier on the train set, compute its predictions on the test set and use the predictions as input to fit the calibrator along with the test labels. Parameters ---------- estimator : estimator instance Cloned base estimator. X : array-like, shape (n_samples, n_features) Sample data. y : array-like, shape (n_samples,) Targets. train : ndarray, shape (n_train_indices,) Indices of the training subset. test : ndarray, shape (n_test_indices,) Indices of the testing subset. method : {'sigmoid', 'isotonic'} Method to use for calibration. classes : ndarray, shape (n_classes,) The target classes. sample_weight : array-like, default=None Sample weights for `X`. fit_params : dict, default=None Parameters to pass to the `fit` method of the underlying classifier. Returns ------- calibrated_classifier : _CalibratedClassifier instance )rjindicesr9r8rXr rZNrO)r.rr7r%rvrwrx)rArfrhr`rarBrbrPrcfit_params_trainX_trainy_trainX_testy_testrrsw_testrs rJrdrd/sf,Aj%PPP%a//51I1IWG#At,,nQ.E.EFFIM'777&6777),o>NK 1!))"a00 #+dd t1T1TG+;w ! rLct||}t|}||j}g} t ||jD]`\} } |dkrtd} nt} | | |dd| f|| | at|| ||} | S)aFit calibrator(s) and return a `_CalibratedClassifier` instance. `n_classes` (i.e. `len(clf.classes_)`) calibrators are fitted. However, if `n_classes` equals 2, one calibrator is fitted. Parameters ---------- clf : estimator instance Fitted classifier. predictions : array-like, shape (n_samples, n_classes) or (n_samples, 1) when binary. Raw predictions returned by the un-calibrated base classifier. y : array-like, shape (n_samples,) The targets. classes : ndarray, shape (n_classes,) All the prediction classes. method : {'sigmoid', 'isotonic'} The method to use for calibration. sample_weight : ndarray, shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- pipeline : _CalibratedClassifier instance )rbr:clip) out_of_boundsN)rBrb) rrr7 transformrVzipTr_SigmoidCalibrationry_CalibratedClassifier)clfrrhrbrBrPY label_encoderpos_class_indices calibrators class_idx this_pred calibratorpipelines rJrxrxxs@ q'***A NN&&w//M%// ==K #$5{} E E'' 9 Z  +&AAAJJ,..Jy!AAAyL/=AAA:&&&&$S+fgVVVH OrLc$eZdZdZdddZdZdS)ra$Pipeline-like chaining a fitted classifier and its fitted calibrators. Parameters ---------- estimator : estimator instance Fitted classifier. calibrators : list of fitted estimator instances List of fitted calibrators (either 'IsotonicRegression' or '_SigmoidCalibration'). The number of calibrators equals the number of classes. However, if there are 2 classes, the list contains only one fitted calibrator. classes : array-like of shape (n_classes,) All the prediction classes. method : {'sigmoid', 'isotonic'}, default='sigmoid' The method to use for calibration. Can be 'sigmoid' which corresponds to Platt's method or 'isotonic' which is a non-parametric approach based on isotonic regression. r;)rBc>||_||_||_||_dSrH)rArrbrB)rIrArrbrBs rJrKz_CalibratedClassifier.__init__s#"&  rLc t|j|ddg\}}|jdkr|dd}t |j}t |j}||jj }tj t||f}t||j|jD]-\}} } |dkr|dz }| | |dd|f<.|dkrd|dddfz |ddd f<n^tj|d ddtjf} tj|d|z } tj|| | | d k }d|d|k|d kz<|S) aCalculate calibrated probabilities. Calculates classification calibrated probabilities for each class, in a one-vs-all manner, for `X`. Parameters ---------- X : ndarray of shape (n_samples, n_features) The sample data. Returns ------- proba : array, shape (n_samples, n_classes) The predicted probabilities. Can be exact zeros. r9r8rXr rZrkN?rr)outwheregrZ| ?)r%rArvrwrrbrr7rrVrrr2rrrrsumnewaxis full_likedivide) rIrfrr n_classesrrrrrr denominator uniform_probas rJr8z#_CalibratedClassifier.predict_probas . N 0/B    Q  q %--b!44K %% $**4<88 )33DN4KLL,q//956603 {}d.>1 1  @ @ ,Iy*A~~Q ","4"4Y"?"?E!!!Y,   >>aaad +E!!!Q$KK&Q///2: >KLI >>MI{ [A=ME 8;sU{u 234 rLN)r|rrrrKr8rLrJrrsM,CL 88888rLrc 2 tt|} d}tjtj }||kr|} |z |dk}6|}|}n1t tj|}|jd|z }tj|j|dz|dzz |dk<d|dzz |dk<t fd} tj dt|dz|dzz g} t| | dd d d tj t jzd  } | j} | d|z | dfS)aNProbability Calibration with sigmoid method (Platt 2000) Parameters ---------- predictions : ndarray of shape (n_samples,) The decision function or predict proba for the samples. y : ndarray of shape (n_samples,) The targets. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- a : float The slope. b : float The intercept. References ---------- Platt, "Probabilistic Outputs for Support Vector Machines" rrNdtypeg@c8|dz|dz j }| \}}|}t j| z| gtj}||fS)Nrr r)y_trueraw_predictionrP)astyper loss_gradientrrasarrayfloat64) ABrlglossgradFrbin_lossrrPs rJ loss_gradz'_sigmoid_calibration..loss_gradAs a519r!u,44;;L4MMM%%)'&  1 uuww zA26AEEGG8,BJ???TzrLzL-BFGS-BTgư>@)gtolftol)rBjacoptionsr )rrmaxabsrfloatshape zeros_likerr arrayrrfinfoepsx)rrhrPmax_abs_prediction_thresholdscale_constantmax_predictionmask_negative_samplesprior0prior1rAB0 opt_resultAB_rrrs` ` @@@rJ_sigmoid_calibrationrs8{++KQAANVBF1II&&N555'   F  56;;==!6 67<<>>rv34455f$ a{0111A #.Aa!eHv|$Aa1fI!!H& (Cfslv|<==> ? ?C  %,,    J ,C q6N "CF **rLc eZdZdZddZdZdS)rzSigmoid regression model. Attributes ---------- a_ : float The slope. b_ : float The intercept. Nct|}t|}t||\}}t|||\|_|_|S)aFit the model using X, y as training data. Parameters ---------- X : array-like of shape (n_samples,) Training data. y : array-like of shape (n_samples,) Training target. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Returns ------- self : object Returns an instance of self. )rrra_b_)rIrfrhrPs rJr7z_SigmoidCalibration.fittsJ& OO OOA1/1mDD rLc`t|}t|j|z|jz S)aPredict new data by linear interpolation. Parameters ---------- T : array-like of shape (n_samples,) Data to predict from. Returns ------- T_ : ndarray of shape (n_samples,) The predicted data. )rrrr)rIrs rJrz_SigmoidCalibration.predicts- OOtw{TW,-...rLrH)r|rrrr7rrrLrJrrhsA  4/////rLrz array-liker>left)closeduniformquantile)ry_probron_binsstrategyTrS)rorrct|}t|}t||t||}|dks|dkrt dt j|}t|dkrt d|d||k}|dkr2t j dd|dz}t j ||dz}n/|d krt j d d |dz}nt d t j |dd |}t j ||t|} t j ||t|} t j |t|} | dk} | | | | z } | | | | z }| |fS)aCompute true and predicted probabilities for a calibration curve. The method assumes the inputs come from a binary classifier, and discretize the [0, 1] interval into bins. Calibration curves may also be referred to as reliability diagrams. Read more in the :ref:`User Guide `. Parameters ---------- y_true : array-like of shape (n_samples,) True targets. y_prob : array-like of shape (n_samples,) Probabilities of the positive class. pos_label : int, float, bool or str, default=None The label of the positive class. .. versionadded:: 1.1 n_bins : int, default=5 Number of bins to discretize the [0, 1] interval. A bigger number requires more data. Bins with no samples (i.e. without corresponding values in `y_prob`) will not be returned, thus the returned arrays may have less than `n_bins` values. strategy : {'uniform', 'quantile'}, default='uniform' Strategy used to define the widths of the bins. uniform The bins have identical widths. quantile The bins have the same number of samples and depend on `y_prob`. Returns ------- prob_true : ndarray of shape (n_bins,) or smaller The proportion of samples whose class is the positive class, in each bin (fraction of positives). prob_pred : ndarray of shape (n_bins,) or smaller The mean predicted probability in each bin. References ---------- Alexandru Niculescu-Mizil and Rich Caruana (2005) Predicting Good Probabilities With Supervised Learning, in Proceedings of the 22nd International Conference on Machine Learning (ICML). See section 4 (Qualitative Analysis of Predictions). Examples -------- >>> import numpy as np >>> from sklearn.calibration import calibration_curve >>> y_true = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1]) >>> y_pred = np.array([0.1, 0.2, 0.3, 0.4, 0.65, 0.7, 0.8, 0.9, 1.]) >>> prob_true, prob_pred = calibration_curve(y_true, y_pred, n_bins=3) >>> prob_true array([0. , 0.5, 1. ]) >>> prob_pred array([0.2 , 0.525, 0.85 ]) rr z!y_prob has values outside [0, 1].rkz9Only binary classification is supported. Provided labels .r dr rrzSInvalid entry to 'strategy' input. Strategy must be either 'quantile' or 'uniform'.rZ)weights minlength)r) rr3r/minrrrrrlinspace percentile searchsortedbincount)rr rorrlabels quantilesbinsbinidsbin_sumsbin_true bin_totalnonzero prob_true prob_preds rJcalibration_curver%sd& ! !F & ! !FFF+++,Y??I zz||a6::<` and more about the scikit-learn visualization API in :ref:`visualizations`. For an example on how to use the visualization, see :ref:`sphx_glr_auto_examples_calibration_plot_calibration_curve.py`. .. versionadded:: 1.0 Parameters ---------- prob_true : ndarray of shape (n_bins,) The proportion of samples whose class is the positive class (fraction of positives), in each bin. prob_pred : ndarray of shape (n_bins,) The mean predicted probability in each bin. y_prob : ndarray of shape (n_samples,) Probability estimates for the positive class, for each sample. estimator_name : str, default=None Name of estimator. If None, the estimator name is not shown. pos_label : int, float, bool or str, default=None The positive class when computing the calibration curve. By default, `pos_label` is set to `estimators.classes_[1]` when using `from_estimator` and set to 1 when using `from_predictions`. .. versionadded:: 1.1 Attributes ---------- line_ : matplotlib Artist Calibration curve. ax_ : matplotlib Axes Axes with calibration curve. figure_ : matplotlib Figure Figure containing the curve. See Also -------- calibration_curve : Compute true and predicted probabilities for a calibration curve. CalibrationDisplay.from_predictions : Plot calibration curve using true and predicted labels. CalibrationDisplay.from_estimator : Plot calibration curve using an estimator and data. Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.model_selection import train_test_split >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.calibration import calibration_curve, CalibrationDisplay >>> X, y = make_classification(random_state=0) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, random_state=0) >>> clf = LogisticRegression(random_state=0) >>> clf.fit(X_train, y_train) LogisticRegression(random_state=0) >>> y_prob = clf.predict_proba(X_test)[:, 1] >>> prob_true, prob_pred = calibration_curve(y_test, y_prob, n_bins=10) >>> disp = CalibrationDisplay(prob_true, prob_pred, y_prob) >>> disp.plot() <...> N)rrocL||_||_||_||_||_dSrHr#r$r rro)rIr#r$r rros rJrKzCalibrationDisplay.__init__es,#" ,"rLT)axnameref_linec |||\|_|_}|j d|jdnd}ddd}|||d <t ||}d }||jd v} |r$| s"|jd d gd d gd ||jj|j|jfi|d |_ |j dd|} d|} |j | | |S)aTPlot visualization. Extra keyword arguments will be passed to :func:`matplotlib.pyplot.plot`. Parameters ---------- ax : Matplotlib Axes, default=None Axes object to plot on. If `None`, a new figure and axes is created. name : str, default=None Name for labeling curve. If `None`, use `estimator_name` if not `None`, otherwise no labeling is shown. ref_line : bool, default=True If `True`, plots a reference line representing a perfectly calibrated classifier. **kwargs : dict Keyword arguments to be passed to :func:`matplotlib.pyplot.plot`. Returns ------- display : :class:`~sklearn.calibration.CalibrationDisplay` Object that stores computed values. )r*r+Nz(Positive class: )s-)marker linestylelabelzPerfectly calibratedr rzk:)r4z lower right)loczMean predicted probability zFraction of positives )xlabelylabel) _validate_plot_paramsax_figure_ror$get_legend_handles_labelsplotr$r#line_legendset) rIr*r+r,kwargsinfo_pos_labeldefault_line_kwargs line_kwargsref_line_labelexisting_ref_liner6r7s rJr<zCalibrationDisplay.plotnsO8(,'A'ARd'A'S'S$$,6:^5O 1 1 1 1 1UW *-3??  +/  (,-@&II /*dh.P.P.R.RST.UU  F- F HMM1a&1a&$nM E E E"TX]4>4>QQ[QQRST  M***?~??:.::  F6 222 rLrr rrror+r,r*c l||||d||\} }}|j|| f|||||| d| S)a Plot calibration curve using a binary classifier and data. A calibration curve, also known as a reliability diagram, uses inputs from a binary classifier and plots the average predicted probability for each bin against the fraction of positive classes, on the y-axis. Extra keyword arguments will be passed to :func:`matplotlib.pyplot.plot`. Read more about calibration in the :ref:`User Guide ` and more about the scikit-learn visualization API in :ref:`visualizations`. .. versionadded:: 1.0 Parameters ---------- estimator : estimator instance Fitted classifier or a fitted :class:`~sklearn.pipeline.Pipeline` in which the last estimator is a classifier. The classifier must have a :term:`predict_proba` method. X : {array-like, sparse matrix} of shape (n_samples, n_features) Input values. y : array-like of shape (n_samples,) Binary target values. n_bins : int, default=5 Number of bins to discretize the [0, 1] interval into when calculating the calibration curve. A bigger number requires more data. strategy : {'uniform', 'quantile'}, default='uniform' Strategy used to define the widths of the bins. - `'uniform'`: The bins have identical widths. - `'quantile'`: The bins have the same number of samples and depend on predicted probabilities. pos_label : int, float, bool or str, default=None The positive class when computing the calibration curve. By default, `estimators.classes_[1]` is considered as the positive class. .. versionadded:: 1.1 name : str, default=None Name for labeling curve. If `None`, the name of the estimator is used. ref_line : bool, default=True If `True`, plots a reference line representing a perfectly calibrated classifier. ax : matplotlib axes, default=None Axes object to plot on. If `None`, a new figure and axes is created. **kwargs : dict Keyword arguments to be passed to :func:`matplotlib.pyplot.plot`. Returns ------- display : :class:`~sklearn.calibration.CalibrationDisplay`. Object that stores computed values. See Also -------- CalibrationDisplay.from_predictions : Plot calibration curve using true and predicted labels. Examples -------- >>> import matplotlib.pyplot as plt >>> from sklearn.datasets import make_classification >>> from sklearn.model_selection import train_test_split >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.calibration import CalibrationDisplay >>> X, y = make_classification(random_state=0) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, random_state=0) >>> clf = LogisticRegression(random_state=0) >>> clf.fit(X_train, y_train) LogisticRegression(random_state=0) >>> disp = CalibrationDisplay.from_estimator(clf, X_test, y_test) >>> plt.show() r8)rYror+rF)!_validate_and_get_response_valuesfrom_predictions) clsrArfrhrrror+r,r*r@r s rJfrom_estimatorz!CalibrationDisplay.from_estimatorsN#&"G"G  + #H# #  4$s#         rLc |||d||\} }t|||||\} } || | ||| } | jd||d| S)a Plot calibration curve using true labels and predicted probabilities. Calibration curve, also known as reliability diagram, uses inputs from a binary classifier and plots the average predicted probability for each bin against the fraction of positive classes, on the y-axis. Extra keyword arguments will be passed to :func:`matplotlib.pyplot.plot`. Read more about calibration in the :ref:`User Guide ` and more about the scikit-learn visualization API in :ref:`visualizations`. .. versionadded:: 1.0 Parameters ---------- y_true : array-like of shape (n_samples,) True labels. y_prob : array-like of shape (n_samples,) The predicted probabilities of the positive class. n_bins : int, default=5 Number of bins to discretize the [0, 1] interval into when calculating the calibration curve. A bigger number requires more data. strategy : {'uniform', 'quantile'}, default='uniform' Strategy used to define the widths of the bins. - `'uniform'`: The bins have identical widths. - `'quantile'`: The bins have the same number of samples and depend on predicted probabilities. pos_label : int, float, bool or str, default=None The positive class when computing the calibration curve. By default `pos_label` is set to 1. .. versionadded:: 1.1 name : str, default=None Name for labeling curve. ref_line : bool, default=True If `True`, plots a reference line representing a perfectly calibrated classifier. ax : matplotlib axes, default=None Axes object to plot on. If `None`, a new figure and axes is created. **kwargs : dict Keyword arguments to be passed to :func:`matplotlib.pyplot.plot`. Returns ------- display : :class:`~sklearn.calibration.CalibrationDisplay`. Object that stores computed values. See Also -------- CalibrationDisplay.from_estimator : Plot calibration curve using an estimator and data. Examples -------- >>> import matplotlib.pyplot as plt >>> from sklearn.datasets import make_classification >>> from sklearn.model_selection import train_test_split >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.calibration import CalibrationDisplay >>> X, y = make_classification(random_state=0) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, random_state=0) >>> clf = LogisticRegression(random_state=0) >>> clf.fit(X_train, y_train) LogisticRegression(random_state=0) >>> y_prob = clf.predict_proba(X_test)[:, 1] >>> disp = CalibrationDisplay.from_predictions(y_test, y_prob) >>> plt.show() N)rPror+)rrror))r*r,r)!_validate_from_predictions_paramsr%r<)rJrr rrror+r,r*r@pos_label_validatedr#r$disps rJrIz#CalibrationDisplay.from_predictions s@%($I$I F$)$%J% % !T 1 F6H     9s)    tyrgs66 """"""""############$#####((((((EEEEEEEEEE77777777DDDDDDDDDDDDXWWWWWWWIIIIIIII ;:::::--------ppppp_.@-pppvF!F!F!F!R----`UUUUUUUUvFHc+c+c+c+L4/4/4/4/4/.-4/4/4/n..CD18Haf===>ZJ 7889 #'     l l l l   l ^w=w=w=w=w=;w=w=w=w=w=rL