0PhedZddlZddlmZmZddlZddlmZm Z ddl m Z m Z m Z ddlmZddlmZdd lmZmZdd lmZdd lmZdd lmZdd lmZmZmZmZm Z ddl!m"Z"m#Z#Gdde e Z$dZ%dZ&e ej'dgeedddeedddgeedddgee(edhze)geeddddge*dgehdgeedddgedgd ddZ+dZ,e ej'gej'gej'geedddgd d!d"Z-e ej'gej'gej'geedddeedddgeedddeeddddgeedddgd#gd$d!dd%d!d&d'Z.d(Z/d)Z0d*Z1d+Z2d,Z3dS)-a}Ordering Points To Identify the Clustering Structure (OPTICS) These routines execute the OPTICS algorithm, and implement various cluster extraction methods of the ordered list. Authors: Shane Grigsby Adrin Jalali Erich Schubert Hanmin Qin License: BSD 3 clause N)IntegralReal)SparseEfficiencyWarningissparse) BaseEstimator ClusterMixin _fit_context)DataConversionWarning)pairwise_distances)_VALID_METRICSPAIRWISE_BOOLEAN_FUNCTIONS)NearestNeighbors) gen_batches)get_chunk_n_rows) HasMethodsInterval RealNotInt StrOptionsvalidate_params) check_memory validate_dataceZdZUdZeedddeedddgeedddgee e d hze geedddge dged d hgeeddddgeedddgd geedddeeddd dgehdgeedddge eddgedgdZe ed<dejdddd ddddddddddZedddZdS)OPTICSu $Estimate clustering structure from vector array. OPTICS (Ordering Points To Identify the Clustering Structure), closely related to DBSCAN, finds core sample of high density and expands clusters from them [1]_. Unlike DBSCAN, keeps cluster hierarchy for a variable neighborhood radius. Better suited for usage on large datasets than the current sklearn implementation of DBSCAN. Clusters are then extracted using a DBSCAN-like method (cluster_method = 'dbscan') or an automatic technique proposed in [1]_ (cluster_method = 'xi'). This implementation deviates from the original OPTICS by first performing k-nearest-neighborhood searches on all points to identify core sizes, then computing only the distances to unprocessed points when constructing the cluster order. Note that we do not employ a heap to manage the expansion candidates, so the time complexity will be O(n^2). Read more in the :ref:`User Guide `. Parameters ---------- min_samples : int > 1 or float between 0 and 1, default=5 The number of samples in a neighborhood for a point to be considered as a core point. Also, up and down steep regions can't have more than ``min_samples`` consecutive non-steep points. Expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2). max_eps : float, default=np.inf The maximum distance between two samples for one to be considered as in the neighborhood of the other. Default value of ``np.inf`` will identify clusters across all scales; reducing ``max_eps`` will result in shorter run times. metric : str or callable, default='minkowski' Metric to use for distance computation. Any metric from scikit-learn or scipy.spatial.distance can be used. If metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays as input and return one value indicating the distance between them. This works for Scipy's metrics, but is less efficient than passing the metric name as a string. If metric is "precomputed", `X` is assumed to be a distance matrix and must be square. Valid values for metric are: - from scikit-learn: ['cityblock', 'cosine', 'euclidean', 'l1', 'l2', 'manhattan'] - from scipy.spatial.distance: ['braycurtis', 'canberra', 'chebyshev', 'correlation', 'dice', 'hamming', 'jaccard', 'kulsinski', 'mahalanobis', 'minkowski', 'rogerstanimoto', 'russellrao', 'seuclidean', 'sokalmichener', 'sokalsneath', 'sqeuclidean', 'yule'] Sparse matrices are only supported by scikit-learn metrics. See the documentation for scipy.spatial.distance for details on these metrics. .. note:: `'kulsinski'` is deprecated from SciPy 1.9 and will be removed in SciPy 1.11. p : float, default=2 Parameter for the Minkowski metric from :class:`~sklearn.metrics.pairwise_distances`. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used. metric_params : dict, default=None Additional keyword arguments for the metric function. cluster_method : str, default='xi' The extraction method used to extract clusters using the calculated reachability and ordering. Possible values are "xi" and "dbscan". eps : float, default=None The maximum distance between two samples for one to be considered as in the neighborhood of the other. By default it assumes the same value as ``max_eps``. Used only when ``cluster_method='dbscan'``. xi : float between 0 and 1, default=0.05 Determines the minimum steepness on the reachability plot that constitutes a cluster boundary. For example, an upwards point in the reachability plot is defined by the ratio from one point to its successor being at most 1-xi. Used only when ``cluster_method='xi'``. predecessor_correction : bool, default=True Correct clusters according to the predecessors calculated by OPTICS [2]_. This parameter has minimal effect on most datasets. Used only when ``cluster_method='xi'``. min_cluster_size : int > 1 or float between 0 and 1, default=None Minimum number of samples in an OPTICS cluster, expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2). If ``None``, the value of ``min_samples`` is used instead. Used only when ``cluster_method='xi'``. algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto' Algorithm used to compute the nearest neighbors: - 'ball_tree' will use :class:`~sklearn.neighbors.BallTree`. - 'kd_tree' will use :class:`~sklearn.neighbors.KDTree`. - 'brute' will use a brute-force search. - 'auto' (default) will attempt to decide the most appropriate algorithm based on the values passed to :meth:`fit` method. Note: fitting on sparse input will override the setting of this parameter, using brute force. leaf_size : int, default=30 Leaf size passed to :class:`~sklearn.neighbors.BallTree` or :class:`~sklearn.neighbors.KDTree`. This can affect the speed of the construction and query, as well as the memory required to store the tree. The optimal value depends on the nature of the problem. memory : str or object with the joblib.Memory interface, default=None Used to cache the output of the computation of the tree. By default, no caching is done. If a string is given, it is the path to the caching directory. n_jobs : int, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- labels_ : ndarray of shape (n_samples,) Cluster labels for each point in the dataset given to fit(). Noisy samples and points which are not included in a leaf cluster of ``cluster_hierarchy_`` are labeled as -1. reachability_ : ndarray of shape (n_samples,) Reachability distances per sample, indexed by object order. Use ``clust.reachability_[clust.ordering_]`` to access in cluster order. ordering_ : ndarray of shape (n_samples,) The cluster ordered list of sample indices. core_distances_ : ndarray of shape (n_samples,) Distance at which each sample becomes a core point, indexed by object order. Points which will never be core have a distance of inf. Use ``clust.core_distances_[clust.ordering_]`` to access in cluster order. predecessor_ : ndarray of shape (n_samples,) Point that a sample was reached from, indexed by object order. Seed points have a predecessor of -1. cluster_hierarchy_ : ndarray of shape (n_clusters, 2) The list of clusters in the form of ``[start, end]`` in each row, with all indices inclusive. The clusters are ordered according to ``(end, -start)`` (ascending) so that larger clusters encompassing smaller clusters come after those smaller ones. Since ``labels_`` does not reflect the hierarchy, usually ``len(cluster_hierarchy_) > np.unique(optics.labels_)``. Please also note that these indices are of the ``ordering_``, i.e. ``X[ordering_][start:end + 1]`` form a cluster. Only available when ``cluster_method='xi'``. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- DBSCAN : A similar clustering for a specified neighborhood radius (eps). Our implementation is optimized for runtime. References ---------- .. [1] Ankerst, Mihael, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. "OPTICS: ordering points to identify the clustering structure." ACM SIGMOD Record 28, no. 2 (1999): 49-60. .. [2] Schubert, Erich, Michael Gertz. "Improving the Cluster Structure Extracted from OPTICS Plots." Proc. of the Conference "Lernen, Wissen, Daten, Analysen" (LWDA) (2018): 318-329. Examples -------- >>> from sklearn.cluster import OPTICS >>> import numpy as np >>> X = np.array([[1, 2], [2, 5], [3, 6], ... [8, 7], [8, 8], [7, 3]]) >>> clustering = OPTICS(min_samples=2).fit(X) >>> clustering.labels_ array([0, 0, 0, 1, 1, 1]) For a more detailed example see :ref:`sphx_glr_auto_examples_cluster_plot_optics.py`. rNleftclosedrboth precomputeddbscanxibooleanright>autobrutekd_tree ball_treecache) min_samplesmax_epsmetricp metric_paramscluster_methodepsr"predecessor_correctionmin_cluster_size algorithm leaf_sizememoryn_jobs_parameter_constraints minkowski皙?Tr%c||_||_| |_| |_||_||_||_| |_||_||_ ||_ | |_ | |_ ||_ dSN)r+r*r2r3r,r.r-r4r/r0r"r1r5r6)selfr*r+r,r-r.r/r0r"r1r2r3r4r5r6s W/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/sklearn/cluster/_optics.py__init__zOPTICS.__init__ so$ & 0" *",&<#  Fprefer_skip_nested_validationc |jtvrtnt}|tur5|jtkr%d|jd}t j|tt|||d}|jdkrt|r| }t j 5t j dt||dddn #1swxYwYt!|j}|t&||j|j|j|j|j|j|j|j \|_|_|_|_|jd krDtA|j|j|j|j|j!|j"|j# \}}||_$ni|jd kr^|j%|j}n|j%}||jkrtMd |jd |dtO|j|j|j|}||_(|S)aPerform OPTICS clustering. Extracts an ordered list of points and reachability distances, and performs initial clustering using ``max_eps`` distance specified at OPTICS object instantiation. Parameters ---------- X : {ndarray, sparse matrix} of shape (n_samples, n_features), or (n_samples, n_samples) if metric='precomputed' A feature array, or array of distances between samples if metric='precomputed'. If a sparse matrix is provided, it will be converted into CSR format. y : Ignored Not used, present for API consistency by convention. Returns ------- self : object Returns a fitted instance of self. z-Data will be converted to boolean for metric zG, to avoid this warning, you may convert the data prior to calling fit.csr)dtype accept_sparser ignoreN) Xr*r3r4r,r.r-r6r+r" reachability predecessororderingr*r2r"r1r!z Specify an epsilon smaller than z. Got .rKcore_distancesrMr0))r,rboolfloatrFwarningswarnr rrcopycatch_warnings simplefilterrsetdiagdiagonalrr5r)compute_optics_graphr*r3r4r.r-r6r+ ordering_core_distances_ reachability_ predecessor_r/cluster_optics_xir2r"r1cluster_hierarchy_r0 ValueErrorcluster_optics_dbscanlabels_) r>rIyrFmsgr5rc clusters_r0s r?fitz OPTICS.fit.s6 'AAAu D==QW__B;BBB  M#4 5 5 5 $e D D D ;- ' 'HQKK 'A(** ( (%h0GHHH !**,,'''  ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( dk** /FLL- . .(nn;,f;L  N       $ & &!2!/ - ,!%!67'+'B""" GY'0D # #  H , ,xlhT\!! jEI\\\SVSVSVW,!/#3 G  s/AC==DDr=)__name__ __module__ __qualname____doc__rrrrrsetr callabledictstrrr7__annotations__npinfr@r rgrAr?rr's'KK^ HXq$v 6 6 6 HZAf 5 5 5 HT1d6:::;:cc.11]OCDDhOhtQV444 5%:x&6778q$v666=xa62223#,+ HXq$v 6 6 6 HZAg 6 6 6  !j!J!J!JKKLhxD@@@A 7++T2T"+$$D6   #!B\&+ZZZ ZZZrArc<||krtd|||fzdS)Nz=%s must be no greater than the number of samples (%d). Got %d)ra)size n_samples param_names r?_validate_sizerxs9 i K9d+ ,   rAc8|jd}tj|}|tjt d|z||}t ||}|D]1}||||ddddf||<2|S)aCompute the k-th nearest neighbor of each sample. Equivalent to neighbors.kneighbors(X, self.min_samples)[0][:, -1] but with more memory efficiency. Parameters ---------- X : array-like of shape (n_samples, n_features) The data. neighbors : NearestNeighbors instance The fitted nearest neighbors estimator. working_memory : int, default=None The sought maximum memory for temporary distance matrix chunks. When None (default), the value of ``sklearn.get_config()['working_memory']`` is used. Returns ------- core_distances : ndarray of shape (n_samples,) Distance at which each sample becomes a core point. Points which will never be core have a distance of inf. r) row_bytes max_n_rowsworking_memoryN)shaperqemptyfillnanrr kneighbors) rI neighborsr*r}rvrP chunk_n_rowsslicessls r?_compute_core_distances_rs. IXi((N#{"yLL 1 1FPP&11!B%EEaHBOr rAz sparse matrixrrrrr r$>r%r&r'r() rIr*r+r,r-r.r3r4r6FrBc|jd} t|| d|dkr tdt|| z}t j| } | tjt j| t} | dt|||||||} | |t|| |d } tj| | |k<t j | t j | j j| t j|jdt }t j|jdt}t#|jdD]z}t j|dkd}|t j| |}d ||<|||<| |tjkrt)| | | |||| |||| {t jt j| rt/jd t2|| | | fS)uCompute the OPTICS reachability graph. Read more in the :ref:`User Guide `. Parameters ---------- X : {ndarray, sparse matrix} of shape (n_samples, n_features), or (n_samples, n_samples) if metric='precomputed' A feature array, or array of distances between samples if metric='precomputed'. min_samples : int > 1 or float between 0 and 1 The number of samples in a neighborhood for a point to be considered as a core point. Expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2). max_eps : float, default=np.inf The maximum distance between two samples for one to be considered as in the neighborhood of the other. Default value of ``np.inf`` will identify clusters across all scales; reducing ``max_eps`` will result in shorter run times. metric : str or callable, default='minkowski' Metric to use for distance computation. Any metric from scikit-learn or scipy.spatial.distance can be used. If metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays as input and return one value indicating the distance between them. This works for Scipy's metrics, but is less efficient than passing the metric name as a string. If metric is "precomputed", X is assumed to be a distance matrix and must be square. Valid values for metric are: - from scikit-learn: ['cityblock', 'cosine', 'euclidean', 'l1', 'l2', 'manhattan'] - from scipy.spatial.distance: ['braycurtis', 'canberra', 'chebyshev', 'correlation', 'dice', 'hamming', 'jaccard', 'kulsinski', 'mahalanobis', 'minkowski', 'rogerstanimoto', 'russellrao', 'seuclidean', 'sokalmichener', 'sokalsneath', 'sqeuclidean', 'yule'] See the documentation for scipy.spatial.distance for details on these metrics. .. note:: `'kulsinski'` is deprecated from SciPy 1.9 and will be removed in SciPy 1.11. p : float, default=2 Parameter for the Minkowski metric from :class:`~sklearn.metrics.pairwise_distances`. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used. metric_params : dict, default=None Additional keyword arguments for the metric function. algorithm : {'auto', 'ball_tree', 'kd_tree', 'brute'}, default='auto' Algorithm used to compute the nearest neighbors: - 'ball_tree' will use :class:`~sklearn.neighbors.BallTree`. - 'kd_tree' will use :class:`~sklearn.neighbors.KDTree`. - 'brute' will use a brute-force search. - 'auto' will attempt to decide the most appropriate algorithm based on the values passed to `fit` method. (default) Note: fitting on sparse input will override the setting of this parameter, using brute force. leaf_size : int, default=30 Leaf size passed to :class:`~sklearn.neighbors.BallTree` or :class:`~sklearn.neighbors.KDTree`. This can affect the speed of the construction and query, as well as the memory required to store the tree. The optimal value depends on the nature of the problem. n_jobs : int, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- ordering_ : array of shape (n_samples,) The cluster ordered list of sample indices. core_distances_ : array of shape (n_samples,) Distance at which each sample becomes a core point, indexed by object order. Points which will never be core have a distance of inf. Use ``clust.core_distances_[clust.ordering_]`` to access in cluster order. reachability_ : array of shape (n_samples,) Reachability distances per sample, indexed by object order. Use ``clust.reachability_[clust.ordering_]`` to access in cluster order. predecessor_ : array of shape (n_samples,) Point that a sample was reached from, indexed by object order. Seed points have a predecessor of -1. References ---------- .. [1] Ankerst, Mihael, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. "OPTICS: ordering points to identify the clustering structure." ACM SIGMOD Record 28, no. 2 (1999): 49-60. Examples -------- >>> import numpy as np >>> from sklearn.cluster import compute_optics_graph >>> X = np.array([[1, 2], [2, 5], [3, 6], ... [8, 7], [8, 8], [7, 3]]) >>> ordering, core_distances, reachability, predecessor = compute_optics_graph( ... X, ... min_samples=2, ... max_eps=np.inf, ... metric="minkowski", ... p=2, ... metric_params=None, ... algorithm="auto", ... leaf_size=30, ... n_jobs=None, ... ) >>> ordering array([0, 1, 2, 5, 3, 4]) >>> core_distances array([3.16..., 1.41..., 1.41..., 1. , 1. , 4.12...]) >>> reachability array([ inf, 3.16..., 1.41..., 4.12..., 1. , 5. ]) >>> predecessor array([-1, 0, 1, 5, 3, 2]) rr*rrrFr~) n_neighborsr3r4r,r.r-r6N)rIrr*r}decimalsoutT) r\r]r^ point_index processedrInbrsr,r.r-r+z^All reachability values are inf. Set a larger max_eps or all data will be considered outliers.)rrxmaxintrqrrrrrrgraroundfinforF precisionzerosrQrangewhereargmin_set_reach_distallisinfrSrT UserWarning)rIr*r+r,r-r.r3r4r6rvr]r^rr\rrM ordering_idxindexpoints r?rZrZsiv I; =999a!Sy!899:: HY''Mrv8IS111Lb #    D HHQKKK/ tTO24OOg-.I//00: 4000Ix #...Hagaj)) a((+bi e 4556 %!& 5 !RV + +  /+)!#+     vbh}%%&&  D      _m\ AArAc J|||dz} || | dd} tjtj|| | } | jsdS|dkrN||g| f}t |tjrtj|}|}n\|tn| }|dkr d|vr| |d<t| || |fddi|}tj |||}tj |tj|jj| tj|tj|| k}|||| |<||| |<dS) NrF)radiusreturn_distancerr r9r-r6r)radius_neighborsrqcompresstakeru isinstancematrixasarrayravelrnrUr maximumrrrFrr)r\r]r^rrrIrr,r.r-r+Pindicesunprocdists_paramsrdistsimproveds r?rrs + a '(A##Agu#MMaPG["')W555w ? ?F ;;-'( eRY ' ' &Ju%%E )1$&&&}7I7I7K7K [ S%7%7GCL"1aiPPPPPVVXX Z{; < >> import numpy as np >>> from sklearn.cluster import cluster_optics_dbscan, compute_optics_graph >>> X = np.array([[1, 2], [2, 5], [3, 6], ... [8, 7], [8, 8], [7, 3]]) >>> ordering, core_distances, reachability, predecessor = compute_optics_graph( ... X, ... min_samples=2, ... max_eps=np.inf, ... metric="minkowski", ... p=2, ... metric_params=None, ... algorithm="auto", ... leaf_size=30, ... n_jobs=None, ... ) >>> eps = 4.5 >>> labels = cluster_optics_dbscan( ... reachability=reachability, ... core_distances=core_distances, ... ordering=ordering, ... eps=eps, ... ) >>> labels array([0, 0, 0, 1, 1, 1]) rrr~)lenrqrrcumsum)rKrPrMr0rvlabels far_reach near_cores r?rbrbsx@N##I Xis + + +Fs"I#%Iy8!4y7J!JKKaOF8%'F9 z !" MrAr#rJr:)r2r"r1c nt|}t||d|dkr tdt||z}||}t||d|dkr tdt||z}t |||||||||}t ||} | |fS)a Automatically extract clusters according to the Xi-steep method. Parameters ---------- reachability : ndarray of shape (n_samples,) Reachability distances calculated by OPTICS (`reachability_`). predecessor : ndarray of shape (n_samples,) Predecessors calculated by OPTICS. ordering : ndarray of shape (n_samples,) OPTICS ordered point indices (`ordering_`). min_samples : int > 1 or float between 0 and 1 The same as the min_samples given to OPTICS. Up and down steep regions can't have more then ``min_samples`` consecutive non-steep points. Expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2). min_cluster_size : int > 1 or float between 0 and 1, default=None Minimum number of samples in an OPTICS cluster, expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2). If ``None``, the value of ``min_samples`` is used instead. xi : float between 0 and 1, default=0.05 Determines the minimum steepness on the reachability plot that constitutes a cluster boundary. For example, an upwards point in the reachability plot is defined by the ratio from one point to its successor being at most 1-xi. predecessor_correction : bool, default=True Correct clusters based on the calculated predecessors. Returns ------- labels : ndarray of shape (n_samples,) The labels assigned to samples. Points which are not included in any cluster are labeled as -1. clusters : ndarray of shape (n_clusters, 2) The list of clusters in the form of ``[start, end]`` in each row, with all indices inclusive. The clusters are ordered according to ``(end, -start)`` (ascending) so that larger clusters encompassing smaller clusters come after such nested smaller clusters. Since ``labels`` does not reflect the hierarchy, usually ``len(clusters) > np.unique(labels)``. Examples -------- >>> import numpy as np >>> from sklearn.cluster import cluster_optics_xi, compute_optics_graph >>> X = np.array([[1, 2], [2, 5], [3, 6], ... [8, 7], [8, 8], [7, 3]]) >>> ordering, core_distances, reachability, predecessor = compute_optics_graph( ... X, ... min_samples=2, ... max_eps=np.inf, ... metric="minkowski", ... p=2, ... metric_params=None, ... algorithm="auto", ... leaf_size=30, ... n_jobs=None ... ) >>> min_samples = 2 >>> labels, clusters = cluster_optics_xi( ... reachability=reachability, ... predecessor=predecessor, ... ordering=ordering, ... min_samples=min_samples, ... ) >>> labels array([0, 0, 0, 1, 1, 1]) >>> clusters array([[0, 2], [3, 5], [0, 5]]) r*rrNr2)rrxrr _xi_cluster_extract_xi_labels) rKrLrMr*r2r"r1rvclustersrs r?r_r_sVL!!I; =999a!Sy!899:: &#Y0BCCC1q#&6&B"C"CDDXH H ( 3 3F 8 rAct|}d}|}|}||kr/||rd}|}n||s |dz }||krnn|S|dz }||k/|S)aQExtend the area until it's maximal. It's the same function for both upward and downward reagions, depending on the given input parameters. Assuming: - steep_{upward/downward}: bool array indicating whether a point is a steep {upward/downward}; - upward/downward: bool array indicating whether a point is upward/downward; To extend an upward reagion, ``steep_point=steep_upward`` and ``xward_point=downward`` are expected, and to extend a downward region, ``steep_point=steep_downward`` and ``xward_point=upward``. Parameters ---------- steep_point : ndarray of shape (n_samples,), dtype=bool True if the point is steep downward (upward). xward_point : ndarray of shape (n_samples,), dtype=bool True if the point is an upward (respectively downward) point. start : int The start of the xward region. min_samples : int The same as the min_samples given to OPTICS. Up and down steep regions can't have more then ``min_samples`` consecutive non-steep points. Returns ------- index : int The current index iterating over all the samples, i.e. where we are up to in our search. end : int The end of the region, which can be behind the index. The region includes the ``end`` index. rr)r) steep_point xward_pointstartr*rvnon_xward_pointsrends r?_extend_regionrsRK  I E C )   u   CCU#   !  +--.J   )   JrActjrgSfd|D}|D]}t|d|d<|S)zUpdate steep down areas (SDAs) using the new maximum in between (mib) value, and the given complement of xi, i.e. ``1 - xi``. c>g|]}|dzk|S)rrs).0sdamibreachability_plot xi_complements r? z'_update_filter_sdas..s9   s&7G &E &UUUUUUrAr)rqrr)sdasrrrresrs ``` r?_update_filter_sdasrs} x}}          C**US))E JrAc||krN||||kr||fS||}t||D]}|||kr||fcS|dz}||kNdS)aNCorrect for predecessors. Applies Algorithm 2 of [1]_. Input parameters are ordered by the computer OPTICS ordering. .. [1] Schubert, Erich, Michael Gertz. "Improving the Cluster Structure Extracted from OPTICS Plots." Proc. of the Conference "Lernen, Wissen, Daten, Analysen" (LWDA) (2018): 318-329. r)NN)r)rpredecessor_plotrMsep_eis r?_correct_predecessorrs a%% Q "3A"6 6 6a4Kq!q!  Ahqk!!!t " Q a%% :rAc tj|tjf}d|z }g}g} d} d} tjd5|dd|ddz } | |k} | d|z k}| dk}| dk}dddn #1swxYwYt tj| |zD]}|| kr t | tj|| |dz} ||rOt|| ||}|}t||||}||dd}| ||dz} || } t|| ||}|}t| |||}|dz} || } g}|D]*}|d }|}||dz|z|d kr&||d }||z||dzkrN||dz||dzkr5||d kr)|dz }||dz||dzkr ||d k)nA||dz|z|kr/||dz |kr ||kr|dz}||dz |kr||k|rt|||||\}}|||z dz|kr||d kr ||kr| ||f,| | |tj | S) aAutomatically extract clusters according to the Xi-steep method. This is rouphly an implementation of Figure 19 of the OPTICS paper. Parameters ---------- reachability_plot : array-like of shape (n_samples,) The reachability plot, i.e. reachability ordered according to the calculated ordering, all computed by OPTICS. predecessor_plot : array-like of shape (n_samples,) Predecessors ordered according to the calculated ordering. xi : float, between 0 and 1 Determines the minimum steepness on the reachability plot that constitutes a cluster boundary. For example, an upwards point in the reachability plot is defined by the ratio from one point to its successor being at most 1-xi. min_samples : int > 1 The same as the min_samples given to OPTICS. Up and down steep regions can't have more then ``min_samples`` consecutive non-steep points. min_cluster_size : int > 1 Minimum number of samples in an OPTICS cluster. predecessor_correction : bool Correct clusters based on the calculated predecessors. Returns ------- clusters : ndarray of shape (n_clusters, 2) The list of clusters in the form of [start, end] in each row, with all indices inclusive. The clusters are ordered in a way that larger clusters encompassing smaller clusters come after those smaller clusters. rrgrH)invalidNr~)rrrrrr)rqhstackrrerrstateiter flatnonzerorrrappendrreverseextendarray)rrrMr"r*r2r1rrrrrratio steep_upwardsteep_downwarddownwardupward steep_indexD_startD_endDU_startU_end U_clustersc_startc_endD_maxs r?rrsd #4bf"=>>FM DH E C X & & &!#2#&):122)>> - !m"3319 BN<.+HIIJJO(O(     #rv/ a0GHIIJJ + &F (&tS-ARSSD!G">67KPPE!%<>57?? ,EAI6>>57??*%9)+;XwPU&&NGU?7?Q&)999QuX%%7??!!7E"23333    OOJ ' ' ' ' 8H  s1BBBc(tjt|dt}d}|D]L}tj||d|ddzdks|||d|ddz<|dz }M|||<|S)aExtracts the labels from the clusters returned by `_xi_cluster`. We rely on the fact that clusters are stored with the smaller clusters coming before the larger ones. Parameters ---------- ordering : array-like of shape (n_samples,) The ordering of points calculated by OPTICS clusters : array-like of shape (n_clusters, 2) List of clusters i.e. (start, end) tuples, as returned by `_xi_cluster`. Returns ------- labels : ndarray of shape (n_samples,) r~rrr)rqfullrranyrU)rMrrlabelcs r?rrs&WS]]Bc 2 2 2F E vfQqTQqTAX./2566 (-F1Q41Q4!8$ % QJE{{}}F8 MrA)4rkrSnumbersrrnumpyrq scipy.sparserrbaserr r exceptionsr metricsr metrics.pairwiser rrrutilsrutils._chunkingrutils._param_validationrrrrrutils.validationrrrrxrndarrayrlrmrnrZrrbr_rrrrrrsrAr?rs~  """"""""::::::::<<<<<<<<<<......((((((IIIIIIII((((((......;:::::::eeeee\=eeeP    !!!Hj/ * HXq$v 6 6 6 HZAf 5 5 5 HT1d6:::;:cc.11]OCDDhOhtQW555t < j!J!J!JKKLhxD@@@AT"  #("RBRB#"RBj+1+1+1\ :,ZLq$v6667  #'>>>B  |ZL HXq$v 6 6 6 HZAf 5 5 5 HXq$v 6 6 6 HZAf 5 5 5  xa62223#,+ #'#2 llll'&l^<<<~   ,XXXvrA