0Ph@x DdZddlZddlmZmZddlZddlmZm Z m Z ddl m Z ddl mZmZmZddlmZdd lmZmZdd lmZmZdd lmZmZdd lmZmZmZdd l m!Z!ddl"m#Z#dZ$ddddddZ%edddgiddddddddddd Z&Gd!d"eeZ'dS)#z"Algorithms for spectral clusteringN)IntegralReal) LinAlgErrorqrsvd) csc_matrix) BaseEstimator ClusterMixin _fit_context)_spectral_embedding) KERNEL_PARAMSpairwise_kernels)NearestNeighborskneighbors_graph)as_float_arraycheck_random_state)Interval StrOptionsvalidate_params) validate_data)k_meansc ^|jd}t|jd\}}}t||d|ddfj\}}}t t j|t j||}|dS)aFind the discrete partition closest to the eigenvector embedding. This implementation was proposed in [1]_. .. versionadded:: 1.1 Parameters ---------- vectors : array-like, shape: (n_samples, n_clusters) The embedding space of the samples. Returns ------- labels : array of integers, shape: n_samples The cluster labels of vectors. References ---------- .. [1] :doi:`Simple, direct, and efficient multi-way spectral clustering, 2019 Anil Damle, Victor Minden, Lexing Ying <10.1093/imaiai/iay008>` rT)pivotingNaxis) shaperTrabsnpdotconjargmax)vectorsk_pivutvs Y/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/sklearn/cluster/_spectral.py cluster_qrr,s2  aA79t,,,IAq#73rr7AAA:&())HB1"&"&QVVXX"6"67788G >>q> ! !!T)copymax_svd_restarts n_iter_max random_statec t|}t||}tjtj}|j\}}tj|}t|jdD]} |dd| ftj |dd| fz |z|dd| f<|d| fdkr3d|dd| fztj |d| fz|dd| f<|tj|dz dddtj fz }d} d} | |kr| stj||f} |||ddfj| dddf<tj|} td|D]c}| tjtj|| dd|dz fz } || ddfj| dd|f<dd }d}| s)|dz }tj|| }|d}t+tjt/|tjd||ff||f }|j|z} tj |\}}}n$#t4$r| dz } t7d YnawxYwd || z z}t#||z |ks||krd } n!|}tj|j|j} | )| |kr| | st5d|S)apSearch for a partition matrix which is closest to the eigenvector embedding. This implementation was proposed in [1]_. Parameters ---------- vectors : array-like of shape (n_samples, n_clusters) The embedding space of the samples. copy : bool, default=True Whether to copy vectors, or perform in-place normalization. max_svd_restarts : int, default=30 Maximum number of attempts to restart SVD if convergence fails n_iter_max : int, default=30 Maximum number of iterations to attempt in rotation and partition matrix search if machine precision convergence is not reached random_state : int, RandomState instance, default=None Determines random number generation for rotation matrix initialization. Use an int to make the randomness deterministic. See :term:`Glossary `. Returns ------- labels : array of integers, shape: n_samples The labels of the clusters. References ---------- .. [1] `Multiclass spectral clustering, 2003 Stella X. Yu, Jianbo Shi `_ Notes ----- The eigenvector embedding is used to iteratively search for the closest discrete partition. First, the eigenvector embedding is normalized to the space of partition matrices. An optimal discrete partition matrix closest to this normalized embedding multiplied by an initial rotation is calculated. Fixing this discrete partition matrix, an optimal rotation matrix is calculated. These two calculations are performed until convergence. The discrete partition matrix is returned as the clustering solution. Used in spectral clustering, this method tends to be faster and more robust to random initialization than k-means. )r0rNrr rF)rz2SVD did not converge, randomizing and trying againg@TzSVD did not converge)rrr!finfofloatepsrsqrtrangelinalgnormsignsumnewaxiszerosrandintrr r"argminr$roneslenarangerrprint)r%r0r1r2r3r9 n_samples n_components norm_onesi svd_restarts has_convergedrotationcjlast_objective_valuen_iter t_discretelabelsvectors_discretet_svdUSVh ncut_values r+ discretizer[7sn&l33LW4000G (5// C%mI| ""I 7=# $ $HH A1 )F)FF)S1 1a4=A  A.A1G1GGGAAAqDM ! 0 0a 0 8 899!!!RZ-HHGLM * * *M *8\<899 !5!5i!@!@!!!!CDFA HY  q,'' 6 6A wAE(:;;<< `. Parameters ---------- affinity : {array-like, sparse matrix} of shape (n_samples, n_samples) The affinity matrix describing the relationship of the samples to embed. **Must be symmetric**. Possible examples: - adjacency matrix of a graph, - heat kernel of the pairwise distance matrix of the samples, - symmetric k-nearest neighbours connectivity matrix of the samples. n_clusters : int, default=None Number of clusters to extract. n_components : int, default=n_clusters Number of eigenvectors to use for the spectral embedding. eigen_solver : {None, 'arpack', 'lobpcg', or 'amg'} The eigenvalue decomposition method. If None then ``'arpack'`` is used. See [4]_ for more details regarding ``'lobpcg'``. Eigensolver ``'amg'`` runs ``'lobpcg'`` with optional Algebraic MultiGrid preconditioning and requires pyamg to be installed. It can be faster on very large sparse problems [6]_ and [7]_. random_state : int, RandomState instance, default=None A pseudo random number generator used for the initialization of the lobpcg eigenvectors decomposition when `eigen_solver == 'amg'`, and for the K-Means initialization. Use an int to make the results deterministic across calls (See :term:`Glossary `). .. note:: When using `eigen_solver == 'amg'`, it is necessary to also fix the global numpy seed with `np.random.seed(int)` to get deterministic results. See https://github.com/pyamg/pyamg/issues/139 for further information. n_init : int, default=10 Number of time the k-means algorithm will be run with different centroid seeds. The final results will be the best output of n_init consecutive runs in terms of inertia. Only used if ``assign_labels='kmeans'``. eigen_tol : float, default="auto" Stopping criterion for eigendecomposition of the Laplacian matrix. If `eigen_tol="auto"` then the passed tolerance will depend on the `eigen_solver`: - If `eigen_solver="arpack"`, then `eigen_tol=0.0`; - If `eigen_solver="lobpcg"` or `eigen_solver="amg"`, then `eigen_tol=None` which configures the underlying `lobpcg` solver to automatically resolve the value according to their heuristics. See, :func:`scipy.sparse.linalg.lobpcg` for details. Note that when using `eigen_solver="lobpcg"` or `eigen_solver="amg"` values of `tol<1e-5` may lead to convergence issues and should be avoided. .. versionadded:: 1.2 Added 'auto' option. assign_labels : {'kmeans', 'discretize', 'cluster_qr'}, default='kmeans' The strategy to use to assign labels in the embedding space. There are three ways to assign labels after the Laplacian embedding. k-means can be applied and is a popular choice. But it can also be sensitive to initialization. Discretization is another approach which is less sensitive to random initialization [3]_. The cluster_qr method [5]_ directly extracts clusters from eigenvectors in spectral clustering. In contrast to k-means and discretization, cluster_qr has no tuning parameters and is not an iterative method, yet may outperform k-means and discretization in terms of both quality and speed. For a detailed comparison of clustering strategies, refer to the following example: :ref:`sphx_glr_auto_examples_cluster_plot_coin_segmentation.py`. .. versionchanged:: 1.1 Added new labeling method 'cluster_qr'. verbose : bool, default=False Verbosity mode. .. versionadded:: 0.24 Returns ------- labels : array of integers, shape: n_samples The labels of the clusters. Notes ----- The graph should contain only one connected component, elsewhere the results make little sense. This algorithm solves the normalized cut for `k=2`: it is a normalized spectral clustering. References ---------- .. [1] :doi:`Normalized cuts and image segmentation, 2000 Jianbo Shi, Jitendra Malik <10.1109/34.868688>` .. [2] :doi:`A Tutorial on Spectral Clustering, 2007 Ulrike von Luxburg <10.1007/s11222-007-9033-z>` .. [3] `Multiclass spectral clustering, 2003 Stella X. Yu, Jianbo Shi `_ .. [4] :doi:`Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method, 2001 A. V. Knyazev SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541. <10.1137/S1064827500366124>` .. [5] :doi:`Simple, direct, and efficient multi-way spectral clustering, 2019 Anil Damle, Victor Minden, Lexing Ying <10.1093/imaiai/iay008>` .. [6] :doi:`Multiscale Spectral Image Segmentation Multiscale preconditioning for computing eigenvalues of graph Laplacians in image segmentation, 2006 Andrew Knyazev <10.13140/RG.2.2.35280.02565>` .. [7] :doi:`Preconditioned spectral clustering for stochastic block partition streaming graph challenge (Preliminary version at arXiv.) David Zhuzhunashvili, Andrew Knyazev <10.1109/HPEC.2017.8091045>` Examples -------- >>> import numpy as np >>> from sklearn.metrics.pairwise import pairwise_kernels >>> from sklearn.cluster import spectral_clustering >>> X = np.array([[1, 1], [2, 1], [1, 0], ... [4, 7], [3, 5], [3, 6]]) >>> affinity = pairwise_kernels(X, metric='rbf') >>> spectral_clustering( ... affinity=affinity, n_clusters=2, assign_labels="discretize", random_state=0 ... ) array([1, 1, 1, 0, 0, 0]) precomputed) rcrIrdr3rer\rfrgrh)SpectralClusteringfitlabels_) r\rcrIrdr3rerfrgrh clusterers r+spectral_clusteringrosO\#!!!#    c(mm  r-ceZdZUdZeedddgehddgeeddddgdgeedddgeedddgeee e hd zgeedddgeed dded hgehd geedddgeeddd ge dgedgdgdZ e e d< ddddddddd ddddddddZedd dZd fd ZfdZxZS)!rka$Apply clustering to a projection of the normalized Laplacian. In practice Spectral Clustering is very useful when the structure of the individual clusters is highly non-convex, or more generally when a measure of the center and spread of the cluster is not a suitable description of the complete cluster, such as when clusters are nested circles on the 2D plane. If the affinity matrix is the adjacency matrix of a graph, this method can be used to find normalized graph cuts [1]_, [2]_. When calling ``fit``, an affinity matrix is constructed using either a kernel function such the Gaussian (aka RBF) kernel with Euclidean distance ``d(X, X)``:: np.exp(-gamma * d(X,X) ** 2) or a k-nearest neighbors connectivity matrix. Alternatively, a user-provided affinity matrix can be specified by setting ``affinity='precomputed'``. Read more in the :ref:`User Guide `. Parameters ---------- n_clusters : int, default=8 The dimension of the projection subspace. eigen_solver : {'arpack', 'lobpcg', 'amg'}, default=None The eigenvalue decomposition strategy to use. AMG requires pyamg to be installed. It can be faster on very large, sparse problems, but may also lead to instabilities. If None, then ``'arpack'`` is used. See [4]_ for more details regarding `'lobpcg'`. n_components : int, default=None Number of eigenvectors to use for the spectral embedding. If None, defaults to `n_clusters`. random_state : int, RandomState instance, default=None A pseudo random number generator used for the initialization of the lobpcg eigenvectors decomposition when `eigen_solver == 'amg'`, and for the K-Means initialization. Use an int to make the results deterministic across calls (See :term:`Glossary `). .. note:: When using `eigen_solver == 'amg'`, it is necessary to also fix the global numpy seed with `np.random.seed(int)` to get deterministic results. See https://github.com/pyamg/pyamg/issues/139 for further information. n_init : int, default=10 Number of time the k-means algorithm will be run with different centroid seeds. The final results will be the best output of n_init consecutive runs in terms of inertia. Only used if ``assign_labels='kmeans'``. gamma : float, default=1.0 Kernel coefficient for rbf, poly, sigmoid, laplacian and chi2 kernels. Ignored for ``affinity='nearest_neighbors'``, ``affinity='precomputed'`` or ``affinity='precomputed_nearest_neighbors'``. affinity : str or callable, default='rbf' How to construct the affinity matrix. - 'nearest_neighbors': construct the affinity matrix by computing a graph of nearest neighbors. - 'rbf': construct the affinity matrix using a radial basis function (RBF) kernel. - 'precomputed': interpret ``X`` as a precomputed affinity matrix, where larger values indicate greater similarity between instances. - 'precomputed_nearest_neighbors': interpret ``X`` as a sparse graph of precomputed distances, and construct a binary affinity matrix from the ``n_neighbors`` nearest neighbors of each instance. - one of the kernels supported by :func:`~sklearn.metrics.pairwise.pairwise_kernels`. Only kernels that produce similarity scores (non-negative values that increase with similarity) should be used. This property is not checked by the clustering algorithm. n_neighbors : int, default=10 Number of neighbors to use when constructing the affinity matrix using the nearest neighbors method. Ignored for ``affinity='rbf'``. eigen_tol : float, default="auto" Stopping criterion for eigen decomposition of the Laplacian matrix. If `eigen_tol="auto"` then the passed tolerance will depend on the `eigen_solver`: - If `eigen_solver="arpack"`, then `eigen_tol=0.0`; - If `eigen_solver="lobpcg"` or `eigen_solver="amg"`, then `eigen_tol=None` which configures the underlying `lobpcg` solver to automatically resolve the value according to their heuristics. See, :func:`scipy.sparse.linalg.lobpcg` for details. Note that when using `eigen_solver="lobpcg"` or `eigen_solver="amg"` values of `tol<1e-5` may lead to convergence issues and should be avoided. .. versionadded:: 1.2 Added 'auto' option. assign_labels : {'kmeans', 'discretize', 'cluster_qr'}, default='kmeans' The strategy for assigning labels in the embedding space. There are two ways to assign labels after the Laplacian embedding. k-means is a popular choice, but it can be sensitive to initialization. Discretization is another approach which is less sensitive to random initialization [3]_. The cluster_qr method [5]_ directly extract clusters from eigenvectors in spectral clustering. In contrast to k-means and discretization, cluster_qr has no tuning parameters and runs no iterations, yet may outperform k-means and discretization in terms of both quality and speed. .. versionchanged:: 1.1 Added new labeling method 'cluster_qr'. degree : float, default=3 Degree of the polynomial kernel. Ignored by other kernels. coef0 : float, default=1 Zero coefficient for polynomial and sigmoid kernels. Ignored by other kernels. kernel_params : dict of str to any, default=None Parameters (keyword arguments) and values for kernel passed as callable object. Ignored by other kernels. n_jobs : int, default=None The number of parallel jobs to run when `affinity='nearest_neighbors'` or `affinity='precomputed_nearest_neighbors'`. The neighbors search will be done in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. verbose : bool, default=False Verbosity mode. .. versionadded:: 0.24 Attributes ---------- affinity_matrix_ : array-like of shape (n_samples, n_samples) Affinity matrix used for clustering. Available only after calling ``fit``. labels_ : ndarray of shape (n_samples,) Labels of each point n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- sklearn.cluster.KMeans : K-Means clustering. sklearn.cluster.DBSCAN : Density-Based Spatial Clustering of Applications with Noise. Notes ----- A distance matrix for which 0 indicates identical elements and high values indicate very dissimilar elements can be transformed into an affinity / similarity matrix that is well-suited for the algorithm by applying the Gaussian (aka RBF, heat) kernel:: np.exp(- dist_matrix ** 2 / (2. * delta ** 2)) where ``delta`` is a free parameter representing the width of the Gaussian kernel. An alternative is to take a symmetric version of the k-nearest neighbors connectivity matrix of the points. If the pyamg package is installed, it is used: this greatly speeds up computation. References ---------- .. [1] :doi:`Normalized cuts and image segmentation, 2000 Jianbo Shi, Jitendra Malik <10.1109/34.868688>` .. [2] :doi:`A Tutorial on Spectral Clustering, 2007 Ulrike von Luxburg <10.1007/s11222-007-9033-z>` .. [3] `Multiclass spectral clustering, 2003 Stella X. Yu, Jianbo Shi `_ .. [4] :doi:`Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method, 2001 A. V. Knyazev SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541. <10.1137/S1064827500366124>` .. [5] :doi:`Simple, direct, and efficient multi-way spectral clustering, 2019 Anil Damle, Victor Minden, Lexing Ying <10.1093/imaiai/iay008>` Examples -------- >>> from sklearn.cluster import SpectralClustering >>> import numpy as np >>> X = np.array([[1, 1], [2, 1], [1, 0], ... [4, 7], [3, 5], [3, 6]]) >>> clustering = SpectralClustering(n_clusters=2, ... assign_labels='discretize', ... random_state=0).fit(X) >>> clustering.labels_ array([1, 1, 1, 0, 0, 0]) >>> clustering SpectralClustering(assign_labels='discretize', n_clusters=2, random_state=0) rNleft)closed>amgarpacklobpcgr3r>rjnearest_neighborsprecomputed_nearest_neighborsr6ra>rbr,r[neitherrhrcrdrIr3regammar\ n_neighborsrfrgdegreecoef0 kernel_paramsn_jobsrh_parameter_constraintsr_r`g?rbfrbF)rdrIr3rerzr\r{rfrgr|r}r~rrhc||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ | |_ ||_ ||_dSNry)selfrcrdrIr3rerzr\r{rfrgr|r}r~rrhs r+__init__zSpectralClustering.__init__ysw&%(((    &"*  *  r-Tr]ct||gdtjd}|jdv}|jd|jdkr|st jd|jdkr0t||jd |j }d ||j zz|_ n|jd krYt|j|j d  |}||d}d ||j zz|_ nj|jd kr||_ nW|j}|i}t|js|j|d<|j|d<|j|d<t'|f|jd d||_ t)|j}|j|jn|j}t1|j ||j||jd} |jrt9d|j|jdkr-t=| |j||j|j\} |_ } n6|jdkrtC| |_ ntE| ||_ |S)a@Perform spectral clustering from features, or affinity matrix. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) or (n_samples, n_samples) Training instances to cluster, similarities / affinities between instances if ``affinity='precomputed'``, or distances between instances if ``affinity='precomputed_nearest_neighbors``. If a sparse matrix is provided in a format other than ``csr_matrix``, ``csc_matrix``, or ``coo_matrix``, it will be converted into a sparse ``csr_matrix``. y : Ignored Not used, present here for API consistency by convention. Returns ------- self : object A fitted instance of the estimator. )csrcsccoor ) accept_sparsedtypeensure_min_samplesrjrwrrzThe spectral clustering API has changed. ``fit``now constructs an affinity matrix from data. To use a custom affinity matrix, set ``affinity=precomputed``.rvT)r{ include_selfrg?rwrj)r{rmetric connectivity)XmodeNrzr|r})r filter_paramsF)rIrdr3rf drop_firstz!Computing label assignment using rb)r3rerhr,)r3)#rr!float64r\rwarningswarnrr{rraffinity_matrix_rrlr~callablerzr|r}rrr3rIrcr rdrfrhrGrgrrermr,r[) rry allow_squaredr estimatorparamsr3rImapsr's r+rlzSpectralClustering.fits.   ///*     *   71: # #M # M0    =/ / /+t/d4;L%(<,.+H$ID ! ! ]= = =( ,T[c!ff %555OOL$'<,.+H$ID ! ! ]m + +$%D ! !'F~DM** -"&*w#';x "&*w$4%-t%%?E%%D !*$*;<< #08DOOd>O #  !%*%n     < L Jd6HJJ K K K   ) )!(){ """ At|QQ < / /%d++DLL%dFFFDL r-cHt||S)aDPerform spectral clustering on `X` and return cluster labels. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) or (n_samples, n_samples) Training instances to cluster, similarities / affinities between instances if ``affinity='precomputed'``, or distances between instances if ``affinity='precomputed_nearest_neighbors``. If a sparse matrix is provided in a format other than ``csr_matrix``, ``csc_matrix``, or ``coo_matrix``, it will be converted into a sparse ``csr_matrix``. y : Ignored Not used, present here for API consistency by convention. Returns ------- labels : ndarray of shape (n_samples,) Cluster labels. )super fit_predict)rrr __class__s r+rzSpectralClustering.fit_predicts,ww""1a(((r-ct}d|j_|jdv|j_|S)NTr)r__sklearn_tags__ input_tagssparser\pairwise)rtagsrs r+rz#SpectralClustering.__sklearn_tags__s?ww''))!%#'=5 $   r-)r_r)__name__ __module__ __qualname____doc__rrrrrsetrdictr__annotations__rr rlrr __classcell__)rs@r+rkrk{sd``F x!T&AAAB#$?$?$?@@$G!(AtFCCCTJ'(8Haf===>(4D8889  JM""WWWX   !1d6BBBC HT3V 4 4 4 Jx  %*%K%K%KLLM8D!T&999:(4tI>>>?T";1$$D:!#!!!!!F\555ddd65dL))))))0r-rk)(rrnumbersrrnumpyr! scipy.linalgrrr scipy.sparserbaser r r manifold._spectral_embeddingr metrics.pairwiserr neighborsrrutilsrrutils._param_validationrrrutils.validationr_kmeansrr,r[rorkr-r+rs(( """"""""----------######<<<<<<<<<<>>>>>>>>>>>>>>::::::::66666666KKKKKKKKKK,,,,,,"""BB2DDDDDDN,01"'  vvvv vrggggg}gggggr-