1-Ph)fdZddlZGddeZGddeZdddd Zd d d d ZdS)aa fisher_vector.py - Implementation of the Fisher vector encoding algorithm This module contains the source code for Fisher vector computation. The computation is separated into two distinct steps, which are called separately by the user, namely: learn_gmm: Used to estimate the GMM for all vectors/descriptors computed for all examples in the dataset (e.g. estimated using all the SIFT vectors computed for all images in the dataset, or at least a subset of this). fisher_vector: Used to compute the Fisher vector representation for a single set of descriptors/vector (e.g. the SIFT descriptors for a single image in your dataset, or perhaps a test image). Reference: Perronnin, F. and Dance, C. Fisher kernels on Visual Vocabularies for Image Categorization, IEEE Conference on Computer Vision and Pattern Recognition, 2007 Origin Author: Dan Oneata (Author of the original implementation for the Fisher vector computation using scikit-learn and NumPy. Subsequently ported to scikit-image (here) by other authors.) NceZdZdS)FisherVectorExceptionN__name__ __module__ __qualname__^/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/skimage/feature/_fisher_vector.pyrrDr rceZdZdS)DescriptorExceptionNrr r r rr"r r r )n_modesgm_argsc ddlm}n#t$rtdwxYwttt jfstdd}ttr)t|t jstdttredj fdD}t|stdfdD}t|std t|tr|dkrtd |r#d |v}|d d k}|r|rtd ttrt j |rd |v}|r |dd|i|} n|d|d d|} n ||d } | | S)a Estimate a Gaussian mixture model (GMM) given a set of descriptors and number of modes (i.e. Gaussians). This function is essentially a wrapper around the scikit-learn implementation of GMM, namely the :class:`sklearn.mixture.GaussianMixture` class. Due to the nature of the Fisher vector, the only enforced parameter of the underlying scikit-learn class is the covariance_type, which must be 'diag'. There is no simple way to know what value to use for `n_modes` a-priori. Typically, the value is usually one of ``{16, 32, 64, 128}``. One may train a few GMMs and choose the one that maximises the log probability of the GMM, or choose `n_modes` such that the downstream classifier trained on the resultant Fisher vectors has maximal performance. Parameters ---------- descriptors : np.ndarray (N, M) or list [(N1, M), (N2, M), ...] List of NumPy arrays, or a single NumPy array, of the descriptors used to estimate the GMM. The reason a list of NumPy arrays is permissible is because often when using a Fisher vector encoding, descriptors/vectors are computed separately for each sample/image in the dataset, such as SIFT vectors for each image. If a list if passed in, then each element must be a NumPy array in which the number of rows may differ (e.g. different number of SIFT vector for each image), but the number of columns for each must be the same (i.e. the dimensionality must be the same). n_modes : int The number of modes/Gaussians to estimate during the GMM estimate. gm_args : dict Keyword arguments that can be passed into the underlying scikit-learn :class:`sklearn.mixture.GaussianMixture` class. Returns ------- gmm : :class:`sklearn.mixture.GaussianMixture` The estimated GMM object, which contains the necessary parameters needed to compute the Fisher vector. References ---------- .. [1] https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html Examples -------- .. testsetup:: >>> import pytest; _ = pytest.importorskip('sklearn') >>> from skimage.feature import fisher_vector >>> rng = np.random.Generator(np.random.PCG64()) >>> sift_for_images = [rng.standard_normal((10, 128)) for _ in range(10)] >>> num_modes = 16 >>> # Estimate 16-mode GMM with these synthetic SIFT vectors >>> gmm = learn_gmm(sift_for_images, n_modes=num_modes) rGaussianMixturemscikit-learn is not installed. Please ensure it is installed in order to use the Fisher vector functionality.zNPlease ensure descriptors are either a NumPy array, or a list of NumPy arrays.z5Please ensure descriptors are a list of NumPy arrays.cZg|]'}t|jtk(Sr )lenshape).0eexpected_shapes r zlearn_gmm..ts.JJJQW^!4!44JJJr zAPlease ensure all elements of your descriptor list are of rank 2.cTg|]$}|jddjdk%S)r)r)rr descriptorss r rzlearn_gmm..ys0KKK! k!n2155KKKr z=Please ensure all descriptors are of the same dimensionality.z,Please ensure n_modes is a positive integer.covariance_typediagzCovariance type must be "diag". n_components)r"r r )sklearn.mixturer ImportError isinstancelistnpndarrayrrallintrvstackfit) rrrrd_mat_1ranksdims has_cov_typecov_type_not_diaggmmrs ` @r learn_gmmr3&sp 3333333     <    kD"*#5 6 6 ! )   !nG+t$$ Z-L-L ! C   +t$$ $Q-JJJJkJJJ5zz %V LKKK{KKK4yy %O  gs # #Tw!||#$RSSSK(G3 #$56&@  K- K'(IJJ J+t$$-i ,,  L(G3  !/BBwB'BBCC!/$f@GCCo7FKKKGGK Js %Fg?)improvedalphac6 ddlm}n#t$rtdwxYwt|tjst dt||std|r$t|tstdt|}|j }|j }|j }| |} | ddj} | j||z } | jt j|d |z } | |z } | | |zz }| t j|d z}| |z}d | z|z}| |z |z|z}t j|}| |z} ||d d tjft j|zz}|t jd |d d tjfz|zz}t j| ||f}|r^t j|t jt j||z}|tj|z }|S) aCompute the Fisher vector given some descriptors/vectors, and an associated estimated GMM. Parameters ---------- descriptors : np.ndarray, shape=(n_descriptors, descriptor_length) NumPy array of the descriptors for which the Fisher vector representation is to be computed. gmm : :class:`sklearn.mixture.GaussianMixture` An estimated GMM object, which contains the necessary parameters needed to compute the Fisher vector. improved : bool, default=False Flag denoting whether to compute improved Fisher vectors or not. Improved Fisher vectors are L2 and power normalized. Power normalization is simply f(z) = sign(z) pow(abs(z), alpha) for some 0 <= alpha <= 1. alpha : float, default=0.5 The parameter for the power normalization step. Ignored if improved=False. Returns ------- fisher_vector : np.ndarray The computation Fisher vector, which is given by a concatenation of the gradients of a GMM with respect to its parameters (mixture weights, means, and covariance matrices). For D-dimensional input descriptors or vectors, and a K-mode GMM, the Fisher vector dimensionality will be 2KD + K. Thus, its dimensionality is invariant to the number of descriptors/vectors. References ---------- .. [1] Perronnin, F. and Dance, C. Fisher kernels on Visual Vocabularies for Image Categorization, IEEE Conference on Computer Vision and Pattern Recognition, 2007 .. [2] Perronnin, F. and Sanchez, J. and Mensink T. Improving the Fisher Kernel for Large-Scale Image Classification, ECCV, 2010 Examples -------- .. testsetup:: >>> import pytest; _ = pytest.importorskip('sklearn') >>> from skimage.feature import fisher_vector, learn_gmm >>> sift_for_images = [np.random.random((10, 128)) for _ in range(10)] >>> num_modes = 16 >>> # Estimate 16-mode GMM with these synthetic SIFT vectors >>> gmm = learn_gmm(sift_for_images, n_modes=num_modes) >>> test_image_descriptors = np.random.random((25, 128)) >>> # Compute the Fisher vector >>> fv = fisher_vector(test_image_descriptors, gmm) rrrz+Please ensure descriptors is a NumPy array.z>Please ensure gmm is a sklearn.mixture.GaussianMixture object.z2Please ensure that the alpha parameter is a float.T)axiskeepdimsN)r#rr$r%r'r(rrfloatrweights_means_ covariances_ predict_probameanTdotpowersqueezesqrtnewaxishstackravelsignabslinalgnorm)rr2r4r5rnum_descriptorsmixture_weightsmeans covariancesposterior_probabilitiespp_sumpp_xpp_x_2d_pid_mu d_sigma_t1 d_sigma_t2 d_sigma_t3d_sigmasqrt_mixture_weights fisher_vectors r r[r[sj 3333333     <    k2: . .Q!"OPPP c? + + # L    5%00 # @   +&&OlO JE"K!// <<% ) )q4 ) @ @ BF " $ ( ( 5 5 GD $ & * *28K+C+C D D VF >>  o -D &5. D"(5!,,,J+%JTE!Jg "Z/*r_s4     I        /   ')$rrrrrj16Snnnnnnnr