M/Ph NdZddlZddlmZddlmZddlmcm Zddl m Z ddl m Z ddlZddlZddlZddgdd gd d gd d gd dgd dgddgddgddgddgg ZdZdZdZGddejZGddZGddZGdd eeZGd!d"eeZdS)#a Bayesian inference for generalized linear mixed models. Currently only families without additional scale or shape parameters are supported (binomial and Poisson). Two estimation approaches are supported: Laplace approximation ('maximum a posteriori'), and variational Bayes (mean field approximation to the posterior distribution). All realizations of random effects are modeled to be mutually independent in this implementation. The `exog_vc` matrix is the design matrix for the random effects. Every column of `exog_vc` corresponds to an independent realization of a random effect. These random effects have mean zero and an unknown standard deviation. The standard deviation parameters are constrained to be equal within subsets of the columns. When not using formulas, these subsets are specified through the parameter `ident`. `ident` must have the same length as the number of columns of `exog_vc`, and two columns whose `ident` values are equal have the same standard deviation. When formulas are used, the columns of `exog_vc` derived from a common formula are constrained to have the same standard deviation. In many applications, `exog_vc` will be sparse. A sparse matrix may be passed when constructing a model class. If a dense matrix is passed, it will be converted internally to a sparse matrix. There currently is no way to avoid creating a temporary dense version of `exog_vc` when using formulas. Model and parameterization -------------------------- The joint density of data and parameters factors as: .. math:: p(y | vc, fep) p(vc | vcp) p(vcp) p(fe) The terms :math:`p(vcp)` and :math:`p(fe)` are prior distributions that are taken to be Gaussian (the :math:`vcp` parameters are log standard deviations so the standard deviations have log-normal distributions). The random effects distribution :math:`p(vc | vcp)` is independent Gaussian (random effect realizations are independent within and between values of the `ident` array). The model :math:`p(y | vc, fep)` depends on the specific GLM being fit. N)minimize)sparse)summary2)familiesgp?gyxPÿgyxP?gUz;?gj ۿgj ?g0 ?g"g"?gx8!?g^g^?gb_?g*>*g*>*?ad Generalized Linear Mixed Model with Bayesian estimation The class implements the Laplace approximation to the posterior distribution (`fit_map`) and a variational Bayes approximation to the posterior (`fit_vb`). See the two fit method docstrings for more information about the fitting approaches. Parameters ---------- endog : array_like Vector of response values. exog : array_like Array of covariates for the fixed effects part of the mean structure. exog_vc : array_like Array of covariates for the random part of the model. A scipy.sparse array may be provided, or else the passed array will be converted to sparse internally. ident : array_like Array of integer labels showing which random terms (columns of `exog_vc`) have a common variance. vcp_p : float Prior standard deviation for variance component parameters (the prior standard deviation of log(s) is vcp_p, where s is the standard deviation of a random effect). fe_p : float Prior standard deviation for fixed effects parameters. family : statsmodels.genmod.families instance The GLM family. fep_names : list[str] The names of the fixed effects parameters (corresponding to columns of exog). If None, default names are constructed. vcp_names : list[str] The names of the variance component parameters (corresponding to distinct labels in ident). If None, default names are constructed. vc_names : list[str] The names of the random effect realizations. Returns ------- MixedGLMResults object Notes ----- There are three types of values in the posterior distribution: fixed effects parameters (fep), corresponding to the columns of `exog`, random effects realizations (vc), corresponding to the columns of `exog_vc`, and the standard deviations of the random effects realizations (vcp), corresponding to the unique integer labels in `ident`. All random effects are modeled as being independent Gaussian values (given the variance structure parameters). Every column of `exog_vc` has a distinct realized random effect that is used to form the linear predictors. The elements of `ident` determine the distinct variance structure parameters. Two random effect realizations that have the same value in `ident` have the same variance. When fitting with a formula, `ident` is constructed internally (each element of `vc_formulas` yields a distinct label in `ident`). The random effect standard deviation parameters (`vcp`) have log-normal prior distributions with mean 0 and standard deviation `vcp_p`. Note that for some families, e.g. Binomial, the posterior mode may be difficult to find numerically if `vcp_p` is set to too large of a value. Setting `vcp_p` to 0.5 seems to work well. The prior for the fixed effects parameters is Gaussian with mean 0 and standard deviation `fe_p`. It is recommended that quantitative covariates be standardized. Examples --------{example} References ---------- Introduction to generalized linear mixed models: https://stats.idre.ucla.edu/other/mult-pkg/introduction-to-generalized-linear-mixed-models SAS documentation: https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_intromix_a0000000215.htm An assessment of estimation methods for generalized linear mixed models with binary outcomes https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3866838/ a\ A binomial (logistic) random effects model with random intercepts for villages and random slopes for each year within each village: >>> random = {"a": '0 + C(Village)', "b": '0 + C(Village)*year_cen'} >>> model = BinomialBayesMixedGLM.from_formula( 'y ~ year_cen', random, data) >>> result = model.fit_vb() aP A Poisson random effects model with random intercepts for villages and random slopes for each year within each village: >>> random = {"a": '0 + C(Village)', "b": '0 + C(Village)*year_cen'} >>> model = PoissonBayesMixedGLM.from_formula( 'y ~ year_cen', random, data) >>> result = model.fit_vb() ceZdZ dfd ZdZdZdZdZe dfd Z dd Z dd Z ddZ xZ S)_BayesMixedGLMNc  |jdkr;t|tjr |dddf}nt j|}|jdkrd} t | |jdkr;t|tjr |dddf}nt j|}|jdkrd} t | tj|}|jdkrd} t | t||j dkrd} t | tj |j tj sd} t | |Nt|dr|j}n$d t!|j dD}| 7d t!t#t%|dzD} n>t| tt'|krd } t | t)j|st)j|}|t"}t1|}t1|}|d } n |j d} |d }d }n|j d}t%|dz}||}t5j||fi| ||_||_||_||_| |_ ||_!||_"||_#| |_$| |_%||_&||_'|| z|_(| |xj(| z c_(dSdS) Nr r z#'exog' must have one or two columnsz&'exog_vc' must have one or two columnsz%ident must be a one-dimensional arrayz8len(ident) should match the number of columns of exog_vcz ident must have an integer dtypecolumnscg|] }d|dzz S)zFE_%dr .0ks b/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/genmod/bayes_mixed_glm.py z+_BayesMixedGLM.__init__..s!MMM1WA.MMMcg|] }d|dzz S)zVC_%dr rrs rrz+_BayesMixedGLM.__init__..s!OOOqAE*OOOrz5The lengths of vcp_names and ident should be the samer))ndim isinstancenpndarraypd DataFrame ValueErrorasarraylenshape issubdtypedtypeintegerhasattrr tolistrangeintmaxsetrissparse csr_matrixastypefloatmultiplysuper__init__exog_vcexog_vc2identfamilyk_fepk_vck_vcp fep_names vcp_namesvc_namesfe_pvcp_pnames)selfendogexogr0r2r3r;r:r7r8r9kwargsmsgr4r5r6r1 __class__s rr/z_BayesMixedGLM.__init__s> 9>>$ ++ *AAAtG}|D)) 9>>7CS// ! <1  '2:.. 0!!!!T'*,w// <1  :CS// ! 5!! :??9CS// ! u::q) ) )LCS// !}U["*55 "4CS// !  tY'' N L//11 MMdjm8L8LMMM   OOE#c%jj//A:M4N4NOOOII9~~SZZ00M oo%w'' 1'00G S!!e T{{ <EEJqME ?DEE=#DJJNE##G,,/////        ""    *   JJ( "JJJJ rcd}|d||jz}||jz }||||jz}||jz }||d}|||fS)Nr)r4r6)r=veciifepvcpvcs r_unpackz_BayesMixedGLM._unpack'si "2 ?"# dj "R$*_$% djXC|rc||\}}}d}|jdkr|tj|j|z }|jdkr||j|z }|jj |}|j |j |}|jdkr||j }tj |} |dtj|dz| dzz ztj|zz}|dtj|dz|jdzz zz}|jdkr(|dtj|dz|jdzz zz}|S)z The overall log-density: log p(y, fe, vc, vcp). This differs by an additive constant from the log posterior log p(fe, vc, vcp | y). r?r )rIr4rdotr?r5r0r3linkinverselogliker>r2expsumr;r:) r=paramsrFrGrHlpmullvcp0ss r logposteriorz_BayesMixedGLM.logposterior:sV||F++ S" :>> "&C(( (B 9q== $,""2&& &B [  % %b ) ) [ R 0 0 9q==tz?Dt A #r1uq!t|,,,rvd||; ;B #sAv A 5666 6B :>> #sAv 1 4555 5B rc||\}}}d}|jdkr|tj|j|z }|jdkr||j|z }|jj |}|j |z |jj |z }||j |z}gd}|jdkrtj||j|d<|jdkr/|j ||d<|jdkr~||j} tj| } |dz| dzz dz } tj|j| |d<|dxx|| dzz zcc<|dxx||jdzz zcc<|jdkr|dxx||jdzz zcc<d|D}tj|S)z4 The gradient of the log posterior. r)NNNr r weightscg|]}||SNrrxs rrz4_BayesMixedGLM.logposterior_grad..s - - -Aq}a}}}r)rIr4rrLr?r5r0r3rMrNr>derivvariance transposer2rPbincountr;r: concatenate) r=rRrFrGrHrSrT score_factorterVrWus rlogposterior_gradz _BayesMixedGLM.logposterior_grad^s ||F++ S"  :>> "&C(( (B 9q== $,""2&& &B [  % %b ) ) R4;+;+A+A"+E+EE  ,,R000    :>>F<33BqE 9q==L**,,00>>BqE 9q== tz?Dt AA1 q AK A666BqE qEEER!Q$Y EEE qEEES4:q=( (EEE :>> qEEES49a<' 'EEE - - - - -~b!!!rctj|j}tj|j}tj|j}tj|||f}|S)Nsize) rzerosr4onesr6randomnormalr5rd)r= start_fep start_vcpstart_vcstarts r _get_startz_BayesMixedGLM._get_startsYHTZ(( GDJ'' 9###33 9h?@@ rc 6g}g}g} d} |D]\} } tj| |d} || | | || t j| jdtjz| dz } tj |d}|j }t j |}t|||d|||| || }|S) a Fit a BayesMixedGLM using a formula. Parameters ---------- formula : str Formula for the endog and fixed effects terms (use ~ to separate dependent and independent expressions). vc_formulas : dictionary vc_formulas[name] is a one-sided formula that creates one collection of random effects with a common variance parameter. If using categorical (factor) variables to produce variance components, note that generally `0 + ...` should be used so that an intercept is not included. data : data frame The data to which the formulas are applied. family : genmod.families instance A GLM family. vcp_p : float The prior standard deviation for the logarithms of the standard deviations of the random effects. fe_p : float The prior standard deviation for the fixed effects parameters. r dataframe) return_typer )r!)axisN) datar3subsetr0r2r9r8r:r;)itemspatsydmatrixappendrrmrint_rconcatr r$rdr. from_formula)clsformula vc_formulasryr3r;r:r2r0r8jnafmlmatr9modelrBs rrz_BayesMixedGLM.from_formulas'B "((**  GB-T{CCCC NN3      R LLRWSYq\AAAA B B B FAA)G!,,,?))++u%%$$ %   rBFGSc2|||dS)z fit is equivalent to fit_map. See fit_map for parameter information. Use `fit_vb` to fit the model using variational Bayes. N)fit_map)r=method minim_optss rfitz_BayesMixedGLM.fits VZ(((((rFc|rjd}jd}j_j_t j|dk}jdd|fxx||zcc<jdd|fxx||zcc<fd}fd}} t|| |||} | j sEdt j t j | j dzz} tj| dd lm} | | j|} t j| }| j}|raj_`||xx||zcc<||ddfdd|fxxt j||||zcc<t+||| S) a~ Construct the Laplace approximation to the posterior distribution. Parameters ---------- method : str Optimization method for finding the posterior mode. minim_opts : dict Options passed to scipy.minimize. scale_fe : bool If True, the columns of the fixed effects design matrix are centered and scaled to unit variance before fitting the model. The results are back-transformed so that the results are presented on the original scale. Returns ------- BayesMixedGLMResults instance. r:0yE>Nc0| Sr])rXrRr=s rfunz#_BayesMixedGLM.fit_map..funs%%f--- -rc0| Sr])rhrs rgradz$_BayesMixedGLM.fit_map..grads**6222 2r)rjacoptionsz1Laplace fitting did not converge, |gradient|=%.6fr ) approx_fprime) optim_retvals)r?meanstd _exog_savecopyr flatnonzerortrsuccesssqrtrQrwarningswarnstatsmodels.tools.numdiffrr_linalginvouterBayesMixedGLMResults)r=rrscale_femnscixsrrrsrrArhesscovrRs` rrz_BayesMixedGLM.fit_maps*  )""Bq!!B"iDO ((DI.d++C Iaaaf   C (    Iaaaf   C (    . . . . . 3 3 3 3 3!! S%D* M M My F726!%(++,,-C M#   ;;;;;;}QS$''immD!!  >DI 3KKK2c7 "KKK QQQK3   28BsGRW#=#= =   #D&#QGGGGrc||j}|jd}tj||d|}|s|jj|}|S)aX Return the fitted mean structure. Parameters ---------- params : array_like The parameter vector, may be the full parameter vector, or may be truncated to include only the mean parameters. exog : array_like The design matrix for the mean structure. If omitted, use the model's design matrix. linear : bool If True, return the linear predictor without passing through the link function. Returns ------- A 1-dimensional array of predicted values Nr r)r?rrrLr3rMrN)r=rRr?linearqprs rpredictz_BayesMixedGLM.predictsY* <9D JqM VD&1+ & & .!))"--B r)NNNr r NNN)Nr r )rN)rNFNF)__name__ __module__ __qualname__r/rIrXrhrt classmethodrrrr __classcell__rBs@rrrsf#f#f#f#f#f#P&"""H/"/"/"b ! ;;;;;[;z))))8H8H8H8HtrrcLeZdZdZdZdZdZdZdZ d d Z d Z d Z dS) _VariationalBayesMixedGLMzo A mixin providing generic (not family-specific) methods for variational Bayes mean field fitting. Fctj|j|}tj|jdz|dz}||j|z }||j|dzz }||fSNr )rrLr?r0r1)r=fep_meanfep_sdvc_meanvc_sdtmtvs r _lp_statsz#_VariationalBayesMixedGLM._lp_statsHsq VDIx ( ( VDIqL&!) , , dlw''' dmq)))2v rc jd} tD]E} |j| dz} | | d|| ztj| dz dz zz } F| tjdtjzz} | |jz} | |j|zz } | } | |||||||z } | tjtj |ztjtj |ztjtj |z} | S)a Returns the evidence lower bound (ELBO) for the model. This function calculates the family-specific ELBO function based on information provided from a subclass. Parameters ---------- h : function mapping 1d vector to 1d vector The contribution of the model to the ELBO function can be expressed as y_i*lp_i + Eh_i(z), where y_i and lp_i are the response and linear predictor for observation i, and z is a standard normal random variable. This formulation can be achieved for any GLM with a canonical link function. rr r ) glwrngrrPrpir>rQ _elbo_commonlog) r=hrrvcp_meanrrvcp_sdrivwzrs r vb_elbo_basez&_VariationalBayesMixedGLM.vb_elbo_baseQs%( 2 2A1Q4A !A$1+1uqy 1 11 1BB bga"%i    dh dj2o VVXX d&(FG %'' '"&(( (26"&..+A+A ABF F5MMEErc d} d} d} d} d}d}tD]v}|j|dz}||tj|dz dz ztjdtjzz }|tj|z }| |dtj||jzz } ||d|j |zz }| |d|ztj||jdz|zzz } |j |  |}tj tj |}||d|z|zz }x| |jz} ||jz}| |jz} ||jz}| tj|j|jz } ||j |jz }||||||| \}}}}}}| |z } | |z } | |z } | |z } ||z }||z }| d|z z } | d|z z } |d| z z }tj| | |f}tj| | |f}|jrQt%dtjtj|dztj|dzzz||fS)ze Return the gradient of the ELBO function. See vb_elbo_base for parameters. gr r rz|G|=%f)rrrrPrrrLr?r0rbr1r-squeezerr>_elbo_grad_commonrdverboseprintrQ)r=rrrrrrrrr fep_mean_grad fep_sd_grad vcp_mean_grad vcp_sd_grad vc_mean_grad vc_sd_gradrrrgrvfep_mean_grad_i fep_sd_grad_ivcp_mean_grad_i vcp_sd_grad_ivc_mean_grad_i vc_sd_grad_i mean_gradsd_grads rvb_elbo_grad_basez+_VariationalBayesMixedGLM.vb_elbo_grad_basews        ' 'A1Q4A!rvq!teai(((271ru9+=+==ABGBKKA QqTBF1di$8$88 8M AaD4<#9#9#;#;#?#?#B#BB BL 1Q4!8bfQ 1 v0E&F&FF FK &&u--7799==a@@A 2:a==))A !A$(Q, &JJ!  tx dh  DI666  ..0044TZ@@@ *.)?)? vx%*A*A '--  ( }$ ( }$ & l" q6z! q6z! a%i NM=$0$233 .+{J!GHH < O 2726)Q,#7#7"&!:L:L#LMMM O O O'!!rNrc |_|rjd}jd}j_j_t j|dk} jdd| fxx|| zcc<jdd| fxx|| zcc<jj zj z} jj zj z} |t j | } nHt|| kr!tdt|| fz|} |'ddt j| zz} nHt|| kr!tdt|| fzt j|} jjj z}}t j| ||d kd | ||| ||<t j| d kd | } fd }fd }t j| | f}t'||||| }|jst+jd t|jdz} |jd| }t jd|j| dz}|r=j_`|| xx|| zcc<|| xx|| dzzcc<t3|||S)a Fit a model using the variational Bayes mean field approximation. Parameters ---------- mean : array_like Starting value for VB mean vector sd : array_like Starting value for VB standard deviation vector fit_method : str Algorithm for scipy.minimize minim_opts : dict Options passed to scipy.minimize scale_fe : bool If true, the columns of the fixed effects design matrix are centered and scaled to unit variance before fitting the model. The results are back-transformed so that the results are presented on the original scale. verbose : bool If True, print the gradient norm to the screen each time it is calculated. Notes ----- The goal is to find a factored Gaussian approximation q1*q2*... to the posterior distribution, approximately minimizing the KL divergence from the factored approximation to the actual posterior. The KL divergence, or ELBO function has the form E* log p(y, fe, vcp, vc) - E* log q where E* is expectation with respect to the product of qj. References ---------- Blei, Kucukelbir, McAuliffe (2017). Variational Inference: A review for Statisticians https://arxiv.org/pdf/1601.00670.pdf rrNz#mean has incorrect length, %d != %dgg?rjz!sd has incorrect length, %d != %dct|dz}|d|tj||d Sr)rvb_elborrP)r_nr=s relboz._VariationalBayesMixedGLM.fit_vb..elbo sAA! ALL2A2qu 666 6rc t|dz}|d|tj||d\}}|tj||dz}tj||f Sr)r vb_elbo_gradrrPrd)r_rgmgsr=s r elbo_gradz3_VariationalBayesMixedGLM.fit_vb..elbo_gradstA! A&&q!ubfQqrrUmm<.<<.rrTyperz Post. Meanr zPost. SDSDzSD (LB)zSD (UB)cg|]}d|zSz%.3frr^s rrz0BayesMixedGLMResults.summary..s...1FQJ...rcg|]}d|zSrrr^s rrz0BayesMixedGLMResults.summary..;;;!;;;rcg|]}d|zSrrr^s rrz0BayesMixedGLMResults.summary..rrz Mixed GLM ResultszAParameter types are mean structure (M) and variance structure (V)z:Variance parameters are modeled as log standard deviations)rrrr4r6r%rRrrrr rrPrlocrr7r8rSummary add_titler3rBradd_dfadd_text)r=dfrrsumms rsummaryzBayesMixedGLMResults.summarys \^^ J tz/ /<>!! tz(2;+, - - - B / 0 0 0 , - - - rc>|j}|j}|jj|_|jj|}t j|jj|k}||}||}fd|Dtj ||d}|_|S)a2 Posterior mean and standard deviation of random effects. Parameters ---------- term : int or None If None, results for all random effects are returned. If an integer, returns results for a given set of random effects. The value of `term` refers to an element of the `ident` vector, or to a position in the `vc_formulas` list. Returns ------- Data frame of posterior means and posterior standard deviations of random effects. Nc g|] }| Srr)rirs rrz7BayesMixedGLMResults.random_effects..s$$$A"Q%$$$r)Meanr) rrrr9r8rrrr2rr)r=termrrWtermixrEr_rs @rrandom_effectsz#BayesMixedGLMResults.random_effectss& L J Z   Z)//55F 0F :;;B"A"A$$$$$$$B L!1-- . . >AGrFcD|j|j||S)a Return predicted values for the mean structure. Parameters ---------- exog : array_like The design matrix for the mean structure. If None, use the model's design matrix. linear : bool If True, returns the linear predictor, otherwise transform the linear predictor using the link function. Returns ------- A one-dimensional array of fitted values. )rrrR)r=r?rs rrzBayesMixedGLMResults.predicts $z!!$+tV<<r?r0r2r;r:r7r8r9rArBs rr/zBinomialBayesMixedGLM.__init__s   $&&    vbi&&"%+566 "GCS// ! " "rc tj}t||||||}t |j|j|j|j|j |j |j |j |j  }|j|_|S)Nr3r;r:)r0r2r;r:r7r8r9)rr5rrr1r>r?r0r2r;r:r7r8r9ry) rrrryr;r:famr_mods rrz"BinomialBayesMixedGLM.from_formula s!!  ' ' [$s%d ( L L$ G FI''kkZ ! ! !6 rc  ||\}}}||\}}}|||||\ fd} || ||||||S)H Returns the evidence lower bound (ELBO) for the model. c tjdtjtj|zzz S)Nr )rrrPrrrrs rrz(BinomialBayesMixedGLM.vb_elbo..h,s6F1rvb272;;?&:;;;<<< .h;sRWR[[1_$A a  AA&&B2BRVRC[[)AbEQ''B2BF2JJ!bfRjj.1AbE2IrrIrrrBs @@rrz"BinomialBayesMixedGLM.vb_elbo_grad2s '+ll7&;&;#(G $ U 3 3&'5AAB      %%aR8W&,fe== =rr r NNN)r r ) rrr _init_docformat_logit_examplerr/rrrrrrs@rr1r1s~66G""""""6[* 0 0 0=======rr1ceZdZeeZ d fd Ze d dZ dZ dZ xZ S) PoissonBayesMixedGLMr2r r Nc t||||||tj|||  dS)N) r>r?r0r2r;r:r3r7r8r9)r.r/rPoisson) r=r>r?r0r2r;r:r7r8r9rBs rr/zPoissonBayesMixedGLM.__init__NsW #%%      rc tj}t||||||} t | j| j| j| j| j | j | j | j | j  } | j| _| S)Nr:) r>r?r0r2r;r:r7r8r9)rrRrrrPr>r?r0r2r;r:r7r8r9ry) rrrryr;r:r8r9r;r_r<s rrz!PoissonBayesMixedGLM.from_formulaes    ' '    (  #'I''kkZ ! ! !6 rc  ||\}}}||\}}}|||||\ fd} || ||||||S)r>c^tjtj|zz Sr]rrPrr@s rrz'PoissonBayesMixedGLM.vb_elbo..hs'F2 a/000 0rrArBs @@rrzPoissonBayesMixedGLM.vb_elbos '+ll7&;&;#(G $ U 3 3&'5AAB 1 1 1 1 1 1  B(GV!'00 0rc  ||\}}}||\}}}|||||\ fd} || |||||| S)rFcbtjtj|zz }|Sr]rV)ryrrs rrz,PoissonBayesMixedGLM.vb_elbo_grad..hs+RWR[[1_,---AHrrJrBs @@rrz!PoissonBayesMixedGLM.vb_elbo_grads '+ll7&;&;#(G $ U 3 3&'5AAB      %%aR8W&,fe== =rrK)r r NN) rrrrLrM_poisson_examplerr/rrrrrrs@rrPrPJs'788G. #"[B 0 0 0=======rrP)rnumpyrscipy.optimizerscipyrstatsmodels.base.modelbaserstatsmodels.iolibrstatsmodels.genmodrpandasrrr|rrLrNrZModelrrrr1rPrrrrds(..`######%%%%%%%%%&&&&&&'''''' ,-+,,-+,,-+,,-+,,-+, Z  |xxxxxTZxxxv Z*Z*Z*Z*Z*Z*Z*Z*zU=U=U=U=U=U=U=U=pY=Y=Y=Y=Y=5~Y=Y=Y=xZ=Z=Z=Z=Z=4nZ=Z=Z=Z=Z=r