L-PhMdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZmZe e jZdd ZddZddZddZddZdS)z"Better tokenizing for coverage.py.) annotationsN)Iterable)env)TLineNoTSourceTokenLinestoks TokenInfosreturnc #Kd}d}d}|D]H\}}\}}\}} } ||kr|r|drd} |drd} nv|tjkrC|dr+|d|rd} n&d} n#tjjr|tjkrd} | rLt| d d d z } tj d d|| f|| d zf|V| }|tj tj fvr|}tj ||||f|| f| V|}JdS)aBReturn all physical tokens, even line continuations. tokenize.generate_tokens() doesn't return a token for the backslash that continues lines. This wrapper provides those tokens so that we can re-create a faithful representation of the original source. Returns the same values as generate_tokens() Nz\ T\Fz \  i)endswithtokenSTRINGrstripr PYBEHAVIORfstring_syntaxFSTRING_MIDDLElensplittokenize TokenInfoNEWLINENL) r last_line last_lineno last_ttextttypettextslinenoscolelinenoecolltextinject_backslashccols S/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/coverage/phystokens.py _phys_tokensr-s!IKJAE22=uow ' ! !+ Y//77+  $( &&t,,-',$$el**!**622 1!((11:::FF 1,0((,1((^2-u@T7T7T',$#yt44R899A=D",v $'46):! I )8;7 7 7J $QVWWWWW e22sourcestr set[TLineNo]ct}tjtj|D]}tjdkr^t |tjrD||j |j D]!}||j j "ptjdkr4t |tj r||j |S)zCHelper for finding lines with soft keywords, like match/case lines.) )r3 ) setastwalkparsesys version_info isinstanceMatchaddlinenocasespattern TypeAlias)r/soft_key_linesnodecases r,find_soft_key_linesrF[s#&55N6**++,,  w & &:dCI+F+F &   t{ + + +  8 8""4<#67777 8   ( (Zcm-L-L (   t{ + + + r.rc#Ktjtjtjtjh}g}d}|ddd}t|}tj j rt|}nt}t|D]\}}\}} \} } } d} tjd|D]} | dkr |Vg}d}d}n}| dkrd}ns||vrd}nktj jrL|tjkr<| d d d d } | t'| z} | r%| |kr|d d| |z zfd} tj|ddd}|tjkrt3j|rd}nmtj j r\t3j|rHt'|dkrd}n*t'|dkr|ddd krd}nd}|r||vrd}||| fd}d} |r| }|r|VdSdS)aGenerate a series of lines, one for each line in `source`. Each line is a list of pairs, each pair is a token:: [('key', 'def'), ('ws', ' '), ('nam', 'hello'), ('op', '('), ... ] Each pair has a token class, and the token text. If you concatenate all the token texts, and then join them with newlines, you should have your original `source` back, with two differences: trailing white space is not preserved, and a final line with no newline is indistinguishable from a final line with a newline. rz rTz( )Fr {z{{}z}}ws xxNr3keyr)rINDENTDEDENTrrr expandtabsreplacegenerate_tokensrr soft_keywordsrFr6r-rerrrrappendtok_namegetlowerNAMEkeyword iskeyword issoftkeyword)r/ ws_tokenslinecoltokgenrCr#r$sliner&_r( mark_startpartmark_end tok_classis_start_of_lines r,source_token_linesrijs u|U]HKHI"$D C   q ! ! ) )&$ 7 7F V $ $F ~#,V445A&5I5I&&1umudYa HVU++" " Dt||   )## >0,Ue>R5R5R<<T22::3EED#d))+D'$**KKsdSj'9 :;;;!&J$-11%>>DDFFrrJ EJ&&(// .$) 5 .':OPU:V:V .t99>>/3,,!$ii1nn$q'!*2D2D/3,,/4,+.0G0G(-I Y-...DD  C  r.textc\tj|j}tj|S)zA helper around `tokenize.generate_tokens`. Originally this was used to cache the results, but it didn't seem to make reporting go faster, and caused issues with using too much memory. )ioStringIOreadlinerrS)rjrns r,rSrSs'{4  )H  #H - --r.bytesct|dj}tj|dS)zDetermine the encoding for `source`, according to PEP 263. `source` is a byte string: the text of the program. Returns a string, the name of the encoding. Tr)iter splitlines__next__rdetect_encoding)r/rns r,source_encodingrus8F%%d++,,5H  #H - -a 00r.)rr r r )r/r0r r1)r/r0r r)rjr0r r )r/ror r0)__doc__ __future__rr7rlr[rUr:rrcollections.abcrcoveragercoverage.typesrrrr r-rFrirSrur.r,r|s)(""""""  $$$$$$55555555h( ) ????D    EEEEP.... 1 1 1 1 1 1r.