Mh UdZdZddlZddlmZddlmZddlZddlmZm Z ddl m Z ddl m Z mZmZmZmZmZerdd lmZddlZddlZddlZddlZddlZddlZd'd Zd(d d dede d dfdZGddeZd ed dfdZ dZ!ee"d<dZ#ee"d<d)de$d efdZ%d*de$d efdZ&d+de$d efd Z'd,de$d dfd"Z(d-de$d$ed dfd%Z)e*d&kr$eej+,dSdS).z=Diagnostic functions, mainly for use when doing tech support.MITN)BytesIO) HTMLParser) BeautifulSoup __version__)builder_registry)AnyIOListOptionalTuple TYPE_CHECKING)_IncomingMarkupdatarreturnc tdtztdtjzgd}|D]C}tjD] }||jvrn(||td|zDd|vry|d ddl m }td d tt|jzn#t$rtd YnwxYwd |vr< dd l}td|jzn#t$rtdYnwxYwt#|dr|}|D]}td|zd} t'||}d}n5#t($r(td|zt+jYnwxYw|r3td|zt|tdd S)zDiagnostic suite for isolating common problems. :param data: Some markup that needs to be explained. :return: None; diagnostics are printed to standard output. z'Diagnostic running on Beautiful Soup %szPython version %s) html.parserhtml5liblxmlz;I noticed that %s is not installed. Installing it may help.rzlxml-xmlretreezFound lxml version %s.z.lxml is not installed or couldn't be imported.rNzFound html5lib version %sz2html5lib is not installed or couldn't be imported.readz#Trying to parse your markup with %sF)featuresT%s could not parse the markup.z#Here's what %s did with the markup:zP--------------------------------------------------------------------------------)printrsysversionrbuildersrremoveappendrrjoinmapstr LXML_VERSION ImportErrorrhasattrrr Exception traceback print_excprettify) r basic_parsersnamebuilderrrparsersuccesssoups L/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/bs4/diagnose.pydiagnoser3 s  4{ BDDD  ,...777M'0  Gw'''(   & & & NQUU   Z((( D " " " " " " *SXXc#u?Q6R6R-S-SS U U U U D D D B C C C C C D]"" H OOO .1EE G G G G H H H F G G G G G HtVyy{{   4v=??? " 777DGG " " " 3f< > > >   ! ! ! ! ! "  % 86A C C C 4==?? $ $ $ x  s7ACC87C8DD87D88F  /F>=F>Thtmlkwargsc Lddlm}|dd}t|tr|d}t|t st|}|j|f||d|D]'\}}t|d|j dd|j (d S) aPrint out the lxml events that occur during parsing. This lets you see how lxml parses a document when no Beautiful Soup code is running. You can use this to determine whether an lxml-specific problem is in Beautiful Soup's lxml tree builders or in lxml itself. :param data: Some markup. :param html: If True, markup will be parsed with lxml's HTML parser. if False, lxml's XML parser will be used. rrrecoverTutf8)r4r7z, z>4N) rrpop isinstancer$encoder r iterparsertagtext)rr4r5rr7readereventelements r2 lxml_tracerBXsjjD))G$#{{6"" dB  )%/&WtWWWPVWWDDw w{{{{GLLACCCCDDc eZdZdZdeddfdZ ddedeeeeefd e ddfd Z dded e ddfd Z d eddfdZ deddfdZ deddfdZd eddfdZd eddfdZd eddfdZd eddfdZdS)AnnouncingParserzSubclass of HTMLParser that announces parse events, without doing anything else. You can use this to get a picture of how html.parser sees a given document. The easiest way to do this is to call `htmlparser_trace`. srNc$t|dS)N)r)selfrFs r2_pzAnnouncingParser._pws arCTr-attrshandle_empty_elementc<||d|ddS)N z STARTrI)rHr-rJrKs r2handle_starttagz AnnouncingParser.handle_starttagzs- 4''%'''(((((rCcheck_already_closedc6|d|zdS)Nz%s ENDrN)rHr-rPs r2 handle_endtagzAnnouncingParser.handle_endtags 4     rCrc6|d|zdS)Nz%s DATArNrHrs r2 handle_datazAnnouncingParser.handle_data  D !!!!!rCc6|d|zdS)Nz %s CHARREFrNrHr-s r2handle_charrefzAnnouncingParser.handle_charref  t#$$$$$rCc6|d|zdS)Nz %s ENTITYREFrNrXs r2handle_entityrefz!AnnouncingParser.handle_entityrefs %&&&&&rCc6|d|zdS)Nz %s COMMENTrNrTs r2handle_commentzAnnouncingParser.handle_commentrZrCc6|d|zdS)Nz%s DECLrNrTs r2 handle_declzAnnouncingParser.handle_declrVrCc6|d|zdS)Nz%s UNKNOWN-DECLrNrTs r2 unknown_declzAnnouncingParser.unknown_decls  !D()))))rCc6|d|zdS)Nz%s PIrNrTs r2 handle_pizAnnouncingParser.handle_pis $rCT)__name__ __module__ __qualname____doc__r$rIr r r boolrOrRrUrYr\r^r`rbrdrCr2rErEosCD&* )))E#x},-.)# )  ))))!!#!T!T!!!!""""""%3%4%%%%'S'T''''%3%4%%%%""""""****** c d      rCrEcLt}||dS)zPrint out the HTMLParser events that occur during parsing. This lets you see how HTMLParser parses a document when no Beautiful Soup code is running. :param data: Some markup. N)rEfeed)rr/s r2htmlparser_tracerns'  F KKrCaeiou_vowelsbcdfghjklmnpqrstvwxyz _consonantslengthcd}t|D]1}|dzdkrt}nt}|tj|z }2|S)zzrsentence..s6GGAE&.A..//GGGGGGrC)r"rx)rts r2 rsentencers+ 88GGvGGG G GGrC num_elementsc gd}g}t|D]}tjdd}|dkr-tj|}|d|zJ|dkr6|t tjdd|dkr,tj|}|d|zd d |zd zS) zDRandomly generate an invalid HTML document. :meta private: )pdivspanr{bscripttablerz<%s>r~rwzz z)rxryrrzr!rr")r tag_nameselementsr{rztag_names r2rdocrs BAAIH < 0 01%% Q;;}Y//H OOFX- . . . . q[[ OOIfnQ&:&:;; < < < < q[[}Y//H OOGh. / / / dii)) )I 55rC順c6tdtzt|}tdt|zdddgddfD]}d} t j}t ||t j}d}n5#t $r(td |ztjYnwxYw|rtd |||z fzd d l m }t j}| |t j}td ||z zd dl }| }t j}||t j}td||z zdS)z.Very basic head-to-head performance benchmark.z1Comparative parser benchmark on Beautiful Soup %sz3Generated a large invalid HTML document (%d bytes).rr4rrFTrz"BS4+%s parsed the markup in %.2fs.rrz$Raw lxml parsed the markup in %.2fs.Nz(Raw html5lib parsed the markup in %.2fs.)rrrlentimerr(r)r*rrHTMLrrparse) rr parser_namer0arrrr/s r2benchmark_parsersrs > LNNN   D @3t99 LNNN 0*mL Q Q  " A $ , , , AGG " " " 3kA C C C   ! ! ! ! ! "  Q 7;A:NN P P P A JJt A 1QU ;===OOO  " "F A LL A 5Q ?AAAAAs8B  /B>=B>rr/c2tj}|j}t|}t t ||}t jd|||tj |}| d| dddS)z7Use Python's profiler on a randomly generated document.)bs4rr/zbs4.BeautifulSoup(data, parser) cumulativez _html5lib|bs42N) tempfileNamedTemporaryFiler-rdictrcProfilerunctxpstatsStats sort_stats print_stats)rr/ filehandlefilenamervarsstatss r2profilers,..JH   D Cd6 2 2 2D O5tT8LLL L " "E \""" or*****rC__main__)rrrNre)rs)r~)r)r)rr)-ri __license__rior html.parserrrrr bs4.builderrtypingr r r r r r bs4._typingrrryrrr)rr3rjrBrEr$rnrp__annotations__rrintr}rrrrrfstdinrrkrCr2rsCCC """""" ********((((((,++++++   5555pDD&DdDSDTDDDD.) ) ) ) ) z) ) ) X 3 4    * S***   #  c     HHcH#HHHH66s6c6666, B BC BT B B B BF + +# + + + + + +  z HSY^^  rC