Mh : U d Z dZddlmZ ddlmZ ddlZddlmZ ddlZddl m
Z
m Z ddl m
Z
dd lmZmZmZmZmZmZmZmZmZmZ dd
lmZ dd lm Z m Z ddl Z dZ ee
e d < ddl!Z!e!Z n1# e"$ r) ddl#Z#e#Z n # e"$ r ddl$Z$e$Z n # e"$ r Y nw xY wY nw xY wY nw xY wd
e%dee& fdZ'dZ(e&e d< dZ)e&e d< e* Z+eeee&ef f e d< ej, e)- d ej. ej, e(- d ej. de+e%< ej, e)ej. ej, e(ej. de+e&< G d de/ Z0e01 G d d Z2 G d d Z3dS ) a~ Beautiful Soup bonus library: Unicode, Dammit
This library converts a bytestream to Unicode through any means
necessary. It is heavily based on code from Mark Pilgrim's `Universal
Feed Parser `_, now maintained
by Kurt McKee. It does not rewrite the body of an XML or HTML document
to reflect a new encoding; that's the job of `TreeBuilder`.
MIT )codepoint2name) defaultdictN)html5)Logger getLogger)
ModuleType)
DictIteratorListOptionalPatternSetTupleTypeUnioncast)Literal) _Encoding
_Encodingschardet_modulesreturnc t t | t rdS t }| | d S )z?Try as hard as possible to detect the encoding of a bytestring.Nencoding)r
isinstancestrdetect)r modules J/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/bs4/dammit.py_chardet_dammitr! G s8 As!3!3 t
F == J '' z$^\s*<\?.*encoding=['"](.*?)['"].*\?> xml_encodingz0<\s*meta[^>]+charset\s*=\s*["']?([^>]*?)[ /;'">] html_meta encoding_resascii)htmlxmlc e Zd ZU dZeeef ed< eeef ed< ee ed< ee ed< ed$d Z d d
d d d
dZ
eeef ed< e j de j
Z e j d Zee ed< e j d Zee ed< ede j defd Zede j defd Zede j defd Zede j defd Zededefd Zed%ded edefd Ze d%ded edefd Zed edefd! Zed edefd" Zed edefd# Z dS )&EntitySubstitutionzFThe ability to substitute XML or HTML entities for certain characters.HTML_ENTITY_TO_CHARACTERCHARACTER_TO_HTML_ENTITYCHARACTER_TO_HTML_ENTITY_RE*CHARACTER_TO_HTML_ENTITY_WITH_AMPERSAND_REr Nc " i }i }t }t t }t t j D ]\ }}| d r |dd }n|}||vr|||< |||< t
| dk rt | dk r|dvr`t
| dk rt d |D rt
| dk r |dk r| | ||d | t }|D ]\} || }
|
s| | "d
d |
D } | | d | d
]t | D ] } | D ]}
| |
dd
| z }| d dd
| z }t t j D ]\ }}t | }|||< || _ || _ t# j | | _ t# j | | _ dS )u Initialize variables used by this class to manage the plethora of
HTML5 named entities.
This function sets the following class variables:
CHARACTER_TO_HTML_ENTITY - A mapping of Unicode strings like "⦨" to
entity names like "angmsdaa". When a single Unicode string has
multiple entity names, we try to choose the most commonly-used
name.
HTML_ENTITY_TO_CHARACTER: A mapping of entity names like "angmsdaa" to
Unicode strings like "⦨".
CHARACTER_TO_HTML_ENTITY_RE: A regular expression matching (almost) any
Unicode string that corresponds to an HTML5 named entity.
CHARACTER_TO_HTML_ENTITY_WITH_AMPERSAND_RE: A very similar
regular expression to CHARACTER_TO_HTML_ENTITY_RE, but which
also matches unescaped ampersands. This is used by the 'html'
formatted to provide backwards-compatibility, even though the HTML5
spec allows most ampersands to go unescaped.
;N z<>c 3 < K | ]}t | d k V dS )r3 N)ord.0xs r z?EntitySubstitution._populate_class_variables.. s, )J)J1#a&&3,)J)J)J)J)J)Jr" &r c g | ]
}|d S )r2 r6 s r
z@EntitySubstitution._populate_class_variables.. s !>!>!>1!A$!>!>!>r" z(?![z])z(%s)|)setr sortedr itemsendswithlenr5 alladdjoinlistvaluesr chrr, r+ recompiler- r. )clsunicode_to_namename_to_unicodeshort_entities long_entities_by_first_charactername_with_semicolon charactername particlesshort
long_versionsignore
long_entities long_entity
re_definition re_definition_with_ampersand codepoints r _populate_class_variablesz,EntitySubstitution._populate_class_variables s 0 +6s+;+;(.4U[]].C.C = N = N * #++C00
+*3B3/*
?**(1%
*.OI & 9~~""s9~~';'; QU@U@U
9~~!!c)J)J )J)J)J&J&J! 9~~""yC'7'7 ""9----01 >BB9MMMM EE # > >E
e$$$$!>!>
!>!>!>??
eeeVVV <==== !"B"I"I"K"K L L + +M ,
+
+
k****
+ )!4!44
c'-0C0C'C$ $N$8$:$: ; ; . .OItII)-OI & &'6$'6$*,*]*C*C'9; (:
:
666r" aposquotampltgt)'"r: <>CHARACTER_TO_XML_ENTITYz&(#\d+|#x[0-9a-fA-F]+|\w+);z&([<>]|&(?!#\d+;|#x[0-9a-fA-F]+;|\w+;))BARE_AMPERSAND_OR_BRACKETz([<>&])AMPERSAND_OR_BRACKETmatchobjc x | d }| j | }|d|z S d|z S )zpUsed with a regular expression to substitute the
appropriate HTML entity for a special character string.r N&%s;&%s;)groupr, get)rM rk original_entityentitys r _substitute_html_entityz*EntitySubstitution._substitute_html_entity sE #..++-11/BB > / / r" c L | j | d }d|z S )zoUsed with a regular expression to substitute the
appropriate XML entity for a special character string.r rn )rh ro )rM rk rr s r _substitute_xml_entityz)EntitySubstitution._substitute_xml_entity) s' ,X^^A->->? r" c 2 d| d z S )Nrm r2 )ro )rM rk s r _escape_entity_namez&EntitySubstitution._escape_entity_name0 s HNN1 - ---r" c R | d }|| j v rd|z S d|z S )Nr2 rn rm )ro r+ )rM rk possible_entitys r _escape_unrecognized_entity_namez3EntitySubstitution._escape_unrecognized_entity_name4 s7 "..++ c : : :O+ +O++r" valuec \ d}d|v r d|v rd}| d| }nd}||z |z S )a Make a value into a quoted XML attribute, possibly escaping it.
Most strings will be quoted using double quotes.
Bob's Bar -> "Bob's Bar"
If a string contains double quotes, it will be quoted using
single quotes.
Welcome to "my bar" -> 'Welcome to "my bar"'
If a string contains both single and double quotes, the
double quotes will be escaped, and the string will be quoted
using double quotes.
Welcome to "Bob's Bar" -> Welcome to "Bob's bar"
:param value: The XML attribute value to quote
:return: The quoted value
re rd z")replace)rM r{
quote_with replace_withs r quoted_attribute_valuez)EntitySubstitution.quoted_attribute_value; sN ,
%<<e|| (
c<88 !
E!J..r" Fmake_quoted_attributec t | j | j | }|r| | }|S )av Replace special XML characters with named XML entities.
The less-than sign will become <, the greater-than sign
will become >, and any ampersands will become &. If you
want ampersands that seem to be part of an entity definition
to be left alone, use `substitute_xml_containing_entities`
instead.
:param value: A string to be substituted.
:param make_quoted_attribute: If True, then the string will be
quoted, as befits an attribute value.
:return: A version of ``value`` with special characters replaced
with named entities.
)rj subru r rM r{ r s r substitute_xmlz!EntitySubstitution.substitute_xmlc s? & (,,S-GOO 6..u55E r" c t | j | j | }|r| | }|S )a Substitute XML entities for special XML characters.
:param value: A string to be substituted. The less-than sign will
become <, the greater-than sign will become >, and any
ampersands that are not part of an entity defition will
become &.
:param make_quoted_attribute: If True, then the string will be
quoted, as befits an attribute value.
)ri r ru r r s r "substitute_xml_containing_entitiesz5EntitySubstitution.substitute_xml_containing_entities| s? -11#2LeTT 6..u55E r" r c B | j | j | S )a Replace certain Unicode characters with named HTML entities.
This differs from ``data.encode(encoding, 'xmlcharrefreplace')``
in that the goal is to make the result more readable (to those
with ASCII displays) rather than to recover from
errors. There's absolutely nothing wrong with a UTF-8 string
containg a LATIN SMALL LETTER E WITH ACUTE, but replacing that
character with "é" will make it more readable to some
people.
:param s: The string to be modified.
:return: The string with some Unicode characters replaced with
HTML entities.
)r. r rs rM r s r substitute_htmlz"EntitySubstitution.substitute_html s' " =AA '
r" c | j | j | }| j | j | }|S )aH Replace certain Unicode characters with named HTML entities
using HTML5 rules.
Specifically, this method is much less aggressive about
escaping ampersands than substitute_html. Only ambiguous
ampersands are escaped, per the HTML5 standard:
"An ambiguous ampersand is a U+0026 AMPERSAND character (&)
that is followed by one or more ASCII alphanumerics, followed
by a U+003B SEMICOLON character (;), where these characters do
not match any of the names given in the named character
references section."
Unlike substitute_html5_raw, this method assumes HTML entities
were converted to Unicode characters on the way in, as
Beautiful Soup does. By the time Beautiful Soup does its work,
the only ambiguous ampersands that need to be escaped are the
ones that were escaped in the original markup when mentioning
HTML entities.
:param s: The string to be modified.
:return: The string with some Unicode characters replaced with
HTML entities.
)
ANY_ENTITY_REr rw r- rs r s r substitute_html5z#EntitySubstitution.substitute_html5 sA 6
! !#"91 = =
+ / /0KQ O Or" c | j | j | }| j | j | }|S )a Replace certain Unicode characters with named HTML entities
using HTML5 rules.
substitute_html5_raw is similar to substitute_html5 but it is
designed for standalone use (whereas substitute_html5 is
designed for use with Beautiful Soup).
:param s: The string to be modified.
:return: The string with some Unicode characters replaced with
HTML entities.
)r r rz r- rs r s r substitute_html5_rawz'EntitySubstitution.substitute_html5_raw sA
! !#"F J J
+ / /0KQ O Or" )r N)F) __name__
__module__ __qualname____doc__r
r __annotations__r classmethodr^ rh rK rL Ir ri rj Matchrs ru rw rz r boolr r r r r r= r" r r* r* b s" PP
#38n,,, #38n,,, ") --- 18 <<<A
A
A
[A
N
/ /T#s(^ BJ >EEM /9bj8/ / ws| *4I)>)>'#,>>> rx C [ bh 3 [ .28 . . . . [. , ,S , , , [, %/3 %/3 %/ %/ %/ [%/N 3 t PS [0 7< 04 [*
[
( [ B S S [ r" r* c e Zd ZU dZ ddedee dee dee dee d ee f d
Zee d< ee d< ee d< ee
e d < ee d< ee
e d < ee d< ee
e d
< dee
de e
defdZ e
dee
fd Zededeeee
f fd Ze ddeeef dededee
fd ZdS )EncodingDetectora This class is capable of guessing a number of possible encodings
for a bytestring.
Order of precedence:
1. Encodings you specifically tell EncodingDetector to try first
(the ``known_definite_encodings`` argument to the constructor).
2. An encoding determined by sniffing the document's byte-order mark.
3. Encodings you specifically tell EncodingDetector to try if
byte-order mark sniffing fails (the ``user_encodings`` argument to the
constructor).
4. An encoding declared within the bytestring itself, either in an
XML declaration (if the bytestring is to be interpreted as an XML
document), or in a tag (if the bytestring is to be
interpreted as an HTML document.)
5. An encoding detected through textual analysis by chardet,
cchardet, or a similar external library.
6. UTF-8.
7. Windows-1252.
:param markup: Some markup in an unknown encoding.
:param known_definite_encodings: When determining the encoding
of ``markup``, these encodings will be tried first, in
order. In HTML terms, this corresponds to the "known
definite encoding" step defined in `section 13.2.3.1 of the HTML standard `_.
:param user_encodings: These encodings will be tried after the
``known_definite_encodings`` have been tried and failed, and
after an attempt to sniff the encoding by looking at a
byte order mark has failed. In HTML terms, this
corresponds to the step "user has explicitly instructed
the user agent to override the document's character
encoding", defined in `section 13.2.3.2 of the HTML standard `_.
:param override_encodings: A **deprecated** alias for
``known_definite_encodings``. Any encodings here will be tried
immediately after the encodings in
``known_definite_encodings``.
:param is_html: If True, this markup is considered to be
HTML. Otherwise it's assumed to be XML.
:param exclude_encodings: These encodings will not be tried,
even if they otherwise would be.
NFmarkupknown_definite_encodingsis_htmlexclude_encodingsuser_encodingsoverride_encodingsc Z t |pg | _ |r,t j dt d | xj |z
c_ |pg | _ |pg }t
d |D | _ d | _ |dn|| _ d | _
| | \ | _ | _
d S )NzcThe 'override_encodings' argument was deprecated in 4.10.0. Use 'known_definite_encodings' instead. )
stacklevelc 6 g | ]}| S r= )lowerr6 s r r> z-EncodingDetector.__init__../ s %K%K%KAaggii%K%K%Kr" F)rH r warningswarnDeprecationWarningr r@ r chardet_encodingr declared_encodingstrip_byte_order_markr sniffed_encoding)selfr r r r r r s r __init__zEncodingDetector.__init__ s )--E-K(L(L% @ Mu"
) )-? ? ) ) , 2 - 3!$%K%K9J%K%K%K!L!L $ 'uuW 04 .2-G-G-O-O* T***r" r r r r triedr c |dS | }|| j v rdS ||vr| | dS dS )zShould we even bother to try this encoding?
:param encoding: Name of an encoding.
:param tried: Encodings that have already been tried. This
will be modified as a side effect.
NFT)r r rF )r r r s r _usablezEncodingDetector._usable@ sX 5>>## t- - -5 5 IIh 4ur" c # K t }| j D ] }| || r|V | j $| | j | r | j V | j D ] }| || r|V | j %| | j | j | _ | j $| | j | r | j V | j t | j | _ | j $| | j | r | j V dD ] }| || r|V dS )zYield a number of encodings that might work for this markup.
:yield: A sequence of strings. Each is the name of an encoding
that *might* work to convert a bytestring into Unicode.
N)utf-8 windows-1252) r@ r r r r r find_declared_encodingr r r r! )r r es r encodingszEncodingDetector.encodingsQ s !$ . A||Au%%
, !52
2
, ' ' ' ' $ A||Au%%
! )%)%@%@ T\& &D " ! -$,, "E3
3
- ( ( ( ( ($3DK$@$@D ! , !52
2
, ' ' ' ' + A||Au%%
r" datac d}t |t r||fS t | dk r)|dd dk r|dd dk r
d}|dd }nt | dk r)|dd dk r|dd dk r
d}|dd }nP|dd d
k r
d }|d d }n5|dd d k r
d
}|dd }n|dd dk r d}|dd }||fS )a If a byte-order mark is present, strip it and return the encoding it implies.
:param data: A bytestring that may or may not begin with a
byte-order mark.
:return: A 2-tuple (data stripped of byte-order mark, encoding implied by byte-order mark)
N s s zutf-16bes zutf-16ler s r s zutf-32bes zutf-32le)r r rD )rM r r s r r z&EncodingDetector.strip_byte_order_mark s6 dC "> !
YY!^^bqb[((ack))!H8DD
YY!^^bqb[((ack))!H8DD
"1"X
(
( H8DD
"1"X,
,
,!H8DD
"1"X,
,
,!H8DX~ r" search_entire_documentc B |rt | x}}n/d}t dt t | dz }t |t rt
t }nt
t }|d }|d }d} | || }
|
s|r| || }
|
|
d } | r?t | t r| d d
} |
S dS ) a' Given a document, tries to find an encoding declared within the
text of the document itself.
An XML encoding is declared at the beginning of the document.
An HTML encoding is declared in a tag, hopefully near the
beginning of the document.
:param markup: Some markup.
:param is_html: If True, this markup is considered to be HTML. Otherwise
it's assumed to be XML.
:param search_entire_document: Since an encoding is supposed
to declared near the beginning of the document, most of
the time it's only necessary to search a few kilobytes of
data. Set this to True to force this method to search the
entire document.
:return: The declared encoding, if one is found.
i i g?r( r' N)endposr r& r} ) rD maxintr bytesr% r searchgroupsdecoder ) rM r r r
xml_endpos html_endposresxml_rehtml_rer declared_encoding_matchs r r z'EncodingDetector.find_declared_encoding s( 2 " ='*6{{ 2J J dCF d(:$;$;< > @ @ C -+U33
Q$5$<$"'26/337P PP #+:"6P $ P
$J/ P !,
P %Z 0P P P P6 )((( !!!! y))))
MMM **** MMM y)))) 3 C N t " 18I. 1 1 1 X1f # # 5@S9S3T # # # [# J ', / /eSj!/ / !% /
) / / / [/ / /r" r c e Zd ZU dZg ddg ddfdedee deed ded ee d
ee d ee fd Z ee
d< ee e
d
< ee
d< ee e
d< ee e
d< e
ee e f e
d< ee
d< dej defdZdddZee e f e
d< g dZee
d< dd e d e dee fd Z dd ed e d e de fd!Zedee fd" Zd#e dee fd$Zd#e dee fd%Zi d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@dAdBdCdDdAdEdAdFdGdHdIdJdKdLdMdNdOdPdQdAdRdSdTZ eee e ee e f f f e
dU< i d&dVd(d)d*dWd,dXd.dYd0dZd2d[d4d\d6d]d8d^d:d_ddad@dAdBdbdDdAdEdAi dcdddedddfdgdhdgdidjdkdldmdndodpdqdrdsdtdudvdwdxdydAdzd{d|d}d~d)ddi dddddddddddd_dddddddddddd)ddddlddddddi dddddddddddjddWddddddddddddddAddddddddi ddddddddddddddddddœddœddœddœddʓdd̓ddΓddΓddΓi ddΓddΓddjddΓdd֓dd֓dd֓dd֓dd}ddܓddޓddddddddddddi dddddddddddddddddddddddddddddddddddddddddddd Z eee f e
d< i dd ddddddddd d
d d d
ddddddddddddddd d d d d i d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;d<d=d>d?d@dAdBi dCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_d`dadbdcddi dedfdgdhdidjdkdldmdndodpdqdrdsdtdudvdwdxdydzd{d|d}d~ddddddddi ddddddddddddddddddddddddddddddddddi ddddddddddddddddddddddddddddÓdĐdœdƐdǓdȐdɓi dʐd˓d̐d͓dΐdϓdАdѓdҐdӓdԐdՓddדdؐdٓdڐdۓdܐdݓdސdߓddddddddddddddddZ ee ef e
d< g dZ!e
ee e e f e
d< e!d d Z"e e
d< e!d d Z#e e
d< e$ ddede de defd Z%dS (
UnicodeDammita# A class for detecting the encoding of a bytestring containing an
HTML or XML document, and decoding it to Unicode. If the source
encoding is windows-1252, `UnicodeDammit` can also replace
Microsoft smart quotes with their HTML or XML equivalents.
:param markup: HTML or XML markup in an unknown encoding.
:param known_definite_encodings: When determining the encoding
of ``markup``, these encodings will be tried first, in
order. In HTML terms, this corresponds to the "known
definite encoding" step defined in `section 13.2.3.1 of the HTML standard `_.
:param user_encodings: These encodings will be tried after the
``known_definite_encodings`` have been tried and failed, and
after an attempt to sniff the encoding by looking at a
byte order mark has failed. In HTML terms, this
corresponds to the step "user has explicitly instructed
the user agent to override the document's character
encoding", defined in `section 13.2.3.2 of the HTML standard `_.
:param override_encodings: A **deprecated** alias for
``known_definite_encodings``. Any encodings here will be tried
immediately after the encodings in
``known_definite_encodings``.
:param smart_quotes_to: By default, Microsoft smart quotes will,
like all other characters, be converted to Unicode
characters. Setting this to ``ascii`` will convert them to ASCII
quotes instead. Setting it to ``xml`` will convert them to XML
entity references, and setting it to ``html`` will convert them
to HTML entity references.
:param is_html: If True, ``markup`` is treated as an HTML
document. Otherwise it's treated as an XML document.
:param exclude_encodings: These encodings will not be considered,
even if the sniffing code thinks they might make sense.
NFr r smart_quotes_to)r& r( r' r r r r c || _ g | _ d| _ || _ t t
| _ t |||||| | _ t |t s|dk r$|| _ t | | _ d | _
d S | j j | _ d }| j j D ]'} | j j }| | }| n(|sP| j j D ]C} | dk r| | d }|#| j d d| _ nD|d | _
d | _ d S || _ d S )NFr" r& r} zSSome characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.T)r tried_encodings contains_replacement_charactersr r r logr detectorr r r unicode_markuporiginal_encodingr
_convert_fromwarning)
r r r r r r r r ur s
r r zUnicodeDammit.__init__ s / ! /4, X&&( $
fc " " fmm DK"%f++D %)D " F m*
/ H])F""8,,A} !M 3
w&& **8Y??A=H$$?
<@D8E !"
9%)D ""&D "#D r" r r r r r matchr c | d }| j dk r,|| j v r | j | }n|}n|| j v r| j | }t | t u rM| j dk r!d|d z dz }nMd|d z dz }n,t t | }| }n|}|S )zChanges a MS smart quote character to an XML or HTML
entity, or an ASCII character.
TODO: Since this is only used to convert smart quotes, it
could be simplified, and MS_CHARS_TO_ASCII made much less
parochial.
r2 r& r( s &#x ; &r ) ro r MS_CHARS_TO_ASCIIencodeMS_CHARStypetupler r )r r origr
substitutionss r _sub_ms_charzUnicodeDammit._sub_ms_charl s kk!nn 7 * *t---,T299;; t}$$ $
d 3
&&%//+u44 $}Q'7'>'>'@'@ @4 G "]1%5%<%<%>%> > E$(m$<$> z1/4 z1/2 z3/4 A AE C E r D N O U b B a ae r i n /y) r r3 s € s ‚ s ƒ s „ s … s † s ‡ s ˆ s ‰ s Š s ‹ s Œ s Ž s ‘ s ’ s “ s ” s • s – s — s ˜ s ™ s š s › s œ s ž s Ÿ s s ¡ s ¢ s £ s ¤ s ¥ s ¦ s § s ¨ s © s ª s « s ¬ s s ® s ¯ s ° s ± s ² s ³ s ´ s µ s ¶ s · s ¸ s ¹ s º s » s ¼ s ½ s ¾ s ¿ s À s Á s  s à s Ä s Å s Æ s Ç s È s É s Ê s Ë s Ì s Í s Î s Ï s Ð s Ñ s Ò s Ó s Ô s Õ s Ö s × s Ø s Ù s Ú s Û s Ü s Ý s Þ s ß s à s â s ã s ä s å s æ s ç s è s é s ê s ë s ì s í s î s ï s ð s ñ s ò s ó s ô s õ s ö s ÷ s ø s ù s ú s ûs üs ýs þ) WINDOWS_1252_TO_UTF8))r r7 r )r8 rG r )rH rL r MULTIBYTE_MARKERS_AND_SIZESr FIRST_MULTIBYTE_MARKERr1 r2 LAST_MULTIBYTE_MARKERutf8r in_bytes
main_encodingembedded_encodingc | dd dvrt d | dvrt d g }d}d}|t | k r|| }|| j k r-|| j k r"| j D ]\ }} }
||k r
|| k r||
z
} nnY|dk rN|| j v rE| ||| | | j | |d z
}|}n|d z
}|t | k |dk r|S | ||d
d | S ) a Fix characters from one encoding embedded in some other encoding.
Currently the only situation supported is Windows-1252 (or its
subset ISO-8859-1), embedded in UTF-8.
:param in_bytes: A bytestring that you suspect contains
characters from multiple encodings. Note that this *must*
be a bytestring. If you've already converted the document
to Unicode, you're too late.
:param main_encoding: The primary encoding of ``in_bytes``.
:param embedded_encoding: The encoding that was used to embed characters
in the main document.
:return: A bytestring similar to ``in_bytes``, in which
``embedded_encoding`` characters have been converted to
their ``main_encoding`` equivalents.
r r )r windows_1252zPWindows-1252 and ISO-8859-1 are the only currently supported embedded encodings.)r[ r z4UTF-8 is the only currently supported main encoding.r r3 r2 Nr" )
r} r NotImplementedErrorrD rY rZ rX rW r rG ) rM r\ r] r^ byte_chunks chunk_startposbytestartendsizes r detwinglezUnicodeDammit.detwingle; s . $ $S# . . 4 4 6 6 ?
&&
(9 9 9%F
CMM!!C=Ds111dc>W6W6W ),(G $E3u}}t $#*B"B"B ""8KO#<=== ""3#;D#ABBBq ! q+ CMM!!, ! O
x 5 6 6 6xx $$$r" )r )r[ r )&r r r r r r
r r r r r r r r r r rK r r r r
r r r r r r r r r r rW r rX rY rZ r ri r= r" r r r sG & &V : ?B+@ AB+B CB+D EB+ B+ B+F GB+H IB+J KB+N OB+P QB+R SB+T UB+V WB+X YB+Z [B+\ ]B+^ _B+` aB+b cB+d eB+f gB+h iB+ B+ B+j kB+l mB+n oB+p qB+r sB+t uB+v wB+x yB+z {B+| }B+~ B+@ AB+B CB+D EB+F GB+H IB+J KB+ B+ B+L MB+N OB+P QB+R SB+T UB+V WB+X YB+Z [B+\ ]B+^ _B+` aB+b cB+d eB+f gB+h iB+j kB+l mB+ B+ B+n oB+p qB+r sB+t uB+v wB+x yB+z {B+| }B+~ B+@ AB+B CB+D EB+F GB+H IB+J KB+L MB+N OB+ B+ B+P QB+R SB+T UB+V WB+X YB+Z [B+\ ]B+^ _B+` aB+b cB+d eB+f gB+h iB+j kB+l mB+n oB+p qB+ B+r CB+ B+ B+tE3J' B B BV{. o{. o{.
k{.
o {.
o {.
o
{.
o{.
k{.
o{.
k{.
o{.
k{.
k{.
o {.
o {.
o!{."
o#{. {.$
o%{.&
o'{.(
o){.*
k+{.,
o-{..
k/{.0
o1{.2
k3{.4
k5{.6
k7{.8
k9{.:
k;{.<
k={.>
k?{.@
kA{.B
kC{.D
kE{. {. {.F
kG{.H
kI{.J
kK{.L
kM{.N
kO{.P
kQ{.R
kS{.T
kU{.V
kW{.X
kY{.Z
k[{.\
k]{.^
k_{.`
ka{.b
kc{.d
ke{.f
kg{. {. {.h
ki{.j
kk{.l
km{.n
ko{.p
kq{.r
ks{.t
ku{.v
kw{.x
ky{.z
k{{.|
k}{.~
k{.@
kA{.B
kC{.D
kE{.F
kG{.H
kI{. {. {.J
kK{.L
kM{.N
kO{.P
kQ{.R
kS{.T
kU{.V
kW{.X
kY{.Z
k[{.\
k]{.^
k_{.`
ka{.b
kc{.d
ke{.f
kg{.h
ki{.j
kk{. {. {.l
km{.n
ko{.p
kq{.r
ks{.t
ku{.v
kw{.x
ky{.z
g{{.|
k}{.~
k{.@
kA{.B
kC{.D
kE{.F
kG{.H
kI{.J
kK{.L
kM{. {. {.N
kO{.P
kQ{.R
kS{.T
kU{.V
kW{.X
kY{.Z
k[{.\
k]{.^
k_{.`
ka{.b
kc{.d
ke{.f
kg{.h
ki{.j
kk{.l
km{.n
ko{. {.p u{. {. {.$sEz* { { {|? ? ? eCcM&:!; #>a"@"CCCCC "=R!@!C3CCC $*'5 D% D%D% !D% % D%
D% D% D% [D% D% D%r" r )4r __license__
html.entitiesr collectionsr r r rK loggingr r typesr typingr
r r r
r r r r r r typing_extensionsr bs4._typingr r r r r cchardet ImportErrorchardetcharset_normalizerr r r! r# r$ dictr% rL r r objectr* r^ r r r= r" r rx s ( ( ( ( ( ( # # # # # #
% % % % % % % % & % % % % % (,$ + + +OOO NN % % % % /NN D
$(u (# ( ( ( ( @ c ? ? ?;
3
04tvv d4c7l++, 5 5 5 BJy''00"$ 7 7 2:l))'22BD 9 9 U
BJy"$ ' ' 2:lBD ) ) S } } } } } } } }@ , , . . .v v v v v v v vrb
% b
% b
% b
% b
% b
% b
% b
% b
% b
%sZ &A- -B3A:9B:B BBB BBBBBBB