.Ph&dZddlZddlZddlZdZddgZeddfdZ GddZed krd Z e dSdS) z``jsonutils`` aims to provide various helpers for working with JSON. Currently it focuses on providing a reliable and intuitive means of working with `JSON Lines`_-formatted files. .. _JSON Lines: http://jsonlines.org/ N JSONLIteratorreverse_iter_linesTc#K |p|j}d}n#t$rd}YnwxYw|} |}n#ttjf$rYnwxYwd\}}}|r |dt j|}|} d| krt|| } | | z} || t j | | } | |z}| } t| dks | d|kr|dd|kr|r|n|V| dddD]} |r| |n| V| d}d| k|r|r||n|VdSdS)a;Returns an iterator over the lines from a file object, in reverse order, i.e., last line first, first line last. Uses the :meth:`file.seek` method of file objects, and is tested compatible with :class:`file` objects, as well as :class:`StringIO.StringIO`. Args: file_obj (file): An open file object. Note that ``reverse_iter_lines`` mutably reads from the file and other functions should not mutably interact with the file object after being passed. Files can be opened in bytes or text mode. blocksize (int): The block size to pass to :meth:`file.read()`. Warning: keep this a fairly large multiple of 2, defaults to 4096. preseek (bool): Tells the function whether or not to automatically seek to the end of the file. Defaults to ``True``. ``preseek=False`` is useful in cases when the file cursor is already in position, either at the end of the file or in the middle for relative reverse line generation. zutf-8N) r)encodingAttributeErrordetachioUnsupportedOperationseekosSEEK_ENDtellminSEEK_SETread splitlineslendecode)file_obj blocksizepreseekr orig_obj empty_bytes newline_bytes empty_textbuffcur_pos read_sizecurlineslines Q/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/boltons/jsonutils.pyrr3s00x0  H ??$$ B3 4     .<*K & a%%% DmmooG g++ 7++ 9 gr{+++mmI&&Tz!! u::>>U1X44  9 % % (9**k 9 9 9%1R%L > >D+3=$++h''' = = = =Qx g++ :'/9dkk(###T99999::s  :AAcNeZdZdZ d dZedZdZdZdZ d Z e Z dS) raThe ``JSONLIterator`` is used to iterate over JSON-encoded objects stored in the `JSON Lines format`_ (one object per line). Most notably it has the ability to efficiently read from the bottom of files, making it very effective for reading in simple append-only JSONL use cases. It also has the ability to start from anywhere in the file and ignore corrupted lines. Args: file_obj (file): An open file object. ignore_errors (bool): Whether to skip over lines that raise an error on deserialization (:func:`json.loads`). reverse (bool): Controls the direction of the iteration. Defaults to ``False``. If set to ``True`` and *rel_seek* is unset, seeks to the end of the file before iteration begins. rel_seek (float): Used to preseek the start position of iteration. Set to 0.0 for the start of the file, 1.0 for the end, and anything in between. .. _JSON Lines format: http://jsonlines.org/ FNc~t||_||_||_||rd}n,d|cxkrdksnt d|z|dkrd|z }||_d|_|||jr#t|j|jd|_ dSt|j|_ dS)N?gz8'rel_seek' expected a float between -1.0 and 1.0, not %rrrF)rr) bool_reverse _file_obj ignore_errors ValueError _rel_seek _blocksize_init_rel_seekr _line_iteriter)selfrr/reverserel_seeks r(__init__zJSONLIterator.__init__sW  !*   &&&&3&&&&57?@AA A \\X~H!      ! ! ! = 30;??9>@@@DOOO#4>22DOOOrc4|jS)zBA property representing where in the file the iterator is reading.)r.rr6s r( cur_byte_poszJSONLIterator.cur_byte_poss~""$$$rc |j|j}}d\}}|}d|vr||}||z }d|v |d|z|z }n#t $rwxYw|||zdS)z6Aligns the file object's position to the next newline.)r r N)r.r2rrindexr0r)r6fobsizer% total_readr#newline_offsets r(_align_to_newlinezJSONLIterator._align_to_newlinesNDOEZ''))#oo''%..C % J#oo  YYt__z9EANN      .()))))s A'' A3c|j|j}}|dkr"|dtjdS|dtj|}|dkr ||_dSt||z}||tj| ||_dS)zCSets the file object's position to the relative location set above.grr+N) r1r.rrrrr_cur_posintrD)r6rsr@sizetargets r(r3zJSONLIterator._init_rel_seeksB 99 GGAr{ # # # # # GGAr{ # # #7799DSyy $ TBY ,,,&&((( "  rc|S)Nr;s r(__iter__zJSONLIterator.__iter__s rc t|j}|s* tj|}n#t $r |jsYTwxYw|S)zYields one :class:`dict` loaded with :func:`json.loads`, advancing the file object by one line. Raises :exc:`StopIteration` upon reaching the end of the file (or beginning, if ``reverse`` was set to ``True``. )nextr4lstripjsonloads Exceptionr/)r6r'objs r(rOzJSONLIterator.nextsx ((//11D  j&&   ) JsAAA)FFN) __name__ __module__ __qualname____doc__r9propertyr<rDr3rMrO__next__rLrr(rr{s.?C33332%%X% * * **** "HHHr__main__c ddl}d|jvs d|jvrtddSd}d|jvs d|jvrd}d \}}|jd d}|D]}|d vr|d z }t|d 5}t |}d} t |nH#t $r0td |d zd|jd|YddddSt$rYnXwxYw|d z }|d z }|rG|rE|dzdkr<|j d|dzr|j d|z dddn #1swxYwY|r$td|ztd|zdS)Nr-h--helpz7loads one or more JSON Line files for basic validation.F-v --verboseT)rr)r]r^r_r`rbzerror reading object #z around byte z in d.i'z%s zfiles checked: %szobjects loaded: %s) sysargvprintopenrrOr0r< StopIterationstdoutwrite) reverbose file_count obj_count filenamesfilenameriterator cur_obj_counts r(_mainrss 38  x3833 K L L L F 38  {ch66G $ IHQRRL ! A AH>>> !OJh%% A(22 ! AX%!.!2!2!2H4I4I4I88UVVV A A A A A A A A A)NI!Q&MA9ASA1E1E ((---$u,AJ,,Vi-?@@@A A A A A A A A A A A A A A A A$  4 % 2 3 3 3 &2 3 3 3sC0D=BD=*C>D= CD=CAD==E E ) rXrrrQDEFAULT_BLOCKSIZE__all__rrrUrsrLrr(rvs>  0 1,=dUY=:=:=:=:BggggggggT z###J EGGGGGMr