y-Ph{'ddlZddlZddlmZmZmZmZmZm Z m Z m Z m Z m Z mZmZmZmZmZmZddlmZGddejZdZGddejZGdd ejZGd d ejZd Zd ZddddZ d!ee _"ddddZ#ddddZ$d!ee$_"dddddZ%ddddZ&dddZ'dS)N)IpcReadOptionsIpcWriteOptions ReadStats WriteStatsMessage MessageReaderRecordBatchReader_ReadPandasMixinMetadataVersion read_messageread_record_batch read_schema read_tensor write_tensorget_record_batch_sizeget_tensor_sizec eZdZdZddddZdS)RecordBatchStreamReadera Reader for the Arrow streaming binary format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. If you want to use memory map use MemoryMappedFile as source. options : pyarrow.ipc.IpcReadOptions Options for IPC deserialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. Noptions memory_poolcTt|}||||dS)Nr _ensure_default_ipc_read_options_open)selfsourcerrs K/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/pyarrow/ipc.py__init__z RecordBatchStreamReader.__init__2s-27;; 67 DDDDD__name__ __module__ __qualname____doc__rr rrr"sE  +/DEEEEEEEr raNParameters ---------- sink : str, pyarrow.NativeFile, or file-like Python object Either a file path, or a writable file object. schema : pyarrow.Schema The Arrow schema for data to be written to the file. use_legacy_format : bool, default None Deprecated in favor of setting options. Cannot be provided with options. If None, False will be used unless this default is overridden by setting the environment variable ARROW_PRE_0_15_IPC_FORMAT=1 options : pyarrow.ipc.IpcWriteOptions Options for IPC serialization. If None, default values will be used: the legacy format will not be used unless overridden by setting the environment variable ARROW_PRE_0_15_IPC_FORMAT=1, and the V5 metadata version will be used unless overridden by setting the environment variable ARROW_PRE_1_0_METADATA_VERSION=1.cFeZdZdeZddddZdS)RecordBatchStreamWriterz0Writer for the Arrow streaming binary format {}Nuse_legacy_formatrcVt||}||||dSNr_get_legacy_format_defaultrrsinkschemar*rs rrz RecordBatchStreamWriter.__init__S0,->HH 4 11111r r"r#r$format_ipc_writer_class_docr%rr&r rr(r(NsL f "## ;?2222222r r(c"eZdZdZdddddZdS)RecordBatchFileReadera Class for reading Arrow record batch data from the Arrow binary file format Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. If you want to use memory map use MemoryMappedFile as source. footer_offset : int, default None If the file is embedded in some larger file, this is the byte offset to the very end of the file data options : pyarrow.ipc.IpcReadOptions Options for IPC serialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. NrcVt|}|||||dS)N footer_offsetrrr)rrr;rrs rrzRecordBatchFileReader.__init__ks=27;; 6"   = = = = =r Nr!r&r rr8r8XsA$=d!=======r r8cFeZdZdeZddddZdS)RecordBatchFileWriterz1Writer to create the Arrow binary file format {}Nr)cVt||}||||dSr,r.r0s rrzRecordBatchFileWriter.__init__xr3r r4r&r rr>r>rsL f "## ;?2222222r r>c||td|rFt|ts/tdt ||St j}|:tttj dd}tttj ddr t j }t||S)Nz8Can provide at most one of options and use_legacy_formatz expected IpcWriteOptions, got {}ARROW_PRE_0_15_IPC_FORMAT0ARROW_PRE_1_0_METADATA_VERSION)r*metadata_version) ValueError isinstancer TypeErrorr5typer V5boolintosenvirongetV4)r*rrDs rr/r/}s$)< FHH H '?33 4>#VDMM2244 4&) RZ^^$?EEFF G G  C ?EE F FGG.*- ->,< > > >>r c|rDt|ts/tdt ||p tS)Nzexpected IpcReadOptions, got {})rFrrGr5rHr-s rrrsW z'>::  - 4 4T']] C C     &n&&&r r)c(t||||SNr))r(r1r2r*rs r new_streamrTs# "45F+2 4 4 44r zCreate an Arrow columnar IPC stream writer instance {} Returns ------- writer : RecordBatchStreamWriter A writer for the given sink rc&t|||S)a Create reader for Arrow streaming format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. options : pyarrow.ipc.IpcReadOptions Options for IPC serialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. Returns ------- reader : RecordBatchStreamReader A reader for the given source r)r)rrrs r open_streamrVs"& #67/: < < <rSs rnew_filerXs# v3D)0 2 2 22r zCreate an Arrow columnar IPC file writer instance {} Returns ------- writer : RecordBatchFileWriter A writer for the given sink c(t||||S)a Create reader for Arrow file format. Parameters ---------- source : bytes/buffer-like, pyarrow.NativeFile, or file-like Python object Either an in-memory buffer, or a readable file object. footer_offset : int, default None If the file is embedded in some larger file, this is the byte offset to the very end of the file data. options : pyarrow.ipc.IpcReadOptions Options for IPC serialization. If None, default values will be used. memory_pool : MemoryPool, default None If None, default memory pool is used. Returns ------- reader : RecordBatchFileReader A reader for the given source r:)r8)rr;rrs r open_filerZs%, !m[ 2 2 22r nthreadspreserve_indexc"tj|||}tj}tj||j5}||dddn #1swxYwY|S)a Serialize a pandas DataFrame into a buffer protocol compatible object. Parameters ---------- df : pandas.DataFrame nthreads : int, default None Number of threads to use for conversion to Arrow, default all CPUs. preserve_index : bool, default None The default of None will store the index as a column, except for RangeIndex which is stored as metadata only. If True, always preserve the pandas index data as a column. If False, no index information is saved and the result will have a default RangeIndex. Returns ------- buf : buffer An object compatible with the buffer protocol. r[N)pa RecordBatch from_pandasBufferOutputStreamr(r2 write_batchgetvalue)dfr\r]batchr1writers rserialize_pandasrhs( N & &rH6D ' F FE " "D #D%, 7 7"65!!!""""""""""""""" ==??sA22A69A6T use_threadsctj|}tj|5}|}dddn #1swxYwY||S)aDeserialize a buffer protocol compatible object into a pandas DataFrame. Parameters ---------- buf : buffer An object compatible with the buffer protocol. use_threads : bool, default True Whether to parallelize the conversion using multiple threads. Returns ------- df : pandas.DataFrame The buffer deserialized as pandas DataFrame Nri)r_ BufferReaderrread_all to_pandas)bufrj buffer_readerreadertables rdeserialize_pandasrs sOC((M #M 2 2"f!!""""""""""""""" ??{? 3 33sA  AAr<)(rLpyarrowr_ pyarrow.librrrrrrr r r r r rrrrrlib_RecordBatchStreamReaderrr6_RecordBatchStreamWriterr(_RecordBatchFileReaderr8_RecordBatchFileWriterr>r/rrTr5r%rVrXrZrhrsr&r rr{s( AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEEEEEc:EEE*).22222c:222=====C6===422222C6222>>>('''3744444  F !! $(T<<<<<.15d22222  F !! 2Tt222226&*$8,04444444r