ak.to_arrow¶
Defined in awkward.operations.convert on line 1986.
- ak.to_arrow(array, list_to32=False, string_to32=True, bytestring_to32=True, allow_tensor=True)¶
- Parameters
array – Data to convert to an Apache Arrow array.
list_to32 (bool) – If True, convert Awkward lists into 32-bit Arrow lists if they’re small enough, even if it means an extra conversion. Otherwise, signed 32-bit
ak.layout.ListOffsetArray
maps to ArrowListType
and all others map to ArrowLargeListType
.string_to32 (bool) – Same as the above for Arrow
string
andlarge_string
.bytestring_to32 (bool) – Same as the above for Arrow
binary
andlarge_binary
.allow_tensor (bool) – If True, convert regular-length lists to
pyarrow.lib.Tensor
; otherwise, makepyarrow.lib.ListArray
(generating offsets). This is used byak.to_parquet
, since Parquet files can’t contain regular-length tensors.
Converts an Awkward Array into an Apache Arrow array.
This produces arrays of type pyarrow.Array
. You might need to further
manipulations (using the pyarrow library) to build a pyarrow.ChunkedArray
,
a pyarrow.RecordBatch
, or a pyarrow.Table
.
Arrow arrays can maintain the distinction between “option-type but no elements are
missing” and “not option-type” at all levels except the top level. Also, there is
no distinction between ?union[X, Y, Z]]
type and union[?X, ?Y, ?Z]
type. Be
aware of these type distinctions when passing data through Arrow or Parquet.
See also ak.from_arrow
, ak.to_arrow_table
, ak.to_parquet
.