Converts content of any type into an OptionType in which all values happen to be valid. More...

#include <UnmaskedArray.h>

Inheritance diagram for UnmaskedArray:

Public Member Functions
	UnmaskedArray (const IdentitiesPtr &identities, const util::Parameters &parameters, const ContentPtr &content)
	Creates an UnmaskedArray from a full set of parameters. More...

const ContentPtr	content () const
	Data to be converted to an OptionType. More...

const ContentPtr	project () const
	Equivalent to calling content; removes OptionType but there are no `None` values to remove. More...

const ContentPtr	project (const Index8 &mask) const
	Applies a given `mask` and calls project. More...

const Index8	bytemask () const
	Returns a byte mask of all `0` values (representing valid data). More...

const ContentPtr	simplify_optiontype () const
	If the content also has OptionType, return the content; otherwise, return shallow_copy. More...

const std::shared_ptr< ByteMaskedArray >	toByteMaskedArray () const
	Converts this array into a ByteMaskedArray. More...

const ContentPtr	toIndexedOptionArray64 () const
	Converts this array into an IndexedOptionArray without missing values. More...

const std::string	classname () const override
	User-friendly name of this class: `"UnmaskedArray"`. More...

void	setidentities () override
	Assign a surrogate index of Identities to this array (in-place). More...

void	setidentities (const IdentitiesPtr &identities) override
	Assign a specified set of Identities to this array (in-place). More...

const TypePtr	type (const util::TypeStrs &typestrs) const override
	High-level Type describing this array. More...

const FormPtr	form (bool materialize) const override
	Low-level Form describing all the features of this array except the actual data buffers (Index, Identities, NumpyArray::ptr, and RawArray::ptr). More...

kernel::lib	kernels () const override
	Returns the kernel library enum for all nested `ptr_lib` within the array's tree structure. If different libraries are associated with nodes of the same tree, this returns `kernel::lib::size` to indicate that it is "mixed". More...

void	caches (std::vector< ArrayCachePtr > &out) const override
	Accumulates all the unique #ArrayCache objects from nested #VirtualArray nodes. (Uniqueness is determined by pointer value.) More...

const std::string	tostring_part (const std::string &indent, const std::string &pre, const std::string &post) const override
	Internal function to build an output string for tostring. More...

void	tojson_part (ToJson &builder, bool include_beginendlist) const override
	Internal function to produce a JSON representation one node at a time. More...

void	nbytes_part (std::map< size_t, int64_t > &largest) const override
	Internal function used to calculate nbytes. More...

int64_t	length () const override
	The number of elements in the array. More...

const ContentPtr	shallow_copy () const override
	Copies this node without copying any nodes hierarchically nested within it or any array/index/identity buffers. More...

const ContentPtr	deep_copy (bool copyarrays, bool copyindexes, bool copyidentities) const override
	Copies this node and all nodes hierarchically nested within it, optionally copying the associated arrays, indexes, and identities, too. More...

void	check_for_iteration () const override
	Performs up-front validity checks on an array so that they don't have to be checked in getitem_at_nowrap for each item. More...

const ContentPtr	getitem_nothing () const override
	Internal function to get an empty slice (with the correct type). More...

const ContentPtr	getitem_at (int64_t at) const override
	Returns the element at a given position in the array, handling negative indexing and bounds-checking like Python. More...

const ContentPtr	getitem_at_nowrap (int64_t at) const override
	Returns the element at a given position in the array, without handling negative indexing or bounds-checking. More...

const ContentPtr	getitem_range (int64_t start, int64_t stop) const override
	Subinterval of this array, handling negative indexing and bounds-checking like Python. More...

const ContentPtr	getitem_range_nowrap (int64_t start, int64_t stop) const override
	Subinterval of this array, without handling negative indexing or bounds-checking. More...

const ContentPtr	getitem_field (const std::string &key) const override
	This array with the first nested RecordArray replaced by the field at `key`. More...

const ContentPtr	getitem_field (const std::string &key, const Slice &only_fields) const override

const ContentPtr	getitem_fields (const std::vector< std::string > &keys) const override
	This array with the first nested RecordArray replaced by a RecordArray of a given subset of `keys`. More...

const ContentPtr	getitem_fields (const std::vector< std::string > &keys, const Slice &only_fields) const override

const ContentPtr	getitem_next (const SliceItemPtr &head, const Slice &tail, const Index64 &advanced) const override
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

const ContentPtr	carry (const Index64 &carry, bool allow_lazy) const override
	Returns an array of the same type with elements filtered, rearranged, and possibly duplicated by the `carry` array of integers. More...

int64_t	purelist_depth () const override
	The list-depth of this array, not counting any contained within a RecordArray. More...

const std::pair< int64_t, int64_t >	minmax_depth () const override
	Returns (a) the minimum list-depth and (b) the maximum list-depth of the array, which can differ if this array "branches" (differs when followed through different fields of a RecordArray or UnionArray). More...

const std::pair< bool, int64_t >	branch_depth () const override
	Returns (a) whether the list-depth of this array "branches," or differs when followed through different fields of a RecordArray or UnionArray and (b) the minimum list-depth. More...

int64_t	numfields () const override
	The number of fields in the first nested tuple or records or `-1` if this array does not contain a RecordArray. More...

int64_t	fieldindex (const std::string &key) const override
	The position of a tuple or record key name if this array contains a RecordArray. More...

const std::string	key (int64_t fieldindex) const override
	The record name associated with a given field index or the tuple index as a string (e.g. `"0"`, `"1"`, `"2"`) if a tuple. More...

bool	haskey (const std::string &key) const override
	Returns `true` if the array contains a RecordArray with the specified `key`; `false` otherwise. More...

const std::vector< std::string >	keys () const override
	A list of RecordArray keys or an empty list if this array does not contain a RecordArray. More...

bool	istuple () const override
	Returns `true` if the outermost RecordArray is a tuple. More...

const std::string	validityerror (const std::string &path) const override
	Returns an error message if this array is invalid; otherwise, returns an empty string. More...

const ContentPtr	shallow_simplify () const override
	Returns an equivalent array simplified at one level only using simplify_optiontype if an option-type array and simplify_uniontype if a union-type array. More...

const ContentPtr	num (int64_t axis, int64_t depth) const override
	The length of this array (as a NumpyArray scalar) if `axis = 0` or the lengths of subarrays (as an array or nested array) if `axis != 0`. More...

const std::pair< Index64, ContentPtr >	offsets_and_flattened (int64_t axis, int64_t depth) const override
	Returns (a) an offsets Index and (b) a flattened version of the array at some `axis` depth. More...

bool	mergeable (const ContentPtr &other, bool mergebool) const override
	Returns `true` if this array can be merged with the `other`; `false` otherwise. More...

bool	referentially_equal (const ContentPtr &other) const override
	Returns `true` if this array has all the same buffers and parameters as `other`; `false` otherwise. More...

const ContentPtr	reverse_merge (const ContentPtr &other) const override
	Merges a single `other` with this array in reverse order: `other` first, this last. More...

const ContentPtr	mergemany (const ContentPtrVec &others) const override
	Returns an array with this and the `others` concatenated (in order, this first, `others` last). More...

const SliceItemPtr	asslice () const override
	Converts this array into a SliceItem that can be used in getitem. More...

const ContentPtr	fillna (const ContentPtr &value) const override
	Returns this array with `None` values replaced by a given `value`. More...

const ContentPtr	rpad (int64_t target, int64_t axis, int64_t depth) const override
	If `axis = 0`, returns a view of this array padded on the right with `None` values to have a minimum length; otherwise, returns an array with nested lists all padded to the minimum length. More...

const ContentPtr	rpad_and_clip (int64_t target, int64_t axis, int64_t depth) const override
	If `axis = 0`, returns a view of this array padded on the right. More...

const ContentPtr	reduce_next (const Reducer &reducer, int64_t negaxis, const Index64 &starts, const Index64 &shifts, const Index64 &parents, int64_t outlength, bool mask, bool keepdims) const override
	This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all). More...

const ContentPtr	sort_next (int64_t negaxis, const Index64 &starts, const Index64 &parents, int64_t outlength, bool ascending, bool stable) const override
	This array sorted. More...

const ContentPtr	argsort_next (int64_t negaxis, const Index64 &starts, const Index64 &shifts, const Index64 &parents, int64_t outlength, bool ascending, bool stable) const override
	This array sorted indices. More...

const ContentPtr	localindex (int64_t axis, int64_t depth) const override
	A (possibly nested) array of integers indicating the positions of elements within each nested list. More...

const ContentPtr	combinations (int64_t n, bool replacement, const util::RecordLookupPtr &recordlookup, const util::Parameters &parameters, int64_t axis, int64_t depth) const override
	Tuples or records of all `n`-tuple combinations of list items at some `axis` depth. More...

const ContentPtr	getitem_next (const SliceAt &at, const Slice &tail, const Index64 &advanced) const override
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

const ContentPtr	getitem_next (const SliceRange &range, const Slice &tail, const Index64 &advanced) const override
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

const ContentPtr	getitem_next (const SliceArray64 &array, const Slice &tail, const Index64 &advanced) const override
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

const ContentPtr	getitem_next (const SliceJagged64 &jagged, const Slice &tail, const Index64 &advanced) const override
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

const ContentPtr	getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceArray64 &slicecontent, const Slice &tail) const override
	Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...

const ContentPtr	getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceMissing64 &slicecontent, const Slice &tail) const override
	Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...

const ContentPtr	getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceJagged64 &slicecontent, const Slice &tail) const override
	Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...

const ContentPtr	copy_to (kernel::lib ptr_lib) const override
	Recursively copies components of the array from main memory to a GPU (if `ptr_lib == kernel::lib::cuda`) or to main memory (if `ptr_lib == kernel::lib::cpu`) if those components are not already there. More...

const ContentPtr	numbers_to_type (const std::string &name) const override
	Change the leaf types to 'totype'. More...

bool	is_unique () const override
	Returns 'true' if all components of the array are unique. More...

const ContentPtr	unique () const override
	Returns an array where all components are unique. More...

bool	is_subrange_equal (const Index64 &start, const Index64 &stop) const override
	Returns 'true' if subranges are equal. More...

Public Member Functions inherited from Content
	Content (const IdentitiesPtr &identities, const util::Parameters &parameters)
	Called by all subclass constructors; assigns identities and parameters upon construction. More...

virtual	~Content ()
	Empty destructor; required for some C++ reason. More...

virtual bool	isscalar () const
	Returns `true` if the data represented by this node is scalar, not a true array. More...

virtual const IdentitiesPtr	identities () const
	Optional Identities for each element of the array (may be `nullptr`). More...

virtual const ContentPtr	getitem (const Slice &where) const
	Entry point for general slicing: Slice represents a tuple of SliceItem nodes applying to each level of nested lists. More...

virtual const ContentPtr	getitem_next_jagged (const Index64 &slicestarts, const Index64 &slicestops, const SliceItemPtr &slicecontent, const Slice &tail) const
	Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis. More...

virtual const std::string	purelist_parameter (const std::string &key) const
	The parameter associated with `key` at the first level that has a non-null value, descending only as deep as the first RecordArray. More...

bool	purelist_isregular () const
	Returns `true` if all nested lists down to the first RecordArray are RegularArray nodes; `false` otherwise. More...

virtual bool	dimension_optiontype () const
	Returns `true` if this dimension has option-type; `false` otherwise. More...

virtual const std::pair< ContentPtrVec, ContentPtrVec >	merging_strategy (const ContentPtrVec &others) const
	Partitions `this` array plus a list of `others` into a `head` sequence and a `tail` sequence: More...

const ContentPtr	merge (const ContentPtr &other) const
	An array with this and the `other` concatenated (this first, `other` last). More...

const std::string	tostring () const
	Returns a string representation of this array (multi-line XML). More...

const std::string	tojson (bool pretty, int64_t maxdecimals, const char nan_string=nullptr, const char infinity_string=nullptr, const char minus_infinity_string=nullptr, const char complex_real_string=nullptr, const char *complex_imag_string=nullptr) const
	Returns a JSON representation of this array. More...

void	tojson (FILE destination, bool pretty, int64_t maxdecimals, int64_t buffersize, const char nan_string=nullptr, const char infinity_string=nullptr, const char minus_infinity_string=nullptr, const char complex_real_string=nullptr, const char complex_imag_string=nullptr) const
	Writes a JSON representation of this array to a `destination` file. More...

int64_t	nbytes () const
	The number of bytes contained in all array buffers, Index buffers, and Identities buffers, not including the lightweight node objects themselves. More...

const ContentPtr	reduce (const Reducer &reducer, int64_t axis, bool mask, bool keepdims) const
	This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all). More...

const ContentPtr	sort (int64_t axis, bool ascending, bool stable) const
	This array with one axis sorted by applying a sorting algorithm. More...

const ContentPtr	argsort (int64_t axis, bool ascending, bool stable) const
	This array indices with one axis sorted by applying a sorting algorithm. More...

const util::Parameters	parameters () const
	String-to-JSON map that augments the meaning of this array. More...

void	setparameters (const util::Parameters &parameters)
	Assign all the parameters for this array node (in-place). More...

const std::string	parameter (const std::string &key) const
	Get one parameter from this array node. More...

void	setparameter (const std::string &key, const std::string &value)
	Assign one parameter for this array node (in-place). More...

bool	parameter_equals (const std::string &key, const std::string &value) const
	Returns `true` if the parameter associated with `key` exists and is equal to `value`; `false` otherwise. More...

bool	parameters_equal (const util::Parameters &other, bool check_all) const
	Returns `true` if all parameters of this array node are equal to the `other` parameters. More...

bool	parameter_isstring (const std::string &key) const
	Returns `true` if the parameter associated with `key` is a string; `false` otherwise. More...

bool	parameter_isname (const std::string &key) const
	Returns `true` if the parameter associated with `key` is a string that matches `[A-Za-z_][A-Za-z_0-9]*`; `false` otherwise. More...

const std::string	parameter_asstring (const std::string &key) const
	Returns the parameter associated with `key` as a string if parameter_isstring; raises an error otherwise. More...

const ContentPtr	merge_as_union (const ContentPtr &other) const
	Concatenates this array with `other` by creating a UnionArray instead of actually merging the data. This does not call UnionArray::simplify_uniontype. More...

const ContentPtr	rpad_axis0 (int64_t target, bool clip) const
	Internal function to handle the `axis = 0` case of rpad and rpad_and_clip. More...

const ContentPtr	localindex_axis0 () const
	Internal function to handle the `axis = 0` case of localindex. More...

const ContentPtr	combinations_axis0 (int64_t n, bool replacement, const util::RecordLookupPtr &recordlookup, const util::Parameters &parameters) const
	Internal function to handle the `axis = 0` case of combinations. More...

virtual const ContentPtr	getitem_next (const SliceEllipsis &ellipsis, const Slice &tail, const Index64 &advanced) const
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

virtual const ContentPtr	getitem_next (const SliceNewAxis &newaxis, const Slice &tail, const Index64 &advanced) const
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

virtual const ContentPtr	getitem_next (const SliceField &field, const Slice &tail, const Index64 &advanced) const
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

virtual const ContentPtr	getitem_next (const SliceFields &fields, const Slice &tail, const Index64 &advanced) const
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

virtual const ContentPtr	getitem_next (const SliceMissing64 &missing, const Slice &tail, const Index64 &advanced) const
	Internal function that propagates a generic getitem request through one axis (including advanced indexing). More...

const int64_t	axis_wrap_if_negative (int64_t axis) const
	Internal function defining the negative axis handling for many operations. More...

Protected Member Functions
template<typename S >
const ContentPtr	getitem_next_jagged_generic (const Index64 &slicestarts, const Index64 &slicestops, const S &slicecontent, const Slice &tail) const

Protected Member Functions inherited from Content
const std::string	validityerror_parameters (const std::string &path) const
	Internal function to determine if a particular set of parameters are allowed on a particular node (centralized by parameter, not by class). More...

const ContentPtr	getitem_next_array_wrap (const ContentPtr &outcontent, const std::vector< int64_t > &shape) const
	Internal function to wrap putative getitem output with enough RegularArray nodes to satisfy a given `shape`. More...

const std::string	parameters_tostring (const std::string &indent, const std::string &pre, const std::string &post) const
	Internal function to convert parameters into a string fragment for tostring. More...

kernel::lib	kernels_compare (kernel::lib from_index, const ContentPtr &content) const
	Internal function to support most of the logic in kernels. More...

Additional Inherited Members
Protected Attributes inherited from Content
IdentitiesPtr	identities_
	See identities. More...

util::Parameters	parameters_
	See parameters. More...

Detailed Description

Converts content of any type into an OptionType in which all values happen to be valid.

See UnmaskedArray for the meaning of each parameter.

Constructor & Destructor Documentation

◆ UnmaskedArray()

UnmaskedArray	(	const IdentitiesPtr &	identities,
		const util::Parameters &	parameters,
		const ContentPtr &	content
	)

Creates an UnmaskedArray from a full set of parameters.

Parameters

identities	Optional Identities for each element of the array (may be `nullptr`).
parameters	String-to-JSON map that augments the meaning of this array.
content	Data to be converted to an OptionType.

Member Function Documentation

◆ argsort_next()

const ContentPtr argsort_next	(	int64_t	negaxis,
		const Index64 &	starts,
		const Index64 &	shifts,
		const Index64 &	parents,
		int64_t	outlength,
		bool	ascending,
		bool	stable
	)		const

overridevirtual

This array sorted indices.

The user's entry point for this operation is argsort.

Parameters

negaxis	The negative axis: `-axis`. That is, `negaxis = 1` means the deepest axis level.
starts	Staring positions of each group to combine as an Index. These are downward pointers from an outer structure into this structure with the same meaning as in ListArray.
shifts	Per-element adjustments that allows for variable-length lists with axis != -1 and for missing values (None).
parents	Groups to combine as an Index of upward pointers from this structure to the outer structure to sort.
outlength	The length of the array, after the operation completes.
ascending	If `true`, the values will be sorted in an ascending order.
stable	If `true`, the values will be sorted by a stable sort algorithm to maintain the relative order of records with equal keys (i.e. values).

Implements Content.

◆ asslice()

const SliceItemPtr asslice ( ) const

overridevirtual

Converts this array into a SliceItem that can be used in getitem.

Implements Content.

◆ branch_depth()

const std::pair<bool, int64_t> branch_depth ( ) const

overridevirtual

Returns (a) whether the list-depth of this array "branches," or differs when followed through different fields of a RecordArray or UnionArray and (b) the minimum list-depth.

If the array does not contain any records or heterogeneous data, the first element is always true and the second is simply the depth.

Implements Content.

◆ bytemask()

const Index8 bytemask ( ) const

Returns a byte mask of all 0 values (representing valid data).

◆ caches()

void caches ( std::vector< ArrayCachePtr > & out ) const

overridevirtual

Accumulates all the unique #ArrayCache objects from nested #VirtualArray nodes. (Uniqueness is determined by pointer value.)

Implements Content.

◆ carry()

const ContentPtr carry	(	const Index64 &	carry,
		bool	allow_lazy
	)		const

overridevirtual

Returns an array of the same type with elements filtered, rearranged, and possibly duplicated by the carry array of integers.

The output has the same length as the carry index, not the array that is being manipulated. For each item i in carry, the output is array[index[i]].

This operation is called take in NumPy and Arrow, although this carry is a low-level function that does not handle negative indexes and is not exposed to the Python layer. It is used by many operations to pass filters/rearrangements/duplications from one typed array node to another without knowing the latter's type.

Taking getitem_at_nowrap as a function from integers to the array's item type, A: [0, len(a)) → T, and the carry array's Index64::getitem_at_nowrap as a function C: [0, len(c)) → [0, len(a)), this method represents function composition, A ∘ C: [0, len(c)) → T.

Note: If the carry array has duplicate elements, the array is only duplicated one level deep. For instance, on a ListArray, only the starts and stops get duplicated, not the content (and similarly for all other node types).

Implements Content.

◆ check_for_iteration()

void check_for_iteration ( ) const

overridevirtual

Performs up-front validity checks on an array so that they don't have to be checked in getitem_at_nowrap for each item.

Implements Content.

◆ classname()

const std::string classname ( ) const

overridevirtual

User-friendly name of this class: "UnmaskedArray".

Implements Content.

◆ combinations()

const ContentPtr combinations	(	int64_t	n,
		bool	replacement,
		const util::RecordLookupPtr &	recordlookup,
		const util::Parameters &	parameters,
		int64_t	axis,
		int64_t	depth
	)		const

overridevirtual

Tuples or records of all n-tuple combinations of list items at some axis depth.

For example, the n = 2 combinations at axis = 0 of

[a, b, c, d, e]

would be

[[(a, b), (a, c), (a, d), (a, e)],
 [(b, c), (b, d), (b, e)],
 [(c, d), (c, e)],
 [(d, e)]]

and the n = 3 combinations at axis = 1 of

[[a, b, c, d], [], [e, f], [g, h, i]]

would be

[[[(a, b, c), (a, b, d), (a, c, d)], [(b, c, d)]],
 [],
 [],
 [[(g, h, i)]]]

Parameters

n	The number of items in each tuple/record.
replacement	If `true`, the tuples/records are allowed to include the same item more than once, such as `(a, a, a)` and `(a, a, b)`. In the above examples, `replacement = false`. (If the output of this function is thought of as the "upper triangle" elements of the Cartesian product of the input with itself `n` times, the `replacement` parameter determines whether elements on the diagonal are allowed.)
recordlookup	If `nullptr`, the output consists of tuples, a RecordArray indexed by `"0"`, `"1"`, `"2"`, etc. If `recordlookup` is a `std::vector<std::string>`, the output consists of records, a RecordArray indexed by names (strings). The length of `recordlookup` must be equal to `n`.
parameters	Parameters assigned to the new RecordArray. This can be used to set `"__record_" = "\"record_name\""` to give the records a custom behavior in Python.
axis	The nesting depth at which this operation is applied. At `axis = 0`, combinations are formed from elements of the whole array (see example above); at `axis = 1`, combinations are formed independently for each element. At a deeper `axis`, outer levels are left untouched. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
depth	The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Note that axis = 0 is qualitatively different from any other axis because a dataset is typically much larger than any one of its elements. As such, axis = 0 is lazily generated with an IndexedArray, while any other axis is eagerly generated by duplicating elements (with carry).

Implements Content.

◆ content()

const ContentPtr content ( ) const

Data to be converted to an OptionType.

◆ copy_to()

const ContentPtr copy_to ( kernel::lib ptr_lib ) const

overridevirtual

Recursively copies components of the array from main memory to a GPU (if ptr_lib == kernel::lib::cuda) or to main memory (if ptr_lib == kernel::lib::cpu) if those components are not already there.

Implements Content.

◆ deep_copy()

const ContentPtr deep_copy	(	bool	copyarrays,
		bool	copyindexes,
		bool	copyidentities
	)		const

overridevirtual

Copies this node and all nodes hierarchically nested within it, optionally copying the associated arrays, indexes, and identities, too.

◆ fieldindex()

int64_t fieldindex ( const std::string & key ) const

overridevirtual

The position of a tuple or record key name if this array contains a RecordArray.

Implements Content.

◆ fillna()

const ContentPtr fillna ( const ContentPtr & value ) const

overridevirtual

Returns this array with None values replaced by a given value.

Parameters

value An array of exactly one element, which need not have the same type as the missing values it's replacing.

Implements Content.

◆ form()

const FormPtr form ( bool materialize ) const

overridevirtual

Low-level Form describing all the features of this array except the actual data buffers (Index, Identities, NumpyArray::ptr, and RawArray::ptr).

Parameters

materialize If true, materialize any VirtualArrays encountered along the way.

Implements Content.

◆ getitem_at()

const ContentPtr getitem_at ( int64_t at ) const

overridevirtual

Returns the element at a given position in the array, handling negative indexing and bounds-checking like Python.

The first item in the array is at 0, the second at 1, the last at -1, the penultimate at -2, etc.

Implements Content.

◆ getitem_at_nowrap()

const ContentPtr getitem_at_nowrap ( int64_t at ) const

overridevirtual

Returns the element at a given position in the array, without handling negative indexing or bounds-checking.

If the array has Identities, the identity bounds are checked.

Implements Content.

◆ getitem_field() [1/2]

const ContentPtr getitem_field ( const std::string & key ) const

overridevirtual

This array with the first nested RecordArray replaced by the field at key.

Implements Content.

◆ getitem_field() [2/2]

const ContentPtr getitem_field	(	const std::string &	key,
		const Slice &	only_fields
	)		const

overridevirtual

Implements Content.

◆ getitem_fields() [1/2]

const ContentPtr getitem_fields ( const std::vector< std::string > & keys ) const

overridevirtual

This array with the first nested RecordArray replaced by a RecordArray of a given subset of keys.

Implements Content.

◆ getitem_fields() [2/2]

const ContentPtr getitem_fields	(	const std::vector< std::string > &	keys,
		const Slice &	only_fields
	)		const

overridevirtual

Implements Content.

◆ getitem_next() [1/5]

const ContentPtr getitem_next	(	const SliceItemPtr &	head,
		const Slice &	tail,
		const Index64 &	advanced
	)		const

overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

Parameters

head	First element of the Slice tuple.
tail	The rest of the elements of the Slice tuple.
advanced	If empty, no array slices (integer or boolean) have been encountered yet; otherwise, positions in any subsequent array slices to select.

In the NumPy documentation, advanced indexes are described as iterating "as one," which requires an Index to be propagated when implemented recursively.

Reimplemented from Content.

◆ getitem_next() [2/5]

const ContentPtr getitem_next	(	const SliceAt &	at,
		const Slice &	tail,
		const Index64 &	advanced
	)		const

overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

◆ getitem_next() [3/5]

const ContentPtr getitem_next	(	const SliceRange &	range,
		const Slice &	tail,
		const Index64 &	advanced
	)		const

overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

◆ getitem_next() [4/5]

const ContentPtr getitem_next	(	const SliceArray64 &	array,
		const Slice &	tail,
		const Index64 &	advanced
	)		const

overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

◆ getitem_next() [5/5]

const ContentPtr getitem_next	(	const SliceJagged64 &	jagged,
		const Slice &	tail,
		const Index64 &	advanced
	)		const

overridevirtual

Internal function that propagates a generic getitem request through one axis (including advanced indexing).

See generic getitem_next for details.

Implements Content.

◆ getitem_next_jagged() [1/3]

const ContentPtr getitem_next_jagged	(	const Index64 &	slicestarts,
		const Index64 &	slicestops,
		const SliceArray64 &	slicecontent,
		const Slice &	tail
	)		const

overridevirtual

Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis.

See generic getitem_next_jagged for details.

Implements Content.

◆ getitem_next_jagged() [2/3]

const ContentPtr getitem_next_jagged	(	const Index64 &	slicestarts,
		const Index64 &	slicestops,
		const SliceMissing64 &	slicecontent,
		const Slice &	tail
	)		const

overridevirtual

Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis.

See generic getitem_next_jagged for details.

Implements Content.

◆ getitem_next_jagged() [3/3]

const ContentPtr getitem_next_jagged	(	const Index64 &	slicestarts,
		const Index64 &	slicestops,
		const SliceJagged64 &	slicecontent,
		const Slice &	tail
	)		const

overridevirtual

Internal function that propagates a jagged array (array with irregular-length dimensions) slice through one axis.

See generic getitem_next_jagged for details.

Implements Content.

◆ getitem_next_jagged_generic()

const ContentPtr getitem_next_jagged_generic	(	const Index64 &	slicestarts,
		const Index64 &	slicestops,
		const S &	slicecontent,
		const Slice &	tail
	)		const

protected

◆ getitem_nothing()

const ContentPtr getitem_nothing ( ) const

overridevirtual

Internal function to get an empty slice (with the correct type).

Implements Content.

◆ getitem_range()

const ContentPtr getitem_range	(	int64_t	start,
		int64_t	stop
	)		const

overridevirtual

Subinterval of this array, handling negative indexing and bounds-checking like Python.

The first item in the array is at 0, the second at 1, the last at -1, the penultimate at -2, etc.

Ranges beyond the array are not an error; they are trimmed to start = 0 on the left and stop = length() - 1 on the right.

This operation only affects the node metadata; its calculation time does not scale with the size of the array.

Implements Content.

◆ getitem_range_nowrap()

const ContentPtr getitem_range_nowrap	(	int64_t	start,
		int64_t	stop
	)		const

overridevirtual

Subinterval of this array, without handling negative indexing or bounds-checking.

If the array has Identities, the identity bounds are checked.

This operation only affects the node metadata; its calculation time does not scale with the size of the array.

Implements Content.

◆ haskey()

bool haskey ( const std::string & key ) const

overridevirtual

Returns true if the array contains a RecordArray with the specified key; false otherwise.

Implements Content.

◆ is_subrange_equal()

bool is_subrange_equal	(	const Index64 &	start,
		const Index64 &	stop
	)		const

overridevirtual

Returns 'true' if subranges are equal.

Implements Content.

◆ is_unique()

bool is_unique ( ) const

overridevirtual

Returns 'true' if all components of the array are unique.

Implements Content.

◆ istuple()

bool istuple ( ) const

overridevirtual

Returns true if the outermost RecordArray is a tuple.

Implements Content.

◆ kernels()

kernel::lib kernels ( ) const

overridevirtual

Returns the kernel library enum for all nested ptr_lib within the array's tree structure. If different libraries are associated with nodes of the same tree, this returns kernel::lib::size to indicate that it is "mixed".

Implements Content.

◆ key()

const std::string key ( int64_t fieldindex ) const

overridevirtual

The record name associated with a given field index or the tuple index as a string (e.g. "0", "1", "2") if a tuple.

Raises an error if the array does not contain a RecordArray.

Implements Content.

◆ keys()

const std::vector<std::string> keys ( ) const

overridevirtual

A list of RecordArray keys or an empty list if this array does not contain a RecordArray.

Implements Content.

◆ length()

int64_t length ( ) const

overridevirtual

The number of elements in the array.

Implements Content.

◆ localindex()

const ContentPtr localindex	(	int64_t	axis,
		int64_t	depth
	)		const

overridevirtual

A (possibly nested) array of integers indicating the positions of elements within each nested list.

Parameters

axis The nesting depth at which this operation is applied. If axis = 0, the output is simply an array of integers from 0 to length(). If axis = 1, the output has one level of nesting containing integers from 0 to the length of the nested list. Higher values of axis leave outer layers of the structure untouched. Negative axis counts backward from the deepest levels (-1 is the last valid axis).

depth The current depth while stepping into the array: this value is set to 0 on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified axis).

Implements Content.

◆ mergeable()

bool mergeable	(	const ContentPtr &	other,
		bool	mergebool
	)		const

overridevirtual

Returns true if this array can be merged with the other; false otherwise.

The merge method will complete without errors if this function returns true.

Parameters

other	The other array to merge with.
mergebool	If `true`, consider boolean types to be equivalent to integers.

Implements Content.

◆ mergemany()

const ContentPtr mergemany ( const ContentPtrVec & others ) const

overridevirtual

Returns an array with this and the others concatenated (in order, this first, others last).

Implements Content.

◆ minmax_depth()

const std::pair<int64_t, int64_t> minmax_depth ( ) const

overridevirtual

Returns (a) the minimum list-depth and (b) the maximum list-depth of the array, which can differ if this array "branches" (differs when followed through different fields of a RecordArray or UnionArray).

Implements Content.

◆ nbytes_part()

void nbytes_part ( std::map< size_t, int64_t > & largest ) const

overridevirtual

Internal function used to calculate nbytes.

Parameters

largest The largest range of bytes used in each reference-counted pointer (size_t).

Note: This method of accounting for overlapping buffers is insufficient: two nodes could use parts of the same buffer in which one doesn't completely overlap the other. It's not likely, but currently the calculation of nbytes is an underestimate.

Implements Content.

◆ num()

const ContentPtr num	(	int64_t	axis,
		int64_t	depth
	)		const

overridevirtual

The length of this array (as a NumpyArray scalar) if axis = 0 or the lengths of subarrays (as an array or nested array) if axis != 0.

Parameters

axis	The axis whose length or lengths to quantify. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
depth	The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

◆ numbers_to_type()

const ContentPtr numbers_to_type ( const std::string & name ) const

overridevirtual

Change the leaf types to 'totype'.

Implements Content.

◆ numfields()

int64_t numfields ( ) const

overridevirtual

The number of fields in the first nested tuple or records or -1 if this array does not contain a RecordArray.

Implements Content.

◆ offsets_and_flattened()

const std::pair<Index64, ContentPtr> offsets_and_flattened	(	int64_t	axis,
		int64_t	depth
	)		const

overridevirtual

Returns (a) an offsets Index and (b) a flattened version of the array at some axis depth.

If axis > 1 (or its negative equivalent), the offsets is empty.

Parameters

axis	The axis to eliminate by flattening. `axis = 0` is invalid. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
depth	The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

◆ project() [1/2]

const ContentPtr project ( ) const

Equivalent to calling content; removes OptionType but there are no None values to remove.

◆ project() [2/2]

const ContentPtr project ( const Index8 & mask ) const

Applies a given mask and calls project.

Parameters

mask	A byte mask that is valid when `0`, `None` when `1`.

◆ purelist_depth()

int64_t purelist_depth ( ) const

overridevirtual

The list-depth of this array, not counting any contained within a RecordArray.

The purelist_depth of a Record is 0, and a RecordArray is 1 (regardless of what its fields contain).

If this array contains a UnionArray with different depths, the return value is -1.

Implements Content.

◆ reduce_next()

const ContentPtr reduce_next	(	const Reducer &	reducer,
		int64_t	negaxis,
		const Index64 &	starts,
		const Index64 &	shifts,
		const Index64 &	parents,
		int64_t	outlength,
		bool	mask,
		bool	keepdims
	)		const

overridevirtual

This array with one axis removed by applying a Reducer (e.g. "sum", "max", "any", "all).

The user's entry point for this operation is reduce.

Parameters

reducer	The choice of Reducer algorithm.
negaxis	The negative axis: `-axis`. That is, `negaxis = 1` means the deepest axis level.
starts	Staring positions of each group to combine as an Index. These are downward pointers from an outer structure into this structure with the same meaning as in ListArray. Only used by a reducer that returns_positions (currently only argmin and argmax).
shifts	Per-element adjustments for any reducer that returns_positions. Allows for variable-length lists with axis != -1 and for missing values (None) in argmin and argmax.
parents	Groups to combine as an Index of upward pointers from this structure to the outer structure to reduce.
outlength	The length of the reduced array, after the operation completes.
mask	If `true`, the Reducer's identity values will be covered by `None` using a ByteMaskedArray. This is desirable for ReducerMin, ReducerMax, ReducerArgmin, and ReducerArgmax to indicate that empty lists have no minimum or maximum.
keepdims	If `true`, the reduced values will be wrapped by a singleton RegularArray to maintain the same number of dimensions in the output.

Implements Content.

◆ referentially_equal()

bool referentially_equal ( const ContentPtr & other ) const

overridevirtual

Returns true if this array has all the same buffers and parameters as other; false otherwise.

Parameters

other The array to compare this with.

Implements Content.

◆ reverse_merge()

const ContentPtr reverse_merge ( const ContentPtr & other ) const

overridevirtual

Merges a single other with this array in reverse order: other first, this last.

Only arrays that need to be reversible have this function: option-type and union-type arrays. Others raise a runtime error.

Reimplemented from Content.

◆ rpad()

const ContentPtr rpad	(	int64_t	target,
		int64_t	axis,
		int64_t	depth
	)		const

overridevirtual

If axis = 0, returns a view of this array padded on the right with None values to have a minimum length; otherwise, returns an array with nested lists all padded to the minimum length.

Parameters

target	The intended length. The output may be longer than this target length, but not shorter (using ListArray).
axis	The axis at which to apply padding. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
depth	The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

◆ rpad_and_clip()

const ContentPtr rpad_and_clip	(	int64_t	target,
		int64_t	axis,
		int64_t	depth
	)		const

overridevirtual

If axis = 0, returns a view of this array padded on the right.

with None values to have exactly the specified length; otherwise, returns an array with nested lists all padded to the specified length.

Parameters

target	The intended length; the output has exactly this target length (using RegularArray).
axis	The axis at which to apply padding. Negative `axis` counts backward from the deepest levels (`-1` is the last valid `axis`).
depth	The current depth while stepping into the array: this value is set to `0` on the array node where the user starts the process and is increased at each level of list-depth (instead of decreasing the user-specified `axis`).

Implements Content.

◆ setidentities() [1/2]

void setidentities ( )

overridevirtual

Assign a surrogate index of Identities to this array (in-place).

This also assigns and possibly replaces Identities in nested arrays.

Note: This mutability is temporary: scikit-hep/awkward-1.0#116 Eventually, this interface will be deprecated and all Content instances will be immutable.

Implements Content.

◆ setidentities() [2/2]

void setidentities ( const IdentitiesPtr & identities )

overridevirtual

Assign a specified set of Identities to this array (in-place).

This also assigns and possibly replaces Identities in nested arrays.

Note: This mutability is temporary: scikit-hep/awkward-1.0#116 Eventually, this interface will be deprecated and all Content instances will be immutable.

Implements Content.

◆ shallow_copy()

const ContentPtr shallow_copy ( ) const

overridevirtual

Copies this node without copying any nodes hierarchically nested within it or any array/index/identity buffers.

◆ shallow_simplify()

const ContentPtr shallow_simplify ( ) const

overridevirtual

Returns an equivalent array simplified at one level only using simplify_optiontype if an option-type array and simplify_uniontype if a union-type array.

For all other types of arrays, this operation is a pass-through.

For UnmaskedArray, this method returns simplify_optiontype.

Implements Content.

◆ simplify_optiontype()

const ContentPtr simplify_optiontype ( ) const

If the content also has OptionType, return the content; otherwise, return shallow_copy.

This is a shallow operation: it only checks the content one level deep.

◆ sort_next()

const ContentPtr sort_next	(	int64_t	negaxis,
		const Index64 &	starts,
		const Index64 &	parents,
		int64_t	outlength,
		bool	ascending,
		bool	stable
	)		const

overridevirtual

This array sorted.

The user's entry point for this operation is sort.

Parameters

negaxis	The negative axis: `-axis`. That is, `negaxis = 1` means the deepest axis level.
starts	Staring positions of each group to combine as an Index. These are downward pointers from an outer structure into this structure with the same meaning as in ListArray.
parents	Groups to combine as an Index of upward pointers from this structure to the outer structure to sort.
outlength	The length of the array, after the operation completes.
ascending	If `true`, the values will be sorted in an ascending order.
stable	If `true`, the values will be sorted by a stable sort algorithm to maintain the relative order of records with equal keys (i.e. values).

Implements Content.

◆ toByteMaskedArray()

const std::shared_ptr<ByteMaskedArray> toByteMaskedArray ( ) const

Converts this array into a ByteMaskedArray.

◆ toIndexedOptionArray64()

const ContentPtr toIndexedOptionArray64 ( ) const

Converts this array into an IndexedOptionArray without missing values.

◆ tojson_part()

void tojson_part	(	ToJson &	builder,
		bool	include_beginendlist
	)		const

overridevirtual

Internal function to produce a JSON representation one node at a time.

Implements Content.

◆ tostring_part()

const std::string tostring_part	(	const std::string &	indent,
		const std::string &	pre,
		const std::string &	post
	)		const

overridevirtual

Internal function to build an output string for tostring.

Parameters

indent	Indentation depth as a string of spaces.
pre	Prefix string, usually an opening XML tag.
post	Postfix string, usually a closing XML tag and carriage return.

Implements Content.

◆ type()

const TypePtr type ( const util::TypeStrs & typestrs ) const

overridevirtual

High-level Type describing this array.

Parameters

typestrs A mapping from "__record__" parameters to string representations of those types, to override the derived strings.

Implements Content.

◆ unique()

const ContentPtr unique ( ) const

overridevirtual

Returns an array where all components are unique.

Implements Content.

◆ validityerror()

const std::string validityerror ( const std::string & path ) const

overridevirtual

Returns an error message if this array is invalid; otherwise, returns an empty string.

Implements Content.

The documentation for this class was generated from the following file:

include/awkward/array/UnmaskedArray.h

copyarrays	If `true`, copy the associated array buffers (in NumpyArray and RawArray), not just the lightweight objects that point to them.
copyindexes	If `true`, copy the Index objects and their buffers as well.
copyidentities	If `true`, copy the Identities objects and their buffers as well.

Public Member Functions

Protected Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ UnmaskedArray()

Member Function Documentation

◆ argsort_next()

◆ asslice()

◆ branch_depth()

◆ bytemask()

◆ caches()

◆ carry()

◆ check_for_iteration()

◆ classname()

◆ combinations()

◆ content()

◆ copy_to()

◆ deep_copy()

◆ fieldindex()

◆ fillna()

◆ form()

◆ getitem_at()

◆ getitem_at_nowrap()

◆ getitem_field() [1/2]

◆ getitem_field() [2/2]

◆ getitem_fields() [1/2]

◆ getitem_fields() [2/2]

◆ getitem_next() [1/5]

◆ getitem_next() [2/5]

◆ getitem_next() [3/5]

◆ getitem_next() [4/5]

◆ getitem_next() [5/5]

◆ getitem_next_jagged() [1/3]

◆ getitem_next_jagged() [2/3]

◆ getitem_next_jagged() [3/3]

◆ getitem_next_jagged_generic()

◆ getitem_nothing()

◆ getitem_range()

◆ getitem_range_nowrap()

◆ haskey()

◆ is_subrange_equal()

◆ is_unique()

◆ istuple()

◆ kernels()

◆ key()

◆ keys()

◆ length()

◆ localindex()

◆ mergeable()

◆ mergemany()

◆ minmax_depth()

◆ nbytes_part()

◆ num()

◆ numbers_to_type()

◆ numfields()

◆ offsets_and_flattened()

◆ project() [1/2]

◆ project() [2/2]

◆ purelist_depth()

◆ reduce_next()

◆ referentially_equal()

◆ reverse_merge()

◆ rpad()

◆ rpad_and_clip()

◆ setidentities() [1/2]

◆ setidentities() [2/2]

◆ shallow_copy()

◆ shallow_simplify()

◆ simplify_optiontype()

◆ sort_next()

◆ toByteMaskedArray()

◆ toIndexedOptionArray64()

◆ tojson_part()

◆ tostring_part()

◆ type()

◆ unique()

◆ validityerror()