Struct sql_ops::partition::cpu_radix_partition::CpuRadixPartitioner

pub struct CpuRadixPartitioner { /* private fields */ }

A CPU radix partitioner that provides partitioning functions.

Implementations

impl CpuRadixPartitioner

pub fn new(
    prefix_sum_algorithm: CpuHistogramAlgorithm,
    partition_algorithm: CpuRadixPartitionAlgorithm,
    radix_bits: u32,
    state_mem_type: DerefMemType
) -> Self

Creates a new CPU radix partitioner.

pub fn prefix_sum<T: DeviceCopy + CpuRadixPartitionable>(
    &mut self,
    partition_attr: RadixPartitionInputChunk<'_, T>,
    partition_offsets: PartitionOffsetsMutSlice<'_, Tuple<T, T>>
) -> Result<()>

Computes the prefix sum.

The prefix sum performs a scan over all partitioning keys. It first computes a histogram. The prefix sum is computed from this histogram.

The prefix sum serves two main purposes:

1. The prefix sums are used in partition as offsets in an array for the output partitions.
2. The prefix sum can also be used to detect skew in the data.

Parallelism

The function is thread-safe, and meant to be externally parallelized by the caller.

pub fn partition<T: DeviceCopy + CpuRadixPartitionable>(
    &mut self,
    partition_attr: RadixPartitionInputChunk<'_, T>,
    payload_attr: RadixPartitionInputChunk<'_, T>,
    partition_offsets: PartitionOffsetsMutSlice<'_, Tuple<T, T>>,
    partitioned_relation: PartitionedRelationMutSlice<'_, Tuple<T, T>>
) -> Result<()>

Radix-partitions a relation by its key attribute.

See the module-level documentation for details on the algorithm.

Trait Implementations

impl Debug for CpuRadixPartitioner

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.