Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

The three domains

The underlying question

A recursion is, at its heart, five closures — a fold’s init, accumulate, and finalize, a graph’s edge function, and (in seed pipelines) a grow. hylic retains these closures across the duration of a run and hands them to executors, lifts, and user code. A single question organises the design:

How shall dyn Fn(&N) -> H be stored?

Rust offers three practical answers:

StorageClone?Send + Sync?
Arc<dyn>cheap (refcount bump)yes, if the closure is
Rc<dyn>cheap (refcount bump)no (single-threaded)
Box<dyn>not Clonepossible, but consumed on use

Each choice is a compromise. Arc pays an atomic instruction on every clone in exchange for the ability to cross thread boundaries. Rc uses a plain counter — faster single-threaded, incompatible with multi-threading. Box avoids any counter but forces transformation pipelines to consume the closure on each rewrite.

Every closure in a recursion must agree on the choice. hylic therefore selects once at the top level and propagates the choice through the entire pipeline; that selection is what is called a domain.

Three choices, three types
%3cluster_SSharedcluster_LLocalcluster_OOwnedarcArc<dyn Fn + Send + Sync>rcRc<dyn Fn>box_Box<dyn Fn>

  • Shared stores closures behind Arc with Send + Sync bounds. The atomic clone grants access to the parallel Funnel executor and makes every pipeline Clone.
  • Local stores closures behind Rc (no Send bound). Clones remain cheap and the pipeline interfaces are unchanged, but execution is confined to a single thread. In return, captures may include Rc<_>, RefCell<_>, or any non-Send type.
  • Owned stores closures in Box. Clones and sharing are both absent; each stage of a pipeline consumes its predecessor. Appropriate for one-shot computations that should avoid reference-counting overhead entirely.

Shared is the conservative default and serves most code. Local is the escape hatch for non-Send captures. Owned is the minimalist one-shot variant.

The Domain trait

The three choices are encoded as marker types implementing the Domain<N> trait:

#![allow(unused)]
fn main() {
pub trait Domain<N: 'static>: 'static {
    type Fold<H: 'static, R: 'static>: FoldOps<N, H, R>;
    type Graph<E: 'static> where E: 'static;
    type Grow<Seed: 'static, NOut: 'static>;

    /// Construct a fold from three closures. Uniform Send+Sync
    /// bound; each domain sheds Send+Sync at storage time if it
    /// doesn't need it.
    fn make_fold<H: 'static, R: 'static>(
        init: impl Fn(&N) -> H + Send + Sync + 'static,
        acc:  impl Fn(&mut H, &R) + Send + Sync + 'static,
        fin:  impl Fn(&H) -> R + Send + Sync + 'static,
    ) -> Self::Fold<H, R>;

    /// Construct a grow closure from a Fn. Uniform Send+Sync bound.
    fn make_grow<Seed: 'static, NOut: 'static>(
        f: impl Fn(&Seed) -> NOut + Send + Sync + 'static,
    ) -> Self::Grow<Seed, NOut>;

    /// Invoke a stored grow closure.
    fn invoke_grow<Seed: 'static, NOut: 'static>(
        g: &Self::Grow<Seed, NOut>,
        s: &Seed,
    ) -> NOut;

    /// Construct a graph (Edgy) closure. Uniform Send+Sync bound.
    fn make_graph<E: 'static>(
        visit: impl Fn(&N, &mut dyn FnMut(&E)) + Send + Sync + 'static,
    ) -> Self::Graph<E>;
}
}

A Domain<N> implementation specifies:

  • the concrete Fold<H, R> type in use (closure storage lives inside Fold),
  • the concrete Graph type,
  • the concrete Grow<Seed, N> type (for seed pipelines),
  • constructor methods (make_fold, make_graph, make_grow) that build each of the above generically.

Code generic over D: Domain<N> constructs any of the three without knowing whether the underlying storage is Arc, Rc, or Box; the constructor methods handle the distinction.

Constructors across domains

Each domain exposes the same construction surface, distinguished only by its bounds:

#![allow(unused)]
fn main() {
    #[test]
    fn domains_three_folds() {
        // Shared: closures must be Send + Sync (they go into Arc).
        let _shared = hylic::domain::shared::fold(
            |n: &u64| *n,                     // init
            |h: &mut u64, c: &u64| *h += c,   // accumulate
            |h: &u64| *h,                     // finalize
        );

        // Local: closures can capture Rc / RefCell.
        use std::cell::RefCell;
        use std::rc::Rc;
        let state = Rc::new(RefCell::new(0u32));
        let state_for_init = state.clone();
        let _local = hylic::domain::local::fold(
            move |n: &u64| { *state_for_init.borrow_mut() += 1; *n },
            |h: &mut u64, c: &u64| *h += c,
            |h: &u64| *h,
        );

        // Owned: one-shot construction; not Clone.
        let _owned = hylic::domain::owned::fold(
            |n: &u64| *n,
            |h: &mut u64, c: &u64| *h += c,
            |h: &u64| *h,
        );
    }
}

The Shared constructor requires Fn + Send + Sync + 'static for every closure; Local requires Fn + 'static; Owned shares Local’s bounds but returns a Box-backed struct. The signatures are aligned so that generic code compiles without modification across domains; the bounds differ so that each domain accepts only those closures it is able to store.

The Fold struct, three times

Because storage differs, each domain ships its own Fold:

#![allow(unused)]
fn main() {
pub struct Fold<N, H, R> {
    pub(crate) impl_init: Arc<dyn Fn(&N) -> H + Send + Sync>,
    pub(crate) impl_accumulate: Arc<dyn Fn(&mut H, &R) + Send + Sync>,
    pub(crate) impl_finalize: Arc<dyn Fn(&H) -> R + Send + Sync>,
}
}

Local and Owned share the same shape, with Rc and Box substituted for Arc. The three are not interchangeable at the type level: the Fused executor reads whichever concrete D::Fold<H, R> the pipeline provides, and crossing domain boundaries requires an explicit conversion that the library does not supply — the expected discipline is to select a single domain per computation.

Parallelism

The parallel Funnel executor requires ShareableLift, a capability that reduces to D = Shared together with Send + Sync on every payload (N, H, R). Local and Owned cannot run in parallel by construction: their storage types do not cross thread boundaries, and the ShareableLift bound does not hold.

The converse is not true — a Shared pipeline runs without issue under Fused. The price of choosing Shared is one atomic operation per closure clone, and nothing more.

Picking one (decision tree)
%3q1Need to run in parallel?q2Closures capture Rc / RefCell / non-Send?q1->q2nosharedShared  (Arc)q1->sharedyesq3One-shot, want zero refcount overhead?q2->q3nolocalLocal  (Rc)q2->localyesq3->sharedno (default)ownedOwned  (Box)q3->ownedyes

In short: Shared by default, Local for non-Send captures, Owned for the one-shot minimal case.

For library authors

Prefer code generic over D: Domain<N>. The three domain markers are not interchangeable at runtime, but almost the whole of hylic compiles once and operates across all three. Select a concrete domain only where its specific capability is required (D = Shared for parallelism; D = Owned for consume-on-use).