arbiter-worker-0.1.0.0
arbiter-worker
Safe HaskellNone
LanguageGHC2024

Arbiter.Worker.Config

Description

Configuration types for the arbiter worker pool.

Synopsis

Worker Configuration

Source #data WorkerConfig (m :: Type -> Type) payload

Configuration for a worker pool.

Constructors

WorkerConfig 

Fields

Source #transactionalWorkerConfig :: forall (n :: Type -> Type) m payload. (MonadArbiter n, MonadIO m) => Int -> JobHandler n payload (ResultOf n payload) -> m (WorkerConfig n payload)

Arguments

:: forall (n :: Type -> Type) m payload. (MonadArbiter n, MonadIO m) 
=> Int

Worker count

-> JobHandler n payload (ResultOf n payload) 
-> m (WorkerConfig n payload) 

Create a WorkerConfig running one job per group in a worker transaction held for the duration of the handler.

The handler returns the result type payload's registry entry declares.

Source #manualWorkerConfig :: (MonadArbiter n, MonadIO m) => Int -> (JobRead payload -> BatchCallbacks n payload (ResultOf n payload) -> n ()) -> m (WorkerConfig n payload)

Arguments

:: (MonadArbiter n, MonadIO m) 
=> Int

Worker count

-> (JobRead payload -> BatchCallbacks n payload (ResultOf n payload) -> n ()) 
-> m (WorkerConfig n payload) 

Create a WorkerConfig running one job at a time, no worker transaction. The handler finalizes the job through BatchCallbacks. An unfinalized job is reprocessed.

Source #defaultBatchedWorkerConfig :: (MonadArbiter n, MonadIO m) => Int -> Int -> (NonEmpty (JobRead payload) -> BatchCallbacks n payload (ResultOf n payload) -> n ()) -> m (WorkerConfig n payload)

Arguments

:: (MonadArbiter n, MonadIO m) 
=> Int

Worker count

-> Int

Batch size (max jobs per group to claim together)

-> (NonEmpty (JobRead payload) -> BatchCallbacks n payload (ResultOf n payload) -> n ()) 
-> m (WorkerConfig n payload) 

Create a WorkerConfig for batched job processing, no worker transaction. The handler receives the batch and a BatchCallbacks record to finalize each job (ack, fail, cancel, or nack). Jobs left untouched are reprocessed. To store a result per job, ack with ackWith or ackAllWith.

Source #withHooks :: forall (m :: Type -> Type) payload. (ObservabilityHooks m payload -> ObservabilityHooks m payload) -> WorkerConfig m payload -> WorkerConfig m payload

Rework a pool's observability hooks.

Source #withMaintenance :: MonadUnliftIO m => (MaintenanceOp -> Int64 -> m ()) -> WorkerConfig m payload -> WorkerConfig m payload

Run report before the pool's own maintenance callback. Each runs even when the other fails.

Source #data HandlerMode (m :: Type -> Type) payload

How the worker claims and runs jobs. Set by this module's config constructors.

Constructors

SingleJobMode (JobHandler m payload (ResultOf m payload))

Automatic single-job mode: claim one job per group and run the handler in a worker transaction, storing its result and acking atomically.

BatchedJobsMode Int (NonEmpty (JobRead payload) -> BatchCallbacks m payload (ResultOf m payload) -> m ())

Batched callback mode: claim up to N jobs per group and hand the batch to the handler with a BatchCallbacks record to finalize each job. No worker transaction. Batch size 1 is the manual single-job case.

Source #handlerBatchSize :: forall (m :: Type -> Type) payload. WorkerConfig m payload -> Int

How many jobs a pool claims per group.

Source #data MaintenanceOp

Which reaper op a maintenance report came from.

Source #maintenanceOpName :: MaintenanceOp -> Text

The op's stable name, used to coordinate replicas and to label its metrics.

#type ResultOf (m :: Type -> Type) payload = ResultFor payload (RegistryOf m)

The result type declared by payload's registry entry. It is not injective. A signature naming it needs another argument to determine payload.

Source #validateWorkerConfig :: forall (m :: Type -> Type) payload. WorkerConfig m payload -> Either Text ()

Validate all invariants required for safe worker execution. Return all violations.

Batch Callbacks

Source #data BatchCallbacks (m :: Type -> Type) payload result

Per-job finalizers handed to a batched handler. Untouched jobs are reprocessed. The With variants store a result for the job's parent rollup or its archive entry.

Each callback runs in its own transaction and commits on return. Call them at the top level of the handler. Wrapping one in your own withDbTransaction enlists the callback into that transaction as a savepoint, committing atomically with your writes. The success hook then fires at savepoint release. An outer rollback reprocesses the job after the visibility timeout.

Constructors

BatchCallbacks 

Fields

  • ack :: JobRead payload -> m ()

    Ack and fire onJobSuccess, storing no result. Available on any queue. A job acked this way is absent from its parent rollup's child results and leaves its archive entry's result NULL. Aborts the handler if another worker holds the job. The abort nacks the siblings still unfinalized.

  • ackWith :: JobRead payload -> result -> m ()

    Ack, store the result for the parent rollup or the job's archive entry, fire onJobSuccess.

  • ackAll :: [JobRead payload] -> m ()

    Bulk-ack in one parent-aware transaction, storing no results. Fires onJobSuccess per acked job. A job another worker holds is reported and skipped. The handler continues.

  • ackAllWith :: [(JobRead payload, result)] -> m ()

    ackAll storing each job's result for its parent rollup or archive entry.

  • failRetry :: JobRead payload -> Text -> m ()

    Retry with backoff, then DLQ at the job's maxAttempts.

  • failPermanent :: JobRead payload -> Text -> m ()

    Straight to the DLQ.

  • cancelBranch :: JobRead payload -> Text -> m ()

    Cancel this job's branch (its parent and all siblings).

  • cancelTree :: JobRead payload -> Text -> m ()

    Cancel the whole tree from the root down.

  • nack :: JobRead payload -> m ()

    Reprocess after the visibility timeout. Records no failure and consumes no attempt.

Worker State

Source #data WorkerState

A worker pool's effective state, read off the shutdown and pause flags on its WorkerConfig. Shutdown wins, then pause, then running.

Constructors

Running 
Paused 
ShuttingDown 

Instances

Instances details
Eq WorkerState Source # 
Instance details

Defined in Arbiter.Worker.WorkerState

Show WorkerState Source # 
Instance details

Defined in Arbiter.Worker.WorkerState

Source #shutdownWorker :: forall m (n :: Type -> Type) payload. MonadIO m => WorkerConfig n payload -> m ()

Initiate graceful shutdown of the worker pool

Stops claiming new jobs. In-flight jobs will complete, then the pool exits.

Source #getWorkerState :: forall m (n :: Type -> Type) payload. MonadIO m => WorkerConfig n payload -> m WorkerState

The pool's state, with a pause reported as paused.

Source #getListenerReady :: forall m (n :: Type -> Type) payload. MonadIO m => WorkerConfig n payload -> m Bool

Whether this pool's LISTEN channels are subscribed (or there is no listener).

Source #readEffectiveState :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> STM WorkerState

Source #writePause :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> Bool -> STM ()

Set the pause flag, superseding any reading a caller has in flight.

Source #writePauseIfCurrent :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> Word64 -> Bool -> STM ()

Apply a pause reading taken at epoch. A newer write wins.

Source #workerStateVar :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar WorkerState

Run/shutdown state for a pool.

Source #pauseVar :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar Bool

Pause state for a pool.

Source #pauseEpoch :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar Word64

Version of the current pause state.

Source #heartbeatSignal :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TMVar ()

Signal used to coordinate worker heartbeats.

Source #listenerReadyVar :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar Bool

State of the pool's notification listener.