arbiter-worker-0.1.0.0
arbiter-worker
Safe HaskellNone
LanguageGHC2024

Arbiter.Worker.Config

Description

Configuration types for the arbiter worker pool.

Synopsis

Worker configuration

data WorkerConfig (m :: Type -> Type) payloadSource #

Configuration for a worker pool.

Constructors

WorkerConfig 

Fields

transactionalWorkerConfig :: forall (n :: Type -> Type) m payload. (MonadArbiter n, MonadIO m) => Int -> JobHandler n payload (ResultOf n payload) -> m (WorkerConfig n payload)Source #

Arguments

:: forall (n :: Type -> Type) m payload. (MonadArbiter n, MonadIO m) 
=> Int

Worker count

-> JobHandler n payload (ResultOf n payload)

Handler returning the result type payload's registry entry declares

-> m (WorkerConfig n payload) 

Create a WorkerConfig running one job per group in a worker transaction held for the duration of the handler.

manualWorkerConfig :: (MonadArbiter n, MonadIO m) => Int -> (JobRead payload -> BatchCallbacks n payload (ResultOf n payload) -> n ()) -> m (WorkerConfig n payload)Source #

Arguments

:: (MonadArbiter n, MonadIO m) 
=> Int

Worker count

-> (JobRead payload -> BatchCallbacks n payload (ResultOf n payload) -> n ())

Job handler

-> m (WorkerConfig n payload) 

Create a WorkerConfig that passes one job per handler call, no worker transaction. The handler finalizes the job through BatchCallbacks.

batchedWorkerConfig :: (MonadArbiter n, MonadIO m) => Int -> Int -> (NonEmpty (JobRead payload) -> BatchCallbacks n payload (ResultOf n payload) -> n ()) -> m (WorkerConfig n payload)Source #

Arguments

:: (MonadArbiter n, MonadIO m) 
=> Int

Worker count

-> Int

Batch size: max jobs per batch. A batch is one group, or ungrouped jobs.

-> (NonEmpty (JobRead payload) -> BatchCallbacks n payload (ResultOf n payload) -> n ())

Batch handler

-> m (WorkerConfig n payload) 

Create a WorkerConfig for batched job processing, no worker transaction. The handler receives the batch and a BatchCallbacks record to finalize each job. To store a result per job, ack with ackWith or ackAllWith.

withHooks :: forall (m :: Type -> Type) payload. (ObservabilityHooks m payload -> ObservabilityHooks m payload) -> WorkerConfig m payload -> WorkerConfig m payloadSource #

Transform a pool's observability hooks.

withMaintenance :: MonadUnliftIO m => (MaintenanceOp -> Int64 -> m ()) -> WorkerConfig m payload -> WorkerConfig m payloadSource #

Run report before the pool's own maintenance callback. Each runs even when the other fails.

data HandlerMode (m :: Type -> Type) payloadSource #

Job claim and handler mode. Set by transactionalWorkerConfig, manualWorkerConfig and batchedWorkerConfig.

Constructors

SingleJobMode (JobHandler m payload (ResultOf m payload))

Automatic single-job mode: claim one job per group and run the handler in a worker transaction, storing its result and acking atomically.

BatchedJobsMode

Batched callback mode: claim batches of up to N jobs and hand each batch to the handler with a BatchCallbacks record to finalize each job. No worker transaction. A batch is one group, or ungrouped jobs. Batch size 1 is the manual single-job case.

Fields

handlerBatchSize :: forall (m :: Type -> Type) payload. WorkerConfig m payload -> IntSource #

The maximum jobs in a batch. A batch is one group, or ungrouped jobs.

data MaintenanceOpSource #

Which reaper op a maintenance report came from.

Constructors

RefreshGroups

Refresh the group summary rows.

SweepStaleWorkers

Delete worker rows whose heartbeat is stale.

SweepExhaustedJobs

Move exhausted jobs to the DLQ.

SweepCancelledJobs

Delete force-cancelled jobs whose lease has lapsed.

PruneRateLimitBuckets

Delete idle rate-limit buckets that are full.

ReconcileConcurrencyStale

Rebuild the concurrency counts after a crash truncated them.

ReconcilePruneConcurrency

Recount the concurrency keys, then delete the unused ones.

PurgeArchives

Delete expired archived jobs.

PurgeDLQ

Delete DLQ jobs older than their queue's DLQ retention.

maintenanceOpName :: MaintenanceOp -> TextSource #

The op's stable name, used to coordinate replicas and to label its metrics.

type ResultOf (m :: Type -> Type) payload = ResultFor payload (RegistryOf m)#

The result type declared by payload's registry entry. It is not injective. A signature naming it needs another argument to determine payload.

validateWorkerConfig :: forall (m :: Type -> Type) payload. WorkerConfig m payload -> Either Text ()Source #

Validate all invariants required for safe worker execution. Return all violations.

defaultMaintenanceSparseInterval :: NominalDiffTimeSource #

Gap the bucket prune and the concurrency reconcile-and-prune keep.

defaultMaintenanceBucketIdle :: NominalDiffTimeSource #

Age after which an idle rate-limit bucket is pruned.

defaultMaintenanceTimeout :: NominalDiffTimeSource #

Statement timeout for one maintenance operation.

Batch callbacks

data BatchCallbacks (m :: Type -> Type) payload resultSource #

Per-job finalizers passed to a batched handler. When the handler returns, untouched jobs are reprocessed after the lease expires, and their attempt is used. When it throws, untouched jobs fail. A JobNackException releases them with their attempt restored. A JobScopedFailure that names jobs in the batch fails only those jobs. A JobGoneException that names jobs in the batch reports only those jobs as unavailable, or as cancelled after a force-cancel. Both release the rest. The With variants store a result for the job's parent rollup or its archive entry.

Each callback commits in its own transaction. Inside withDbTransaction, it runs as a savepoint in the outer transaction. The success hook fires at savepoint release. An outer rollback reprocesses the job after its lease expires.

ack, ackWith and spawn abort the handler if another worker holds the job. The abort nacks the siblings still unfinalized.

Constructors

BatchCallbacks 

Fields

hoistBatchCallbacks :: (forall a. m a -> n a) -> BatchCallbacks m payload result -> BatchCallbacks n payload resultSource #

Callbacks for m, called from n through the given natural transformation. It must be a monad morphism that keeps the worker's connection and schema. Each callback then joins the handler's transaction.

Worker state

data WorkerStateSource #

A worker pool's effective state, read off the shutdown and pause flags on its WorkerConfig. Shutdown wins, then pause, then running.

Constructors

Running

Claims and runs jobs.

Paused

Claims no new jobs.

ShuttingDown

Claims no new jobs and stops.

Instances

Instances details
Eq WorkerState Source # 
Instance details

Defined in Arbiter.Worker.WorkerState

Show WorkerState Source # 
Instance details

Defined in Arbiter.Worker.WorkerState

data WorkerRuntimeSource #

Mutable state owned by one worker pool. The config constructors allocate it.

shutdownWorkerPool :: forall m (n :: Type -> Type) payload. MonadIO m => WorkerConfig n payload -> m ()Source #

Initiate graceful shutdown of the worker pool.

Stops claiming new jobs. In-flight jobs get up to gracefulShutdownTimeout to finish. Then the pool exits.

getWorkerState :: forall m (n :: Type -> Type) payload. MonadIO m => WorkerConfig n payload -> m WorkerStateSource #

The pool's state, with a pause reported as paused.

getListenerReady :: forall m (n :: Type -> Type) payload. MonadIO m => WorkerConfig n payload -> m BoolSource #

Whether this pool's LISTEN channels are subscribed (or there is no listener).

readEffectiveState :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> STM WorkerStateSource #

Internal

Pool runtime access for the arbiter packages. Not covered by the PVP. Use writePause to change the pause flag. A direct write to pauseVar bypasses the epoch guard.

writePause :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> Bool -> STM ()Source #

Set the pause flag, superseding any reading a caller has in flight.

writePauseIfCurrent :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> Word64 -> Bool -> STM ()Source #

Apply a pause reading taken at epoch. A newer write wins.

workerStateVar :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar WorkerStateSource #

Run/shutdown state for a pool.

pauseVar :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar BoolSource #

Pause state for a pool.

pauseEpoch :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar Word64Source #

Version of the current pause state.

heartbeatSignal :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TMVar ()Source #

Signal used to coordinate worker heartbeats.

pulseHeartbeat :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> STM ()Source #

Pulse the heartbeat signal.

listenerReadyVar :: forall (n :: Type -> Type) payload. WorkerConfig n payload -> TVar BoolSource #

State of the pool's notification listener.