arbiter-worker-0.1.0.0
arbiter-worker
Safe HaskellNone
LanguageGHC2024

Arbiter.Worker

Description

Public worker API. Single-pool execution and multi-pool orchestration are implemented separately in Arbiter.Worker.Pool and Arbiter.Worker.MultiQueue.

Synopsis

Running workers

Source #runWorkerPool :: forall payload m. (RegistryAdmissionPolicies (RegistryOf m), EncodeJobResult (ResultOf m payload), QueueOperation m payload, RegistryTables (RegistryOf m)) => WorkerConfig m payload -> m ()

Run a worker pool: a dispatcher and its worker threads.

Source #namedWorkerPool :: forall payload (m :: Type -> Type). (EncodeJobResult (ResultOf m payload), QueueOperation m payload, RegistryAdmissionPolicies (RegistryOf m), RegistryTables (RegistryOf m)) => WorkerConfig m payload -> NamedWorkerPool m

Name a pool from its payload's registry entry.

Source #poolConfigForWorkers :: forall (m :: Type -> Type). RegistryTables (RegistryOf m) => [NamedWorkerPool m] -> IO PoolConfig

A single-stripe pool sized at twice the enabled worker count plus one for the listener, with a minimum size of three.

Source #runSelectedWorkerPools :: MonadUnliftIO m => [Text] -> [NamedWorkerPool m] -> m ()

Run only named pools. A pool that exits winds down its peers. The first failure among them is rethrown after every peer has been joined.

Source #runWorkerPools :: (MonadUnliftIO m, RegistryTables (RegistryOf m)) => [NamedWorkerPool m] -> m ()

Run the pools selected by ARBITER_ENABLED_QUEUES, or every configured pool when it is unset.

Source #shutdownPools :: forall m (m' :: Type -> Type). MonadIO m => [NamedWorkerPool m'] -> m ()

Signal graceful shutdown to every pool atomically.

Source #data NamedWorkerPool (m :: Type -> Type)

A worker pool paired with its registry-derived queue name.

Source #getEnabledQueues :: forall (registry :: JobPayloadRegistry). RegistryTables registry => String -> Proxy registry -> IO [Text]

Arguments

:: forall (registry :: JobPayloadRegistry). RegistryTables registry 
=> String

Environment variable name

-> Proxy registry

Registry proxy

-> IO [Text] 

The comma-separated queue names an environment variable asks for, every one of them checked against the registry. Unset or blank gives the registry's whole queue set, and a name outside it throws.

Job results

Source #childResults :: (FromJSON (ResultOf m payload), MonadArbiter m) => JobRead payload -> m (Map Int64 (Either Text (ResultOf m payload)), Map Int64 Text)

A rollup parent's immediate child results, keyed by child id, and its DLQ errors, keyed by DLQ row id for retryFromDLQ. A decode failure is returned as Left.

Source #mergedChildResults :: (FromJSON (ResultOf m payload), MonadArbiter m, Monoid (ResultOf m payload)) => JobRead payload -> m (ResultOf m payload, Map Int64 Text)

childResults with successfully decoded values combined through Monoid.

Source #mergeChildResults :: Monoid a => Map Int64 (Either Text a) -> a

Combine successful child results, treating decode failures as mempty.

Source #storeJobResult :: (EncodeJobResult result, MonadArbiter m) => Text -> JobRead payload -> result -> m ()

Store a job's result for its parent rollup, if it has one.

Source #storeEncodedResult :: MonadArbiter m => Text -> JobRead payload -> Maybe Value -> m ()

storeJobResult on an already-encoded result. Nothing stores nothing.

Configuration and logging

Source #newWorkerState :: IO (TVar WorkerState)

Create a new worker state initialized to Running.

Source #signalShutdown :: TVar WorkerState -> IO ()

Signal graceful shutdown. The pool stops claiming, finishes what it holds, and exits.

Source #data WorkerState

A worker pool's effective state, read off the shutdown and pause flags on its WorkerConfig. Shutdown wins, then pause, then running.

Constructors

Running 
Paused 
ShuttingDown 

Instances

Instances details
Eq WorkerState Source # 
Instance details

Defined in Arbiter.Worker.WorkerState

Show WorkerState Source # 
Instance details

Defined in Arbiter.Worker.WorkerState

Reaper

Source #runMaintenancePass :: (RegistryAdmissionPolicies (RegistryOf m), MonadArbiter m, RegistryTables (RegistryOf m)) => LogConfig -> (MaintenanceOp -> Int64 -> m ()) -> MaintenancePace -> NominalDiffTime -> m [MaintenanceOp]

One pass of the maintenance the reaper runs, with each operation independently gated across all callers. An operation whose window has not elapsed is skipped. Returns the operations that failed. A failure does not stop the pass.

Source #data MaintenancePace

Gaps a caller holds between runs of each kind of work. A zero gap runs it every pass.

Constructors

MaintenancePace 

Fields

Source #runReaperOp :: MonadArbiter m => LogConfig -> SchemaName -> NominalDiffTime -> Text -> NominalDiffTime -> m a -> m (Maybe a)

Run one gated maintenance operation. Database statement timeouts bound individual statements. Failures are logged and do not stop the loop.

Cron

Source #data CronJob payload

A cron schedule. Built with cronJob, or cronJobInTimezone for a non-UTC one.

Constructors

CronJob 

Fields

Instances

Instances details
Generic (CronJob payload) Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Associated Types

type Rep (CronJob payload) 
Instance details

Defined in Arbiter.Worker.Cron.Types

Methods

#from :: CronJob payload -> Rep (CronJob payload) x

#to :: Rep (CronJob payload) x -> CronJob payload

type Rep (CronJob payload) Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Source #data OverlapPolicy

How overlapping cron ticks are deduplicated.

Constructors

SkipOverlap

At most one pending or running job per schedule.

AllowOverlap

One job per tick. Concurrent execution of prior ticks is allowed.

Instances

Instances details
Eq OverlapPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Generic OverlapPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Associated Types

type Rep OverlapPolicy 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep OverlapPolicy = D1 ('MetaData "OverlapPolicy" "Arbiter.Worker.Cron.Types" "arbiter-worker-0.1.0.0-inplace" 'False) (C1 ('MetaCons "SkipOverlap" 'PrefixI 'False) (U1 :: Type -> Type) :+: C1 ('MetaCons "AllowOverlap" 'PrefixI 'False) (U1 :: Type -> Type))
Show OverlapPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep OverlapPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep OverlapPolicy = D1 ('MetaData "OverlapPolicy" "Arbiter.Worker.Cron.Types" "arbiter-worker-0.1.0.0-inplace" 'False) (C1 ('MetaCons "SkipOverlap" 'PrefixI 'False) (U1 :: Type -> Type) :+: C1 ('MetaCons "AllowOverlap" 'PrefixI 'False) (U1 :: Type -> Type))

Source #data BackfillPolicy

Bound on replay of missed ticks. Applies at startup and mid-flight.

Constructors

NoBackfill

Drop missed minutes silently. Default.

Backfill NominalDiffTime

Replay missed minutes up to the given duration.

Instances

Instances details
Eq BackfillPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Generic BackfillPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Associated Types

type Rep BackfillPolicy 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep BackfillPolicy = D1 ('MetaData "BackfillPolicy" "Arbiter.Worker.Cron.Types" "arbiter-worker-0.1.0.0-inplace" 'False) (C1 ('MetaCons "NoBackfill" 'PrefixI 'False) (U1 :: Type -> Type) :+: C1 ('MetaCons "Backfill" 'PrefixI 'False) (S1 ('MetaSel ('Nothing :: Maybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 NominalDiffTime)))
Show BackfillPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep BackfillPolicy Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep BackfillPolicy = D1 ('MetaData "BackfillPolicy" "Arbiter.Worker.Cron.Types" "arbiter-worker-0.1.0.0-inplace" 'False) (C1 ('MetaCons "NoBackfill" 'PrefixI 'False) (U1 :: Type -> Type) :+: C1 ('MetaCons "Backfill" 'PrefixI 'False) (S1 ('MetaSel ('Nothing :: Maybe Symbol) 'NoSourceUnpackedness 'NoSourceStrictness 'DecidedLazy) (Rec0 NominalDiffTime)))

Source #data TickKind

Whether a tick is for the current minute or a replay of a past minute.

Constructors

Live 
Replay 

Instances

Instances details
Eq TickKind Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Generic TickKind Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

Associated Types

type Rep TickKind 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep TickKind = D1 ('MetaData "TickKind" "Arbiter.Worker.Cron.Types" "arbiter-worker-0.1.0.0-inplace" 'False) (C1 ('MetaCons "Live" 'PrefixI 'False) (U1 :: Type -> Type) :+: C1 ('MetaCons "Replay" 'PrefixI 'False) (U1 :: Type -> Type))
Show TickKind Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep TickKind Source # 
Instance details

Defined in Arbiter.Worker.Cron.Types

type Rep TickKind = D1 ('MetaData "TickKind" "Arbiter.Worker.Cron.Types" "arbiter-worker-0.1.0.0-inplace" 'False) (C1 ('MetaCons "Live" 'PrefixI 'False) (U1 :: Type -> Type) :+: C1 ('MetaCons "Replay" 'PrefixI 'False) (U1 :: Type -> Type))

Source #cronJob :: Text -> Text -> OverlapPolicy -> (TickKind -> UTCTime -> JobWrite payload) -> Either String (CronJob payload)

Arguments

:: Text

Schedule name (used in dedup keys and logging)

-> Text

Cron expression (5-field: minute hour day-of-month month day-of-week)

-> OverlapPolicy 
-> (TickKind -> UTCTime -> JobWrite payload)

Job builder. Receives the tick kind and the tick time.

-> Either String (CronJob payload) 

Build a CronJob. A bad expression returns Left. The expression is evaluated in UTC. cronJobInTimezone is the local-time form. Set backfill by record update.

cronJob "nightly-report" "0 3 * * *" SkipOverlap
  (\_kind _tick -> defaultJob (GenerateReport "nightly"))

Source #cronJobInTimezone :: Text -> Text -> Text -> OverlapPolicy -> (TickKind -> UTCTime -> JobWrite payload) -> Either String (CronJob payload)

Arguments

:: Text

Schedule name

-> Text

IANA tz name (e.g. "America/New_York")

-> Text

Cron expression (5-field)

-> OverlapPolicy 
-> (TickKind -> UTCTime -> JobWrite payload) 
-> Either String (CronJob payload) 

Like cronJob but evaluated in a specific timezone. The tz name is validated eagerly against the bundled tzdata database.

Source #initCronSchedules :: MonadArbiter m => SchemaName -> Text -> [CronJob payload] -> LogConfig -> m ()

Upsert default expression and overlap for each CronJob into the cron_schedules table. Preserves any user overrides and enabled state.

Source #overlapPolicyToText :: OverlapPolicy -> Text

Convert an OverlapPolicy to its text representation.

Source #validateCronScheduleUpdate :: CronScheduleUpdate -> Either Text ()

Check a cron patch before it is written. At tick time a bad override stops firing or falls back to the default.