How to Actually Schedule Splunk SOAR Playbooks
When you create a playbook in Splunk SOAR, you choose between two types.
Automation: Automation runs automatically based on 2 triggers for selected labels: artifact creation or container resolve.
Input: Input accepts configured inputs and provides outputs, and can only be used as a sub-playbook — it cannot be triggered independently at all.
That is the whole list. One type reacts to defined triggers, the other waits to be called by something else. Neither of them runs at a time.
Which is a problem, because some work is time-based. A nightly sweep for stale cases. A weekly rule review. A daily report someone reads with their coffee. A job that closes anything nobody has touched in thirty days. None of these have a triggering event.
There is a way to run them on a schedule, and most of the pieces already ship with the product. This is the setup I use.
Splunk publishes an app called Timer. Its entire job is one line in its own description: generate an empty event which can be used to kick off a playbook at scheduled intervals.
It has a single ingest action. Every time the asset polls, it creates one empty container and nothing else. No data, no artifacts — just a container with a label on it.
That container is the trigger. An Automation playbook set to run on that label fires every time one appears, and the polling interval decides how often that is.
So the wiring is:
- Install Timer and create an asset from it
- Give the asset an ingestion schedule and a label
- Set your playbook active on that label
That is real scheduling, out of the box, with no code.
Why I stopped using it
Every tick leaves a container behind.
That is the whole mechanism, and it is also the whole problem. A timer polling once a minute creates 1,440 containers a day. Every one of them is empty — no artifacts, no data, nothing to look at. They exist only to make a playbook start.
I had jobs on eight different intervals. Even the slow ones add up, and the fast ones add up quickly. Those containers land in the same list analysts work out of, they count toward everything that counts containers, and they never stop arriving. Within a week the case list is mostly noise that nobody created and nobody reads.
The design
The trick is a single inversion.
Timer uses on_poll to create an event, and something downstream reacts to it. I use on_poll to run the playbooks directly, and create nothing at all.
Once the poller is allowed to do the work itself, the empty container has no reason to exist. Everything else follows from that.
Three rules:
Keep the tick. SOAR’s ingestion scheduler already does the hard part. It takes cron expressions, it survives restarts, it is visible and editable in the UI, and it is per-asset. There is no reason to build a scheduler when the platform ships one — the problem was never the timing, it was what happens when the timer fires.
Make the list the registry. A custom list holds the names of the playbooks that should run. One list per interval: a list for the one-minute jobs, one for the hourly jobs, one for the nightly jobs. Scheduling a playbook becomes adding a row to a list. Unscheduling it becomes deleting that row. The playbook itself is never touched, and the schedule finally has one place to live where you can read it.
Run against one container. A playbook run has to be attached to a container, so I keep exactly one and reuse it forever. It is created once, by hand, and every scheduled run points at it. One container, not 1,440 a day.
Put together:
Building the app
A SOAR app is a JSON file and a Python module. The JSON declares what the app is — its identity, the configuration fields its assets will have, the actions it exposes. The module implements those actions. Everything else is packaging.
You do not have to write either one by hand. SOAR ships an app wizard at Apps → Create New App, and it generates the JSON. That is where this app starts.
Let’s create our own app for scheduling. An app in this state installs perfectly and does absolutely nothing, which is the correct place to be. It has an identity, it appears in the Apps list, and you can create an asset from it. What it cannot yet do is be told which SOAR to talk to, and that is the next problem.
The app cannot poll yet
Install it, create an asset from it, and open the asset. There is no Ingest Settings tab.
Nothing is misconfigured. An asset gets ingestion settings when the app it comes from claims to be an ingest source, and that claim is made in one place only: the app’s action list. SOAR reads the actions, sees whether any of them is of type ingest, and renders the tab if one is. My app has exactly one action — the test connectivity the wizard writes for every app — so there is nothing to render.
The wizard will not add it for you. It builds the metadata, the logos, the connector skeleton and that single free action, and stops there. To make the app pollable you edit the JSON by hand.
Open the chronos.json, and add this to the actions array alongside test connectivity.
{
"action": "on poll",
"identifier": "on_poll",
"description": "Run the playbooks listed in the registry custom list",
"verbose": "",
"type": "ingest",
"read_only": true,
"parameters": {
"start_time": { "order": 0, "data_type": "numeric", "description": "Parameter ignored in this app" },
"end_time": { "order": 1, "data_type": "numeric", "description": "Parameter ignored in this app" },
"container_id": { "order": 2, "data_type": "numeric", "description": "Parameter ignored in this app" },
"container_count": { "order": 3, "data_type": "numeric", "description": "Parameter ignored in this app" },
"artifact_count": { "order": 4, "data_type": "numeric", "description": "Parameter ignored in this app" }
},
"output": [],
"versions": "EQ(*)"
}Those five parameters deserve a note. SOAR passes them to every poller it starts, and almost no app uses them — I counted ninety-eight on poll definitions on my instance and the overwhelming majority describe all five as ignored. They still have to be declared, because an action definition without its parameters is incomplete.
Open the asset again.
The tab is there, and it is the real one — the same polling interval and cron scheduling that every ingest app on the platform uses. The app still does nothing when it fires, because the action is declared and not yet implemented. But the schedule now exists, and it belongs to the asset rather than to any playbook.
Asset configuration
An asset is one configured instance of an app, and its configuration fields are declared in the same JSON file, in the configuration object the wizard left empty. Whatever you put there becomes the form on the asset’s Asset Settings tab, and whatever the analyst types into that form arrives in the connector as a dictionary.
Chronos needs six fields, and they fall into three groups.
How to reach SOAR. A connector runs on the SOAR host but it has no privileged way into the platform — the phantom.* helpers that playbooks use do not exist in an app. If the app wants to start a playbook, it goes out through the REST API and comes back in, like any other client. So it needs a base URL and a token.
Get Huseyn Aghazada’s stories in your inbox
Join Medium for free to get updates from this writer.
What to run against. The fixed container from the design, by ID.
What to run. The name of the custom list that holds this schedule’s playbooks. This is the field that turns one app into many independent schedules: point two assets at two different lists and they poll on their own intervals, without knowing about each other.
"configuration": {
"base_url": {
"description": "SOAR base URL, as reachable from this instance",
"data_type": "string",
"required": true,
"value_list": [],
"default": "https://127.0.0.1",
"order": 0,
"name": "base_url",
"id": 0
},
"auth_token": {
"description": "ph-auth-token of an automation user allowed to run playbooks",
"data_type": "password",
"required": true,
"order": 1,
"name": "auth_token",
"id": 1
},
"verify_server_cert": {
"description": "Verify server certificate",
"data_type": "boolean",
"required": false,
"default": "False",
"order": 2,
"name": "verify_server_cert",
"id": 2
},
"container_id": {
"description": "ID of the container every scheduled playbook runs against",
"data_type": "numeric",
"required": false,
"value_list": [],
"default": "1",
"order": 3,
"name": "container_id",
"id": 3
},
"playbook_list": {
"description": "Name of the custom list holding the playbooks for this schedule",
"data_type": "string",
"required": true,
"value_list": [],
"default": "",
"order": 4,
"name": "playbook_list",
"id": 4
},
"run_scope": {
"description": "Artifact scope passed to each playbook run",
"data_type": "string",
"required": false,
"value_list": [
"new",
"all"
],
"default": "new",
"order": 5,
"name": "run_scope",
"id": 5
}
},Three things that have to exist first
The asset form will not save until you can fill it in, and three of its fields point at objects that do not exist yet.
A container to run against. Create one by hand, once. Any label will do, but pick one your analysts do not work out of, give it a name that explains itself to whoever finds it in six months, and close it. Nothing will ever be added to it. Note the ID from the URL — that is the container_id the asset wants.
An automation user and its token. A connector authenticates as somebody, and that somebody should not be you. Create an automation user, give it a role that can run playbooks and little else, and take the ph-auth-token from its user record. That token goes into the asset’s auth_token field, which is why that field was declared as a password.
The registry list itself. This is the interesting one.
The registry list
A custom list in SOAR is a table of strings with an id, a name and a content array of rows. Playbooks read them, the REST API reads them, and — usefully here — you can look one up by name rather than by id:
GET /rest/decided_list/Chronos_1_minThat single fact is what makes the schedule editable by hand. The asset stores a list name, the connector fetches that name, and the name is a thing a human can type into a form.
Create a list, call it something that names the interval rather than the job — Chronos_1_min, Chronos_1_hour — and put one playbook in each row of the first column.
Two conventions make the list nicer to live with, and both cost one line in the connector.
A row can be a name or an id. local/Daily Indicator Expiry is readable and survives a re-import; the numeric playbook id does not change meaning but does change value. The connector sends whichever it finds — digits go as an integer, anything else goes as a string.
Now the asset can be filled in: base URL, token, container id, list name. Save it. The app has a schedule, a target, and a job list, and still does nothing at all when the timer fires.
The on_poll handler
first you should edit the initialize handler:
def initialize(self):
self._state = self.load_state()
config = self.get_config()
self._base_url = config['base_url'].rstrip('/')
self._auth_token = config['auth_token']
self._verify = config.get('verify_server_cert', False)
self._container_id = int(config['container_id'])
self._playbook_list = config['playbook_list']
self._run_scope = config.get('run_scope', 'new')
return phantom.APP_SUCCESS_make_rest_call() needs the token on every request. The skeleton builds the call with no headers at all, so there is nowhere for authentication to go. Two lines fix it:
headers = kwargs.pop('headers', None) or {}
headers['ph-auth-token'] = self._auth_tokenand then pass `headers=headers` and a `timeout` into the request. Give it a timeout even though it is talking to itself — a connector that hangs holds an ingestion slot open, and the next tick arrives regardless.
And then the handler itself. This is the entire app:
def _handle_on_poll(self, param):
action_result = self.add_action_result(ActionResult(dict(param)))
ret_val, resp = self._make_rest_call(
'/rest/decided_list/{0}'.format(self._playbook_list), action_result
)
if phantom.is_fail(ret_val):
return action_result.get_status()
started = 0
failed = 0
for row in resp.get('content', []):
if not row:
continue
entry = str(row[0]).strip()
if not entry or entry.startswith('#'):
continue
body = {
'container_id': self._container_id,
'playbook_id': int(entry) if entry.isdigit() else entry,
'scope': self._run_scope,
'run': True,
}
ret_val, run = self._make_rest_call(
'/rest/playbook_run', action_result, method='post', json=body
)
if phantom.is_fail(ret_val) or not run.get('playbook_run_id'):
self.save_progress("Could not start '{0}'".format(entry))
failed += 1
continue
self.save_progress("Started '{0}' as run {1}".format(entry, run['playbook_run_id']))
started += 1
action_result.update_summary({'playbooks_started': started, 'playbooks_failed': failed})
if failed:
return action_result.set_status(
phantom.APP_ERROR, "Started {0}, failed {1}".format(started, failed)
)
return action_result.set_status(
phantom.APP_SUCCESS, "Started {0} playbook(s)".format(started)
)Read a list. Loop. POST. That is it, and it is worth pausing on how little there is: no scheduling, no state, no queue, no timestamps, nothing remembered between ticks. The platform owns the timing, the list owns the job registry, and the connector owns neither. If the app is stopped for an hour, nothing accumulates and nothing replays — it simply does not run, and then it runs again.
The failure handling is the one place I would not cut corners. A row that names a playbook which has been renamed, or a token that has lost the permission to run something, will fail on that row and succeed on the others. Counting both and returning an error when anything failed means the ingestion shows red on the asset and the message names the count, while the jobs that could run still ran. Failing the whole poll on the first bad row would be simpler and much worse.
Asset Creation
Let’s create an asset:
Wiring the intervals
One asset was the hard part. Every interval after it is the same three objects with different values in them.
The app is installed once. For each schedule you want, you create a custom list, an asset that points at it, and a cron expression on that asset:
Nothing in the connector knows how many of these exist. Each asset reads its own list, runs against its own container if you want separate ones, and fails on its own without touching the others. Adding the nightly schedule does not require thinking about the one-minute schedule, and neither of them requires editing a playbook.
That is the part I actually wanted. The problem in the first place was that eight intervals meant eight labels, eight assets and eight sets of playbook activations, with the schedule smeared across the configuration of every playbook that had one. Now eight intervals mean eight assets and eight lists, the playbooks themselves are untouched, and the full schedule is eight pages you can read.
Two things are worth knowing before you build the second one.
Ingest settings take a real cron expression, not just an interval in minutes. The minute interval is fine for fast jobs, but anything on a daily or weekly cadence should be a cron expression, because “every 1440 minutes” drifts relative to the clock and 0 2 * * * does not.
Give each asset its own automation user token if you care about blast radius. They can share one and it will work. But the token in the asset is the only thing deciding what a schedule is allowed to run, and one token per interval means a compromised or over-permissioned nightly schedule cannot start whatever it likes on the one-minute one.
Where you would take this next
The registry is a plain custom list. That is the whole extensibility story, and it is worth saying out loud because it means nothing in this design is closed to you: anything that can write a row can schedule a playbook.
The obvious next step is to stop editing the list by hand.
Drive it from a label instead. Pick a naming convention and a label — say, playbooks whose name starts with ‘Schedule — ‘and which carry scheduled_1min — and write a playbook that walks /rest/playbook, keeps the ones that match both, and rewrites Chronos_1_min with the result. Then scheduling a playbook is putting a label on it, and unscheduling it is taking the label off. The list stops being something you maintain and becomes something you read.
That playbook is itself a scheduled job, so it goes in the list it maintains. There is something pleasing about a registry that keeps itself current, and something worth being careful about too: if the label is misspelled, the sync writes an empty list and the whole schedule stops. Have it count what it matched and complain when the count is zero.
It is a real trade rather than a strict upgrade. Editing the list by hand gives you one page that is the schedule, and a # in front of a row to take a job off for an afternoon. Deriving the list from labels gives you a schedule that can never drift out of step with the playbooks, at the cost of the schedule no longer living anywhere you can read it in one go. Both are defensible. A sync that preserves commented-out rows gets you most of both, and is about thirty lines.
Or go the other way and remove the list entirely. Nothing stops the connector from querying /rest/playbook itself on every tick and running whatever carries the label. That deletes a moving part and an object to keep in step, and it also deletes the ability to schedule something that has no label, or to schedule the same playbook on two cadences. I kept the list because a registry you can open and read is worth more to me than one fewer object, but the app is thirty lines and the change is about five of them.
Get It
Chronos is on GitHub, MIT licensed:
https://github.com/huseynAgazade/splunk-soar-chronos
The repository has the app JSON, the connector, and a packaged `.tgz` on the releases page that you can upload straight into **Apps → Install App**. The README covers the same ground as this article in a shorter form, including the `on poll` declaration the app wizard will not write for you.
If you build one of the extensions above, or find a version of SOAR where something here behaves differently, open an issue — I would rather know.
Thanks for reading.
