Data Capture Jobs
A data capture job is the main way to fill a dataset from a website. You point it at one or more pages, and it collects their content into a memory dataset — no copy-paste, no manual entry, and it can repeat on a schedule so your knowledge never drifts out of date.
How a Data Capture Job Works
It works outward, page by page. It opens a starting page, extracts and stores its content, reads the links on that page, then follows those discovered links to the next pages — repeating until it reaches the limits you set. Those limits are the whole art of capturing well:
- Where it goes — the starting URLs, the allowed hostnames, and included/excluded paths.
- How far — the link depth (how many hops from the start) and the capture scope, a page cap ranging from Preview (100) to Large (5000).
- How fresh — an optional schedule, plus a content lifespan that re-processes pages automatically as they age.
For example, to turn a product handbook into knowledge without dragging in the marketing site, you'd start at
https://acme.com/docs/, restrict the hostname toacme.com, include only*/docs/*, set a depth of 3, pick the Medium scope, and schedule a weekly re-capture. The result: the handbook stays current on its own, and login and checkout pages never get captured.
Capturing isn't the only way to fill a dataset
A data capture job captures websites, but you can also upload files or push content through the content API (an Enterprise feature). All three land in the same dataset and are searched together — so a dataset is rarely just one source.