Review Discovered Links
Every time a data capture job runs, it builds a map of every URL it found along the way — not just the pages it captured, but also the links it saw and chose not to follow. This is the Discovered links list, and it's the fastest way to understand what the job actually did.
Before You Start
- Open a website (or search) data capture job that has completed at least one run.
Steps
- Open the job's details page from Data capture.
- Select the Discovered links tab. The list shows every URL the job encountered during its latest run.
- Scan the URLs to confirm the pages you expected are present. Each entry shows the URL, whether it was processed or skipped, and how it was discovered (linked from which page).
- Look for surprising entries — pages outside your intended scope, or important pages that appear as skipped. These signals tell you whether the scope settings need adjusting.
Verify the Result
The pages you wanted should appear in the list with a processed status. If important pages are missing, the job's hostnames, path rules, or crawl depth may need widening. If unwanted pages made it in, the scope is too broad.
Troubleshooting
- Missing important links? Widen the hostnames, relax included-path rules, or increase the crawl depth.
- Seeing links you don't want? Add an excluded path rule to filter them out in the next run.