How to turn unstructured website content into structured data
A web page is designed to be read, not to behave like a database row. Headings, prose, cards, and specification tables may all describe the same entity in different ways.
Turning that content into structured data requires two separate steps: capture the source faithfully, then apply a documented schema.
Decide what one record represents
A record might represent a product, company, policy, article, location, or event. Mixing several entities in one row produces fields that are hard to interpret.
Write a short contract for each field:
| Question | Example decision |
|---|---|
| What does the field mean? | publishedDate means the page's stated publication date |
| What type is it? | ISO date, text, number, list, or URL |
| Can it be missing? | Yes; use empty rather than an inferred value |
| Can it repeat? | Tags and certifications may be lists |
| What proves the value? | The record retains a source URL |
| How are conflicts handled? | Flag two published values for review |
Capture representative templates
Choose examples from every page type, not merely the cleanest page. Include a sparse record, a page with a table, an older layout, and an exception.
Seekdown data capture jobs retrieve pages into a memory dataset. The dataset contents view lets you inspect title, URL, body, summary, and other captured information before exporting it.
If a value is absent from the captured body, no later structuring step can recover it reliably from that record.
Normalize without erasing meaning
Normalization should make equivalent values comparable while preserving useful qualifiers.
- Convert "16 bar" into value 16 and unit bar only if both remain available.
- Map "Yes", "Available", and a check mark to one Boolean only when they mean the same thing.
- Preserve "from", "estimated", and "contact us" instead of turning them into exact prices.
- Keep original text for audit when a controlled category is assigned.
- Store dates in one format but retain the published timezone when it matters.
Separate extraction from validation
Use a labelled sample to measure field-level results. Record missing, incorrect, and unsupported values separately.
A page can produce a valid record even when an optional field is absent. It should fail when a value is invented, attached to the wrong entity, or stripped of a material qualifier.
Keep provenance in the final dataset
A structured value without its source URL and capture time is difficult to verify and unsafe to refresh blindly.
Choose an output for the next task
Use CSV for spreadsheet review, JSON for scripts and imports, or an Enterprise OData feed for a refreshable connection to Excel or Power BI. The format should follow the consumer's task; it does not change the quality of the source or schema.