Connecting OKF to RAG: source versions, chunks and query contracts

An index can still split a table after its source has been moved into an OKF bundle. Headings, links and versions make information available; preserving it during ingestion and using it during retrieval require implementation.

This site maintains knowledge/index.md as an entry point for agents reading project conventions. That demonstrates file navigation, not a deployed vector service or a measured retrieval improvement. The following is a proposed contract for connecting such sources to a retrieval system.

Define what each result must carry

For a hypothetical order-retention rule, the path might be policies/orders.md, the section ID retention and the environment prod. Also record the source revision, content hash, index version and effective date. A path locates a document; a revision identifies the particular edition. These are proposed application fields, not claims about mandatory OKF fields.

{
  "source_id": "policies/orders.md#retention",
  "source_revision": "2",
  "environment": "prod",
  "effective_from": "2026-09-01",
  "content_hash": "<computed from source bytes>",
  "index_version": "retention-demo-v2",
  "parent_id": "policies/orders.md"
}

Example: the OKF spec requires only type; the other fields are yours to define

Section 11 of the OKF v0.2 spec lists the conformance conditions. Condition 2 reads: “Every frontmatter block contains a non-empty type field.” Section 4.1 says type is the only always-required key. Title, description, resource and tags are not required. The spec has no source_revision, environment or effective_from field from the JSON above.

Two spec fields come close. In §4.1, resource is a URI for the underlying asset, so it can map to source_id. In §5.2, generated.at is an ISO 8601 datetime that marks the content’s “last meaningful change.” It is a time, not a revision number. To detect changed content, the indexer still has to store its own content hash.

Section 5.5 adds an optional stale_after field: “A concept is stale when now >= stale_after.” A query stage can use it to drop expired concepts. It points the other way from effective_from in the JSON above: one marks when content expires, the other when it takes effect. The effective date still needs its own field.

The spec also states that retrieval is out of scope. The Non-goals list in §1 includes “Prescribing storage, serving, or query infrastructure.” The Goals list in the same section names search indexes as one kind of consumer. Chunking, index versions and deletion sync therefore belong to the indexing stage.

The OKF checks in this site’s build also stop at the source stage. scripts/lib/okf.mjs checks that every non-reserved .md file has frontmatter with a non-empty type. scripts/check-okf.mjs adds a site rule: when a public-bundle resource is a keeponfirst.com URL, it must appear in the sitemap. Both checks read only source files. This site has no vector index, so nothing checks for stale segments left in an index. A first check to add when connecting retrieval: every segment’s source_id in the index must still exist in the current bundle. This is a proposal; this site has not built it.

The chunker must preserve structure

Keep section paths, table headers and scope in retrievable data. If a table exceeds the segment limit, split rows while retaining column names, units and the parent ID. Definitions spanning segments need a way to retrieve their parent. A fixed character-count split can lose meaning even in carefully organized sources.

Explicit links also do not implement traversal. Define allowed edges, a hop limit, cycle handling and a missing-target result. Apply the same authorization to source access and retrieval; do not retrieve restricted content and expect the model to ignore it.

Switch versions instead of mixing policies

When the source moves from revision 1 to 2, build a new index version and verify additions, changes and deletions. Switch the active index only after checks pass. If they fail, retain the previous version and disclose its date. This is a proposed design, not a report of a service already tested on this site.

Check at least four cases: the applicable production rule, isolation of test rules, absence of deleted documents and an unknown answer when evidence is missing. The acceptance condition is use of the correct source version and scope, not simply production of an answer.

Choose retrieval by the information available

With a known document ID, fetch it directly and check its version. For a vague description, use full-text or vector search to find candidates and enforce environment, effective-date and access conditions. Expand relationships through IDs or links when necessary. This is not a universal rule to traverse OKF before every vector search; choose based on available identifiers and observed retrieval failures.

The answer should expose its source ID, revision and cited passage. If only expired or incompatible rules are available, report that gap instead of drawing a conclusion. A file format cannot guarantee correctness; these checks make failures traceable.

Use one case to check the boundaries

For the production-retention question, the failure might be a test document ranking higher or an old production revision remaining active. The first calls for an environment filter; the second calls for index-version checks. Rewording the prompt does not fix either ingestion problem. The companion article supplies a model-free ranking and filtering example; use it to check the fields before adding real embeddings and answer evaluation.

The bundle maintainer owns source content and links. The indexer owns segmentation, versions and deletion. The query service owns constraints and fetching. The answering agent owns citations and handling missing evidence. Explicit ownership identifies which step has fallen behind a source update.