Course 03: Information Research and Evidence

Series · Case · Glossary · References

A controlled library can still fail its users: important information may be missing from the index, a useful source may be difficult to retrieve, and a retrieved document may answer a different question.

This course teaches research directed toward a project decision, in a team where people work with AI agents. You will frame the question, find relevant information, examine its evidential role, and prepare a usable evidence packet. The work begins with supplied sources and remains suitable for offline study.

Prerequisites and outcomes

Complete Courses 01 and 02, and read Northbank Packet A. Use your controlled-basis package as a starting point.

How this course is built

The series index explains how the series is built. With agents, gathering from accessible records can become fast, so the researcher's hard work moves to establishing where information came from, what role a source can play for the question, and catching plausible fabrications, such as an invented citation or a summary that drops a condition. A03's copied, unsourced values are the classical example of that failure, and agents can reproduce it faster and at greater volume.

The teaching arrangement

Use the case's arrangement for working with agents. In this course, brief agents with Packet A as the only permitted source, unless an exercise says otherwise.

Outcomes

By the end, you can:

• Frame a bounded research question.
• Build an index that supports relevant retrieval.
• Explain a source's provenance, authority, and limits.
• Compare conflicting information without hiding the difference.
• Prepare an evidence packet with a scoped conclusion and unresolved matters.
• Brief an agent for research, and examine its return for provenance, dropped conditions, and fabrication.

Lesson 1: Frame the question before searching

The classical method

Research begins with a question that someone needs answered. A broad request such as “research the pumps” leaves the purpose unclear and encourages a large collection with no definite use.

State the decision or activity that the research must support, and identify the object, conditions, needed information, and boundary. “What does Packet A establish about the electrical connection?” is answerable, but “Is the complete electrical design suitable?” needs much more evidence.

Separate a factual question from an applicability question. A survey can establish what was observed, but it may not establish what a law requires. An approved appointment can establish project authority, but it may not establish professional authority outside that appointment.

Identify the consumer. A designer may need an exact observed input, a sponsor may need the consequence of an unresolved input, and a reviewer may need the source and method behind a conclusion. The same information can serve each consumer differently.

Use a research brief:

1. State the question.
2. Identify the decision or activity it supports.
3. Define the object and relevant conditions.
4. Identify the permitted sources.
5. State the expected form of the answer.
6. State what falls outside the enquiry.

For a short enquiry, these fields can fit in a paragraph. Revisit the brief when the available evidence reveals a different problem. Make a scope change explicit instead of quietly extending the search.

What it assumes

The 'What it assumes' passages in this course describe working arrangements for comparison. They are not universal facts about human practice; examine whether they hold in the task at hand.

The classical method assumes that searching is costly. A broad request wastes the researcher's time, so framing the question protects the researcher's effort as well as the consumer's purpose. It also assumes that the researcher notices when the evidence points to a different question and stops to ask.

What changes with agents

A broad request no longer wastes search time, because an agent can search and collect quickly. It now wastes something scarcer: the attention of the person who must examine a large, plausible collection. The framing that protected the researcher's effort now protects the examiner's attention and the scope of the conclusion.

Agents answer the question they are given, and they can answer an adjacent one without saying so. Asked what Packet A establishes about the electrical connection, an agent may go on to say whether the supply is suitable, which is the larger question the classical method warns against. That is the quiet extension of scope the brief must prevent, performed without the pause that would make a person ask.

Agents also bring knowledge from outside the permitted sources. Unless the brief restricts it, an agent can fill a gap in Packet A with what is typical elsewhere, and present it in the same voice as what the records say.

The extended method

The research brief is the agent's brief. Keep all six fields, and give Steps 4 to 6 more weight:

• Permitted sources: name them by identity and revision, and state that the agent must not supply facts from outside them. For the offline exercise, that is Packet A.
• Expected form: ask for each claim with its source, revision, and the quoted supporting passage, and a separate list of questions the sources do not answer.
• Outside the enquiry: state the larger questions the agent must not answer, such as electrical suitability.

Ask the agent to restate the question it answered at the start of its return, and compare that restatement with your question.

Lesson 2: Build a library that answers questions

The classical method

A library index gives people routes into information. Document identities support exact retrieval, subject terms support discovery, and relationships show where one item qualifies, replaces, or supports another.

Index the subjects that project participants use. At Northbank, useful terms include flow, electrical connection, operating conditions, scope proposal, appointment, and external requirements. A file named “survey_final” may not appear in a search for electrical capacity.

Distinguish a source inventory from a complete evidence collection. A07 lists what the shared folder contains and identifies indexing defects, but it does not establish that the project holds every necessary source.

Keep the links between claims and sources. A design note that cites A03 without its planning-estimate note loses relevant context, so add that note to the index or claim entry. Avoid making the next reader rediscover a known limitation.

Access is part of retrieval, so record a location that the intended user can reach. If a source is unavailable, state the missing access and its consequence. A broken link cannot support review just because its title is visible.

Apply this indexing method:

1. List the exact controlled items.
2. Assign useful subject terms.
3. Record each item's origin and scope.
4. Record relationships that affect its meaning.
5. Identify known omissions.
6. Test retrieval with a real project question.

Ask another person to run the retrieval test. Give them the question, not the expected filename, and watch whether the index leads them to the relevant revision and its qualifications.

What it assumes

The classical method assumes that people build the index by hand and retrieve through it, so retrieval is only as good as the subject terms someone thought to assign. A07's omission of the electrical survey is the typical failure: a source missing from the index is effectively missing from the library.

What changes with agents

Agents can search the content of sources, not only the index, so an agent can often find a source that the index omits. That reduces the cost of an omission like A07's. It does not remove the need for the index, for two reasons.

First, an agent's search is not visible in the way an index is. A reader can see what an index covers; they usually cannot see what an agent searched, what it skipped, or why. When an agent reports that no source addresses a question, that report is only as good as its coverage, and the coverage is often unstated.

Second, the index records relationships that content search does not reveal: that A02 Rev 1 replaces Rev 0, that A03's values were copied from a planning estimate, that A05 proposes a change without amending A02. These relationships need someone's judgment to record, and they govern what a retrieved source means.

The extended method

Keep the index for exact retrieval and for the relationships that affect meaning. Use agent search for discovery, and treat what it finds as candidates to place in the index, with their qualifications.

When an agent reports that a source is absent, ask it to state what it searched: which sources, which terms, and which parts it could not read. Rely on an absence only as far as that coverage supports it.

Run the retrieval test with a person, as the classical method requires. An agent can also run it, but an agent that already knows the records is not testing the index.

Lesson 3: Examine the evidential role of a source

The classical method

Evidence supports or challenges a claim, and its value depends on the claim and the proposed reliance. A document can be authoritative for one question and unsuitable for another.

Provenance identifies the origin and history needed to understand information. A checked observation should identify what was observed, where, when, and by whom; a copied value should keep its original source and the transformations applied to it.

Authority is about a source's standing for the question. A02 establishes the approved project-development requirement, but it does not establish a measured plant condition. A04 records a checked site observation, but it does not decide the project scope.

Consider relevance, conditions, method, and completeness. A supplier brochure describes a product family, but without a selected configuration it cannot establish that the proposed equipment is suitable. An accurate document can therefore be insufficient for a decision.

Use a claim-and-evidence table that records the exact claim, source revision, supporting passage, conditions, limitations, and consequence for reliance. When the claim goes beyond the source, state what additional evidence is needed.

Apply this examination method:

1. Write the claim in a testable form.
2. Identify the exact source.
3. Locate the supporting statement or observation.
4. Examine its origin and method.
5. Match its conditions to the claim.
6. State the supported conclusion and remaining gap.

Do not treat an impressive title, approval stamp, or confident author as a substitute for this comparison: examination asks what the source establishes for the actual question.

What it assumes

The classical method assumes that a person who cites a source has read it, and that copying a value takes some effort and leaves a trail. It also assumes that fabrication is rare, because inventing a citation takes intent. Its main concern is the copied value that lost its origin, as A03's did.

What changes with agents

Agents can produce plausible fabrications without any intent to deceive. An agent can give a citation, a section reference, a quoted passage, or a value that the source does not contain, and present it in exactly the same form as a genuine one. A confident, well-formatted citation is no longer evidence that the passage exists.

Agents also drop conditions when they summarize. A summary that says “A02 requires 18 litres per second” has lost “under the operating conditions that the project must establish.” A summary that says “A04 confirms a 600 V supply” has lost what A04 does not establish: spare capacity, protection coordination, and equipment compatibility. Each sentence of such a summary can be accurate and the summary still mislead.

The A03 failure can recur faster with agents. A03's values were copied from a planning estimate and carry no survey reference. An agent's output that states a value without its source is the same kind of record, and if someone later cites the agent's output, it becomes a source with no provenance. Agents can make many such records quickly.

The extended method

Require each claim an agent returns to carry the exact source identity, revision, and the quoted supporting passage. Then check, for each claim that matters, that the passage exists in that revision and says what the claim says. Check every claim that a decision will rest on; for minor claims, check enough to know whether the agent's citations hold.

Compare each agent summary with its source condition by condition. Ask which qualifications the source states and whether each one survived.

Treat agent output as derived information. Its provenance includes the brief, the sources it was given, and the date. Do not let an agent summary enter the library as a source; cite the record it summarizes.

Lesson 4: Resolve conflicts and report limits

The classical method

Sources may disagree because they cover different revisions, locations, conditions, or purposes, or because one contains an error. Identify the kind of difference before you choose a response.

At Northbank, the 15 and 18 litres per second values appear in different revisions and source roles, and the 22 litres per second value is a proposal. Their presence does not create three equally valid scope choices.

The electrical values present another issue. A03 records 480 V from an earlier planning estimate, while A04 records a checked observation of 600 V at the existing connection. Compare their provenance and scope, and preserve the older statement so the team can trace work that used it.

An evidence packet should let another participant reconstruct the answer without repeating the whole search. Include the question, source index, claim table, conflict treatment, conclusion, and unresolved matters, but keep substantial source material in its controlled location when possible.

State limits plainly. “A04 records the observed connection characteristics” is supported, but “The site can supply the selected motor” needs capacity, protection, and compatibility examination. A missing result should remain a gap.

Complete an evidence packet with this method:

1. Compare apparently conflicting statements.
2. Identify differences in revision, purpose, and conditions.
3. Record the consequence for the question.
4. State a conclusion at the supported scope.
5. Assign the remaining enquiry to a suitable owner.
6. Test whether another person can reconstruct your reasoning.

The packet is useful when it supports a decision or next activity. Avoid adding documents that have no identified role, because a large collection can hide the absence of the one source the decision needs.

What it assumes

The classical method assumes that the researcher who meets a conflict has read both sources and sees the difference. It also assumes that assembling a packet takes effort, which keeps packets small.

What changes with agents

Agents can resolve a conflict silently. Given 480 V and 600 V, an agent may report one value, prefer the newer or more confident source, or merge them into a sentence that hides the difference. Any of these can be right by accident, and each one removes the trace the team needs to find work that used A03.

Agents also make packets cheap to fill. The classical warning about documents with no identified role matters more, because an agent can add many relevant-looking sources quickly, and the absence of the one source the decision needs is harder to see in a large packet.

A missing result can also be filled. Asked whether the site can supply the selected motor, an agent may answer from general knowledge instead of reporting that Packet A contains no capacity result. The classical rule that a missing result remains a gap now needs to be stated in the brief.

The extended method

Brief agents to report every conflicting value with its source and revision, and not to resolve the conflict. Classifying the difference, whether a revision, a proposal, a copied estimate, or an error, is the researcher's work, because it depends on each source's role.

Brief agents to report gaps as gaps. Then check the packet against the decision it serves: does each document have a role, and is the source the decision most needs present, or recorded as missing?

Run the reconstruction test with another person, not only with an agent, because the test asks whether a person can follow the reasoning.

Northbank worked example: The electrical enquiry

With the source identities and statuses established, the next question is what those sources can support. The electrical enquiry shows why retrieval must preserve each source's limits.

The research question is: “What does Packet A establish about the existing electrical connection?” The consumer is the electrical engineer preparing further investigation.

The researcher retrieves A03 Rev 0 and A04 Rev 1; A07 Rev 0 explains the indexing omission and the design note's incomplete citation.

The researcher repairs the subject index by listing A04 under electrical connection and site observation. The evidence packet includes A03's source note and flags the design note for review, because that note omitted the qualification.

The conclusion gives the next enquiry a sound observation, while preventing that observation from becoming an unsupported compatibility claim.

The same work with agents

This account is a teaching assumption, not a Northbank record: the records do not say how any of the work was produced.

The researcher briefs an agent with the question, the consumer, and Packet A as the only permitted source, identified by revision. The brief asks for each claim with its quoted passage, every conflicting value reported without resolution, and a separate list of matters the records do not answer. It states that electrical suitability is outside the enquiry.

The agent returns a claim table. The researcher checks each quoted passage against A03 Rev 0 and A04 Rev 1, and checks that the A03 entry keeps its planning-estimate origin and its lack of a survey reference. The researcher checks that the capacity and compatibility rows remain unresolved and are not filled with general knowledge, and that the 480 V and 600 V values both appear with their sources.

The table and its conclusions are the same as in the classical account. The repair of the subject index and the flag on the design note remain the researcher's decisions, because they depend on judging each source's role.

Transfer example: A service response-time report

A software team has a laboratory performance report and a customer complaint. The report measures response time under a small test load, while the complaint is about intermittent delays during peak use.

Both sources can be accurate, because they concern different conditions. The researcher states the measured laboratory result and the reported production symptom separately.

The next enquiry needs evidence under the relevant production conditions. The report cannot establish that every customer's experience is satisfactory, and the complaint alone cannot identify the technical cause.

An agent asked to summarize both sources can merge them into one account of “performance,” losing the difference in conditions. The researcher's brief asks it to keep each source's conditions with its result.

Practice: Build two evidence packets

Use Packet A and Portfolio 02. No external search is required.

The supplied questions are:

• Which transfer-flow value governs project development?
• What does the packet establish about the electrical connection?

First prepare the claim-and-evidence table for the flow question yourself, without an agent. The worked example covers only the electrical question, so this table is your own baseline for judging what an agent's table should contain. Work with an agent for the outputs that follow.

Supply these outputs:

1. A brief for each question with its consumer and boundary.
2. One subject index covering the relevant records.
3. A claim-and-evidence table for each question.
4. A treatment of every apparent conflicting value.
5. A conclusion and remaining-enquiry list for each packet.

Give a partner only the questions and your index, and ask them to retrieve the evidence and challenge one conclusion. Record any indexing defect or unsupported inference, and repair it before assessment.

For each agent-assisted output, keep the brief you gave the agent and a short note of what you checked in its return, what you found, and what you changed.

Then complete three further exercises:

1. Check the passages. For each claim in an agent's table, locate the quoted passage in the cited revision. Record each claim whose passage you found and each you could not, and what you did about it.
2. Check the conditions. Ask an agent for a one-paragraph summary of A02 Rev 1 and A04 Rev 1. Compare it with the records sentence by sentence, and list each condition or limit the summary keeps and each it drops.
3. Mark outside knowledge. Ask an agent, without restricting its sources, what the Northbank electrical connection can support. Mark each statement that comes from outside Packet A, and explain why it cannot establish a Northbank fact. Then write the brief sentence that would keep it within the packet.

Knowledge check

1. Why should a research question name its intended decision or activity?
2. What does a subject index add to a controlled document list?
3. How can a source be authoritative for one claim and insufficient for another?
4. What provenance limits apply to A03 Rev 0?
5. Why do the three Northbank flow values not create three approved scope choices?
6. What must an evidence packet contain to support another person's examination?
7. When gathering from accessible records becomes fast, why does the difficulty of research move toward examination?
8. An agent's evidence table cites A04 Rev 1 with a quoted passage. What must you check before relying on the entry?
9. How can an agent's summary of A02 be accurate in every sentence and still mislead?
10. How is an agent's unsourced value like A03's 480 V, and how do you keep it from becoming a source?
11. An agent reports that no source in the library addresses electrical capacity. What do you need to know before you rely on that report?

Assessed assignment: Prepare information for reliance

Prepare an evidence packet for the project engineer's equipment-basis discussion, working with agents, and address both practice questions. Use Packet A only, and state what the discussion can decide and what needs further input. Add the briefs you used and your examination record.

Then defend the packet to your instructor. Explain its conclusions and what you checked, and reason through changed premises such as a survey reference added to A03, an agent citation that A04 does not contain, or an index that lists A04. Your instructor rates the defense as the teaching guide describes; in text-only study it is rated on changed-premise reasoning with the medium recorded.

An instructor assesses seven observable criteria:

Explain one conclusion in your own words and cite the passage that supports it. Repair any retrieval failure or unsupported inference. Acceptance in training does not establish the complete equipment basis.

Portfolio continuation and references

Keep the evidence packet as Portfolio 03 and link its source index to Portfolio 02. Course 04 will examine the interpretation and applicability of the requirements found here.

The management manual (Edition 9), sections 2.2 and 6.3, supports records that preserve identity, provenance, and useful grounds. Its section 2.4 supports visible treatment of open matters. Section 2.2 also warns that a reliable source can produce an unreliable extraction.

Use the series references for further reading. External readings are optional; no external source can fill a gap in the fictional case without an explicit new exercise input.