2. Evidence and research

Research should change a decision

Research is worthwhile when it reduces uncertainty that could alter an action. It is not a collection of favourable quotations. Begin with the decision from Module 1, list the competing explanations and ask which uncertainty matters most. If a team is choosing between promotion and booking repair, a broad question such as 'Do people like us?' may provide little help. Response delays, eligibility understanding and practical barriers are more directly connected to the choice.

Separate research questions from interview questions. 'Does response time prevent booking?' is a research question. 'What happened after you contacted the service?' is an interview question that might contribute evidence. Asking 'Would faster replies help you?' invites agreement with the researcher's preferred solution. A good question lets an unwelcome explanation emerge.

Match the method to the uncertainty

An administrative record can show when an enquiry arrived and when someone replied, provided the fields mean what you think they mean. It may not show when the person read the reply or whether they wanted an appointment. Observation can reveal a confusing step, but the presence of an observer may change behaviour. An interview can reveal an experience in depth, while a survey can gather comparable responses across more people. Neither automatically represents those who did not respond.

Secondary research offers a different advantage: it can identify mechanisms or methods worth considering before local work. Appraise the original question, population, setting, method, result and limitations. A finding about commercial buyers does not become evidence about advice clients simply because both are called customers. Transfer requires an argument about relevant similarities and differences.

Choose the smallest combination that can distinguish important explanations. A record audit followed by a few voluntary conversations may be more useful than a long survey with vague questions. A small study cannot estimate population prevalence reliably, but it can reveal a barrier worth testing. State that narrower purpose rather than dressing exploratory work as representative research.

Treat sampling as a choice with consequences

A sampling frame is the set of people or records you can reach. A sample drawn from current attendees excludes people who never booked, people who could not attend and people who never discovered the service. Their experiences may differ precisely where your decision matters. Adding more attendees will not fix that missing perspective.

For the fictional case, distinguish enquirers who booked, enquirers who did not book and bookings that did not lead to attendance. A plan might seek different experiences across those groups, but this is a proposed design, not authorisation to contact real people. Explain who remains absent, such as people with no digital access. Do not claim that a deliberately varied sample is statistically representative.

Non-response matters. If people who had a difficult experience are less willing to speak, the results may look too positive. If people with complaints are more motivated, the reverse may occur. You rarely know the full direction or size of this bias. Record the invitation route, response count and missing groups so another reader can judge the limits.

Ask without extracting or exposing

Before real research, the organisation must establish an appropriate purpose, permissions, participant information and handling arrangements. Participation must not be presented as a condition of receiving help. Explain what is being asked, how material will be used, whether a person can decline and whom to contact about concerns. This course supplies fictional material so you do not need to resolve those arrangements by collecting live data.

Prefer the minimum information necessary. For a booking study, a description of a confusing step may be enough; a detailed account of someone's financial or health situation may be unnecessary and risky. Avoid placing identifiable quotes in a shared portfolio. Small combinations of details can identify someone even without a name. If a topic becomes distressing, stop the activity and follow the organisation's approved support process rather than trying to solve the situation as a researcher.

Build an evidence log that preserves disagreement

An evidence log needs an identifier, observation, origin, method, date, limitation and decision relevance. Keep the raw observation separate from your interpretation. 'Reply recorded three working days after enquiry' differs from 'staff ignored the person'. The first may be supported by a record; the second adds intention and effect without evidence.

Code fictional interview notes in two passes. First mark concrete events and phrases close to the speaker's meaning. Then group patterns, retaining exceptions. A code such as 'timing conflict' can contain work shifts and care responsibilities, but it should not erase their different implications. Count coded excerpts only if you explain that these are excerpts, not independent people or prevalence estimates.

Look actively for a negative case. If most notes describe a hard-to-use form and one describes a simple form but an unsuitable appointment time, that exception may change the intervention. A tidy summary can be less accurate than a messy account that preserves conflicting evidence. AI often makes summaries smoother; your task is to keep the distinctions that matter.

Appraise confidence claim by claim

Confidence belongs to a claim, not to a document as a whole. You might have high confidence that a supplied fictional record shows a three-day interval and low confidence that the interval caused non-booking. Label the inference and ask what other observations would support it. Avoid a single numerical confidence score that hides different kinds of uncertainty.

Triangulation means comparing evidence from different methods or positions, not counting repeated copies of the same claim. Three websites quoting one survey are still dependent on that survey. Agreement between records and voluntary accounts may strengthen an explanation, but both could share a sampling gap. Write down where evidence converges, diverges and remains unavailable.

Your portfolio research plan should spend the 20 project hours deliberately. Include preparation, permissions, recruitment design, analysis and reporting, not just time spent asking questions. A plan that leaves no analysis time cannot support its own recommendations. Where a research step cannot happen, define the decision that can still be made and what must remain provisional. Honest limits are more useful than manufactured certainty.

Write questions that can receive an unwelcome answer

Compare three versions of a question. 'Was our form confusing?' suggests a diagnosis and encourages a yes/no response. 'Which part of our form was confusing?' assumes the problem exists. 'Please describe what happened when you tried to arrange help' allows an account that may concern the form, timing, another provider or no problem at all. A neutral opening does not remove every bias, but it reduces one avoidable source.

Follow an account with concrete clarification: 'What happened next?', 'What information did you use?' or 'Was there a point when you decided not to continue?' Do not turn the conversation into a defence of the organisation. If the person describes a useful alternative service, that may challenge a conversion-focused objective but still matter to the mission. A question about what people actually did is often more useful than asking them to predict whether they would like a proposed solution.

For a survey, test whether response options permit a genuine alternative and a decision not to answer. Combining 'the service was clear and easy to book' asks two things at once: someone may understand it but find no usable time. Separate comprehension from practical access. For records, define how missing values will be treated before interpreting a blank as a failure. A missing response date might mean no response, an unrecorded response or a different contact route. The log should preserve that ambiguity until another source resolves it. These checks improve the proposed instrument even when you never collect live data during the course.

Study allocation and completion

Allocate 1.5 hours to this lesson and worked example, 1.5 to critical reading, 2 to the lab, 3 to the assignment, 1 to critique and 1 to reflection and revision: 10 hours in total. These are planning allocations, not measured duration or a timed assessment. Read in short sessions if helpful. All resources are plain text; a notebook, calculator or optional free spreadsheet is sufficient. No video, AI account or paid service is needed.

For critical reading, use the linked selection if accessible and the internal reading comparison. Record its question, argument, evidence type, assumptions and one reason it might not transfer to your organisation. A missing external reading does not prevent completion: compare the two original internal positions and state that you have not appraised the external source. Keep a source log with title, URL, access date if accessed, claim supported and limitation. Links are references, not permission to copy or upload readings to AI.

Complete the lab manually first. The supplied simulated AI output is authored teaching material, not a transcript, real research or verified model performance. Mark each claim supported, unsupported or contradicted. Optional live AI may generate alternatives from these fictional inputs only; record tool, date, instructions, checks and revisions. Never supply personal, confidential or unauthorised material. If not using AI, compare the supplied output against your manual work and label that route accurately.

For critique, use the rubric and annotated example, alone or with a consenting peer; no peer is required. For reflection, revise one consequential decision and record the old choice, new choice, evidence and unresolved uncertainty. Keep work locally. Self-completion records learning activity, not professional competence or an independently assessed award. Version: 1 November 2026; report suspected errors through the course host's available contact route, or retain a correction note if no route exists.

Practical lab — 2 hours

Use the six fictional notes in the resource. Code events manually, then compare your coding with the supplied AI summary. Preserve a negative case. Design a sample across journey stages without contacting anyone. Allocate six of the case project hours to a bounded research design, including preparation and analysis. Explain what that design cannot establish.

Portfolio assignment — 3 hours

Write a research plan and evidence log, then integrate Module 1 into a 1,500-word diagnosis with an appendix for methods and limitations. This is the evidence-and-diagnosis component worth 20% of the course portfolio. State research questions, methods, proposed sample, ethical boundaries, rival explanations and a next decision. Use fictional notes as fictional evidence; do not report the proposed study as completed.

Critique — 1 hour

Apply the rubric to your work and the annotated example. Identify the strongest contrary explanation, one unsupported claim and one feasibility problem. If studying alone, write the objections before consulting the example.

Reflection and revision — 1 hour

Revise the weakest consequential decision. Keep both versions and a change note. State what further evidence would change your position; do not invent it.

For the shared advice case, include a two-page decision memo, a simple calculation sheet and the evidence plan as concise extracts or appendices to the diagnosis, not another weighted assessment. Before optional assistance, write a short manual explanation of your preferred diagnosis and its strongest rival; retain it to check your own understanding.

Use these self-review weights: evidence and source appraisal 25%; critical reasoning 25%; contextual fit 20%; feasibility and evaluation 20%; clarity and reflective judgement 10%. Score each criterion from 0 to 4 and calculate sum(weight × score / 4), out of 100. These are illustrative learning criteria, not validated academic grades.

Evidence: trace each claim to a note or source and identify dependent sources. Reasoning: preserve a disconfirming case. Context: discuss missing non-users. Feasibility: budget preparation and analysis within the proposed research hours. Clarity: distinguish a planned method from a completed result.

Across criteria: 0 means missing or misleading; 1 asserts a choice without support; 2 offers relevant but incomplete support; 3 connects evidence, alternatives and a feasible action; 4 tests a strong contrary account, quantifies limits where possible and specifies evidence that would reverse the choice. Do not reward length, fluent AI prose or an expensive tactic. A privacy breach, invented evidence or an unsupported public promise must be corrected before self-completion, regardless of the arithmetic.

Weak: 'Four people mentioned access, so 67% of clients face access barriers.' Six authored notes are not a representative sample, and category boundaries need explanation. Competent: 'Timing and response handling occur in these fictional notes; the sample cannot estimate prevalence.' Strong: 'N2 suggests eligibility uncertainty, while N4 reports a clear form but unsuitable times. We should not collapse both into form usability. An audit can check recorded handling intervals, but only further appropriate enquiry could clarify people's reasons.' The annotation rewards preserving an exception and matching a method to its limits. A fluent summary claiming all non-booking is digital exclusion would remain weak even if its prose were excellent.