Singapore AI evaluation events: bring a failure question and check the exercise

The problem

Your prototype works in demonstrations but you need peers who understand evaluation and deployment constraints.

Start with the behaviour you cannot yet explain

Write a short failure note before searching: the user task; input you can discuss; expected behaviour and its basis; observed output; application version or setup; and unresolved question. In an invented example, an assistant answers a delivery-day question using an older policy passage although the supplied current passage states a different day. The builder can show the discrepancy without knowing whether the cause is retrieval, instructions or another step. Keep those explanations as hypotheses. A general request to improve reliability gives a facilitator little to examine. Your first decision is which session can help inspect this named gap. Do not broaden one observed case into a claim that every answer is wrong, or treat a persuasive demonstration as evidence that the gap is resolved.

Read the evaluation activity behind the event title

For each candidate, record the task discussed, what participants do, inputs provided, assessment method, feedback opportunity and setup requirement. Look for those details in the actual agenda rather than counting evaluation, agents or production-ready as complete descriptions. A speaker can explain a method without providing individual case review; a lab can run supplied examples without accepting your own. Ask: “I have a shareable expected-versus-observed example. Will participants inspect results and their limitations, or only watch a prepared demonstration?” Confirm any assumed coding, accounts, installation and usage charges separately. SGInnovate’s calendar remains a discovery route, not confirmation that a particular programme offers this exercise, welcomes your starting level or has a current place.

• SGInnovate event calendar ↗ · Source checked 20 October 2026

Draw the boundary of the application being examined

Put the relevant steps on one line: input, retrieved material if used, model response, displayed answer and subsequent action. Mark each as examined, supplied, excluded or unknown in the proposed exercise. The AI Verify Foundation’s 2025 pilot lessons distinguish testing an LLM from inspecting steps inside an application pipeline. This is historical testing context, not a promise about a current workshop. In the fictional delivery example, a model test given the correct passage and an application test that retrieves its own passage address different questions. Ask which one the session exposes. You do not need to reveal internal architecture to discuss this boundary; a small invented flow can identify the evidence missing from a final-answer screenshot.

• AI Verify Foundation: lessons from the 2025 assurance pilot ↗ · Source checked 20 October 2026

Bring a bounded case the session actually permits

Keep the failure note with the smallest invented or appropriately permitted input that makes its question understandable. State whether the example reproduces the problem in your own permitted environment or merely illustrates it. Confirm whether the session supports offline discussion, participant screen sharing or a named practice environment; those arrangements can require different preparation. For replacing a real customer interaction without losing the failure mechanism, use the existing customer-data case guide. Do not send the original material simply because a facilitator asks for a more realistic case. If failed requests or excluded outputs distort a translation score, use the existing execution-accounting guide for that separate reporting question. Here the case serves selection of a suitable exercise; its preparation does not establish a benchmark or sharing permission.

• Prepare a useful case without sharing customer data →

Compare three fictional offers against the same gap

These offers are invented and describe no current Singapore event. Offer A demonstrates a tool on prepared questions and provides general discussion, with no participant test stated. Offer B supplies a small application example, lets participants inspect expected and observed answers and discusses which steps the exercise excludes. Offer C tours deployment charts but does not state how answer quality is assessed. The builder with the delivery-passage discrepancy has a reason to investigate B’s starting skills and case-feedback arrangement. A may explain the tool without examining the discrepancy; C leaves the connection between an answer failure and its charts unanswered. Neither B’s title nor this comparison confirms access to its environment or individual advice. Keep the useful activity and its unresolved condition together instead of choosing by tool novelty.

Check what monitoring after launch would actually examine

If ongoing behaviour is your gap, add four questions to the agenda check: which task is observed; which output or feedback is retained; how that observation is assessed; and what limitation participants discuss. A chart of request completion does not by itself assess whether an answer used the intended passage. The 2025 pilot lessons also discuss human calibration and ongoing monitoring of LLM-based judges to identify silent failures. That historical observation explains why a checker deserves scrutiny too; it does not supply a monitoring interval, threshold or approved production procedure. Ask whether the exercise examines the application, its evaluator or only service operation. For the separate question of who responds to an operational alert and what they inspect next, use the existing cloud alert guide. Keep both scopes visible without requiring every first evaluation workshop to cover them.

• AI Verify Foundation: lessons from the 2025 assurance pilot ↗ · Source checked 20 October 2026

Record what the take-home product contains

Use three rows for any promised output: work you perform; components supplied by the session; and dependencies or behaviours left untested. Add what you can retain and the conditions for later use. A working classroom application can depend on a fixed input set, supplied retrieval or a disabled next action. In an invented learning record, you edit an instruction and examine the answer against a supplied policy passage, while the retrieval stage remains provided and unchanged. That supports describing the instruction exercise; it does not show that your own application retrieves the right material. Ask where the take-home claim stops before treating a file, certificate or successful demonstration as an operating service. Keep a supplied result separate from work you independently reproduce after the session.

Optional: compare the work behind a running-cost example

Use this additional worksheet only when running cost is part of your question. Bring six entries: the user task and comparable workload; period; application attempts; any separate evaluator or other component use; response-time requirement and actual observation; and the applicable charge record still needed. Ask which entries the session can help you observe in a permitted exercise. Keep completed tasks separate from work performed while attempting them. All quantities in this example are invented: five named practice tasks finish, while two tasks each need a second application attempt. That gives seven application attempts for five tasks. A separate fictional evaluator is invoked once for each completed task, giving five evaluator invocations. Record five tasks, seven application attempts and five evaluator invocations, rather than twelve independent tasks or twelve automatically billable units. Keep any other components and their applicable terms on separate rows. Another offer completing five tasks in five attempts has a different execution record; without assessment results and an actual charging basis, neither example establishes a cheaper or better system. A missing charge record stays missing. Keep response time and task quality separate from spending. Use the cloud service guide for equal-work and period comparisons, and the translation execution guide for failed cases and attempt-selection reporting. A presentation bill, proposed retry or small workshop sample cannot supply your applicable rate or forecast production workload.

• Keep failed cases and repeat attempts visible →

Leave with an evidence question and a bounded next step

Complete four fields after the session: observation you actually examined; explanation supported by that observation; limitation still open; and next permitted check or question. Label an experiment proposed by a peer as proposed until you perform it. In the fictional delivery case, inspecting which passage the practice flow supplied can narrow a question while leaving your own retrieval behaviour unresolved. Use the reviewer-disagreement guide when assessment criteria are disputed, or the process-versus-technical-testing guide when the missing output is a governance record rather than a behavioural test. Use Boardzip’s free AI search for candidates, then apply the activity and boundary notes to the specific programme. Admission and participation terms still require their own checks. If no verified session meets the essential task, retain the gap rather than inventing a clinic or counting an unrelated demo as evaluation.

• Compare conflicting reviewer judgements →
• Choose process checks or technical testing →