The core loop
Build
Create a fact pattern, document set or workflow that resembles real legal work. The files must agree with one another, remain usable by ordinary tools, and contain everything the task expects.
Break
Introduce a genuine reasoning challenge: competing authority, close facts, an incomplete record or a plausible but consequential mistake. Obscurity alone is usually a retrieval test, not a judgment test.
Answer
Write or evaluate the reference work at a standard another lawyer could defend. Correct conclusions are not enough if the analysis skips the controlling step or cites support that does not bear the proposition.
Grade
Turn expert judgment into criteria another reviewer can apply consistently. Separate material legal failures from preference, avoid double-counting, and align every required element with the task.
World building is document-system design
In one recorded Moonlux walkthrough, a contributor described a six-stage process: plan the matter, clear plan QC, create a specification for every required document, clear specification QC and human review, generate the file set, then repair it through repeated pipeline checks. The specification becomes the frozen source of truth.
Her concrete lesson was that the whole file set has to agree. A changed person, date, entity, reference or planted error may need to propagate across dozens of artifacts. A local edit can create a distant contradiction.
| Risk | What it looks like | Why it matters |
|---|---|---|
| Answer leakage | Emphasis, comments, author notes or suspiciously obvious errors reveal what the evaluator expects. | The task becomes easier for the wrong reason and stops measuring independent reasoning. |
| Cross-file drift | Names, dates, IDs, facts or authorities disagree between artifacts. | The model can fail because the world is broken, not because the reasoning is weak. |
| Machine unreadability | A malformed row, inaccessible PDF, or document reading order hides evidence from ordinary extraction tools. | A file can look correct to a person and still fail in the actual environment. |
| Trap loss | A repair silently “fixes” the deliberate error the task was designed around. | The benchmark loses its challenge while the corpus still appears clean. |
| Provenance residue | Tool names, timestamps, build notes or stale metadata survive into final files. | Internal authoring history enters a submission that should look like ordinary work product. |
Quality control is a loop, not a button
The interviewed contributor estimated that repeated QC cycles could consume roughly 90 minutes to two hours on a task: run a checker, interpret a failure, repair the files, rerun, and discover that one fix disturbed something else. That number is her observed experience, not a market-wide average.
Her proposed solution was a pre-linter: check the final submission surface once, disclose exactly what ran, and collect mechanical problems before the slower platform loop. That insight became Merit.
The working kit itself contained checks with paths where a clean-looking result did not mean the check had completed. That changed Merit’s product rule: a skip is never a pass. A trustworthy checker must prove it can detect a planted failure and say when coverage is missing.
Attached does not mean read
In the recordings, the contributor repeatedly discovered that a model had skipped an uploaded file or decided that a ZIP did not need to be opened. The omission only became visible after QC found something the skipped material already explained. Her practical rule is simple: inventory the required inputs, ask what was actually inspected, and do not treat an attachment as proof of ingestion.
The same problem appears in failure analysis. A model with the full task history still attributed quotations to the wrong trajectory and elevated the wrong weakness. Model-generated comparisons are useful starting material, but every quotation, cited location and claimed failure needs to be traced back to the underlying output.
A useful playbook remembers the corrections
A recurring burden in the recordings was having to teach a new session “the easy dumb stuff” again. A project playbook should not be a giant undifferentiated prompt. It should route only the material needed for the current stage and keep a current record of:
- The latest project instructions and the date they were retrieved.
- Which tools are permitted in each clearly marked phase.
- The required inputs for this stage and confirmation that each was inspected.
- Known QC failure classes, prior repairs and checks proven against planted failures.
- The task’s frozen facts, intentional traps and decisions that must persist across files.
This is a product direction Moonlux is testing, not a claim that Merit already performs full project orchestration.
“Can I use AI?” is the wrong level of question
Permissions can differ by project and by phase. In the interviewed workflow, model assistance was allowed in some creation and testing phases but restricted for the contributor’s own analysis. That is not a rule for any other project.
- Read the current project instruction—not a rule remembered from another project.
- Identify the exact phase you are in and which tools are permitted in that phase.
- Keep client, employer and platform-confidential material out of unapproved tools.
- If the rule is silent or ambiguous, ask the platform in writing before continuing.
- Preserve your own work only when your agreement and project rules permit it.
What strong contributors appear to do differently
Current public listings and our early interviews converge on a few behaviors: they explain reasoning, apply a consistent standard, notice when authority is merely reachable rather than supportive, distinguish material errors from stylistic ones, and operate carefully inside detailed instructions. Teaching, editing and supervising experience may transfer well because the work requires judgment made explicit.
What we do not yet know: how consistent these patterns are across platforms, how much each project’s rubric changes them, and which interventions measurably improve acceptance. Moonlux is collecting that evidence through a small founder beta rather than promising an untested formula.
Turn the guide into a preflight.
Use the checklist to inspect scope, files, consistency, metadata and incomplete checks before submission. Then run the fictional Merit sample—nothing leaves the browser.
Open the checklistSee current roles