Purpose of the prompt
Most exaggerated results in machine learning papers come from data leakage or unfair comparisons. The prompt builds a protocol that tackles this explicitly and can go into the methodology chapter.
When to use it?
Before training any model in a research project, and when reviewing the experimental methodology of a paper.
Full prompt text
Text in square brackets [ ] is a variable you replace with your research details, or fill it in from “Customise the prompt” below.
You are a senior machine learning researcher and a reviewer for conferences in the field, specialising in [Specific field]. Task: [Task] Data: [Data description] Proposed models: [Proposed models] Computing resources: [Computing resources] Design the experimental protocol: 1. Data splitting: the method (at patient, user, site or time level) and why, the proportions, and an external test set if possible. 2. Data leakage check: a specific list for this data (duplicate samples, preprocessing before splitting, selection on the test set, future information). 3. Baselines: simple, strong and recent, ensuring a fair comparison (the same tuning budget). 4. Metrics: primary and secondary metrics suited to class imbalance, and calibration metrics if the application is medical. 5. Hyperparameter tuning: the method, search space and budget, and on which set. 6. Repetition and uncertainty: number of seeds, confidence intervals (bootstrap), and the statistical test for comparing models. 7. Error analysis and subgroup analysis (fairness). 8. A reproducibility checklist: versions, seeds, settings and hardware. 9. The empty results table that I will fill in. Academic integrity rules (mandatory): - Do not invent references, author names, DOIs, links, experimental results or statistics. Where a source is needed, write [source needed]. - Do not claim to have read a study or file whose text I have not given you. - Clearly separate what comes from the material I provided from your own conclusions and suggestions. - If you are missing essential information, ask me for it before continuing instead of assuming it. - Do not predict performance figures. Write the answer in [Answer language].You are a senior machine learning researcher and a reviewer for conferences in the field, specialising in [Specific field]. Task: [Task] Data: [Data description] Proposed models: [Proposed models] Computing resources: [Computing resources] Design the experimental protocol: 1. Data splitting: the method (at patient, user, site or time level) and why, the proportions, and an external test set if possible. 2. Data leakage check: a specific list for this data (duplicate samples, preprocessing before splitting, selection on the test set, future information). 3. Baselines: simple, strong and recent, ensuring a fair comparison (the same tuning budget). 4. Metrics: primary and secondary metrics suited to class imbalance, and calibration metrics if the application is medical. 5. Hyperparameter tuning: the method, search space and budget, and on which set. 6. Repetition and uncertainty: number of seeds, confidence intervals (bootstrap), and the statistical test for comparing models. 7. Error analysis and subgroup analysis (fairness). 8. A reproducibility checklist: versions, seeds, settings and hardware. 9. The empty results table that I will fill in. Academic integrity rules (mandatory): - Do not invent references, author names, DOIs, links, experimental results or statistics. Where a source is needed, write [source needed]. - Do not claim to have read a study or file whose text I have not given you. - Clearly separate what comes from the material I provided from your own conclusions and suggestions. - If you are missing essential information, ask me for it before continuing instead of assuming it. - Do not predict performance figures. Write the answer in [Answer language]. You are a senior machine learning researcher and a reviewer for conferences in the field, specialising in {{field}}. Task: {{task}} Data: {{data}} Proposed models: {{models_list}} Computing resources: {{compute}} Design the experimental protocol: 1. Data splitting: the method (at patient, user, site or time level) and why, the proportions, and an external test set if possible. 2. Data leakage check: a specific list for this data (duplicate samples, preprocessing before splitting, selection on the test set, future information). 3. Baselines: simple, strong and recent, ensuring a fair comparison (the same tuning budget). 4. Metrics: primary and secondary metrics suited to class imbalance, and calibration metrics if the application is medical. 5. Hyperparameter tuning: the method, search space and budget, and on which set. 6. Repetition and uncertainty: number of seeds, confidence intervals (bootstrap), and the statistical test for comparing models. 7. Error analysis and subgroup analysis (fairness). 8. A reproducibility checklist: versions, seeds, settings and hardware. 9. The empty results table that I will fill in. Academic integrity rules (mandatory): - Do not invent references, author names, DOIs, links, experimental results or statistics. Where a source is needed, write [source needed]. - Do not claim to have read a study or file whose text I have not given you. - Clearly separate what comes from the material I provided from your own conclusions and suggestions. - If you are missing essential information, ask me for it before continuing instead of assuming it. - Do not predict performance figures. Write the answer in {{lang}}.
Customise the prompt
Fill in only what suits you. The line of an empty optional field is removed from the final text, and empty required fields stay in brackets for you to complete later.
Preview of the customised prompt
You are a senior machine learning researcher and a reviewer for conferences in the field, specialising in [Specific field]. Task: [Task] Data: [Data description] Proposed models: [Proposed models] Computing resources: [Computing resources] Design the experimental protocol: 1. Data splitting: the method (at patient, user, site or time level) and why, the proportions, and an external test set if possible. 2. Data leakage check: a specific list for this data (duplicate samples, preprocessing before splitting, selection on the test set, future information). 3. Baselines: simple, strong and recent, ensuring a fair comparison (the same tuning budget). 4. Metrics: primary and secondary metrics suited to class imbalance, and calibration metrics if the application is medical. 5. Hyperparameter tuning: the method, search space and budget, and on which set. 6. Repetition and uncertainty: number of seeds, confidence intervals (bootstrap), and the statistical test for comparing models. 7. Error analysis and subgroup analysis (fairness). 8. A reproducibility checklist: versions, seeds, settings and hardware. 9. The empty results table that I will fill in. Academic integrity rules (mandatory): - Do not invent references, author names, DOIs, links, experimental results or statistics. Where a source is needed, write [source needed]. - Do not claim to have read a study or file whose text I have not given you. - Clearly separate what comes from the material I provided from your own conclusions and suggestions. - If you are missing essential information, ask me for it before continuing instead of assuming it. - Do not predict performance figures. Write the answer in [Answer language].
Variables you can customise
- [Specific field]
- Required
- [Task]
- Required
- [Data description]
- Required
- [Proposed models]
- Optional
- [Computing resources]
- Optional
- [Answer language]
- Optional, choose from: Arabic, English, Arabic with English terms in parentheses, default value: Arabic with English terms in parentheses
Usage example
Example input
Expected output
How to use it
- Put the protocol in the methodology chapter.
- Use it later to write the evaluation report.
Tips for better results
- State whether each patient or user has several samples; this is the most common source of leakage.
Limitations and warnings
- The protocol is general; check the reporting guidelines in your field (such as TRIPOD+AI or CLAIM for medical applications).
Suggested tools for using this prompt
Tools from the directory of research and academic publishing tools that are actually linked to this prompt's task, with the reason each tool is suggested.
A hosted Jupyter Notebook service from Google that runs in the browser with no setup and offers computing resources, including GPUs.
Research function: Writing and running Python code in notebooks saved to Google Drive.
Why it is suggested: To run experiments on cloud GPUs.
A free, AI-powered academic search engine from the Allen Institute for AI that indexes more than 200 million papers.
Research function: Searching for and discovering scientific papers, with automatic summaries and highlights that help you grasp a paper quickly.
Why it is suggested: To find recent baselines for the same task.
A general AI assistant from OpenAI used for brainstorming, drafting and summarising.
Research function: General text chat that helps you generate and organise ideas and improve your wording.
Why it is suggested: A general language model where you can paste the prompt directly and continue the conversation to improve the result.
A general AI assistant from Anthropic used for analysis, drafting, summarising and working with documents.
Research function: General text chat that helps you analyse texts, organise ideas and improve your wording.
Why it is suggested: A general language model where you can paste the prompt directly and continue the conversation to improve the result.
A free, open-source analytics platform for building data processing and analysis workflows visually.
Research function: Building visual workflows for data preparation, analysis and machine learning, with optional coding.
Why it is suggested: To build visual processing pipelines for simple experiments.
In research workflows
- AI Research Project Workflow (step 4)
Similar prompts
Designing and Interpreting an Ablation Study
Identifies the components of your model that need testing, designs the ablation experiment matrix with the fewest runs, and helps you interpret the results honestly.
Writing a Model Evaluation Report
Turns your actual experimental results into an organised results section: comparison tables, statistical significance, error analysis and limitations, without embellishment.
Preparing a Reproducibility Package
Prepares the checklist and files for a complete reproducibility package: README, environment, commands, data, licence and citation, following conference and journal practice.
تصميم مقارنة عادلة بين النماذج
وحّد البيانات والمعالجة والتقسيم والميزانية وآلية الضبط بين النماذج.
