Define the decision first.
Begin with the question the evaluation needs to answer. Is the aim to establish energy efficiency on one workload, explore integration, or determine whether a broader trial is justified?
Select a bounded environment and define success criteria together. Include unacceptable performance regressions, correctness failures and operational stop conditions.
A pilot is easier to interpret when the decision, workload and measurement boundary are explicit before the first result is collected.
Make the baseline inspectable.
Record the baseline configuration and workload. Document how power and energy are measured, how valid work is counted and how performance is assessed.
Consider the existing optimisations already in use. A baseline should represent an appropriate operating configuration, not an artificially weak comparison.
Use repeated runs and account for warm-up, background load and run order. State uncertainty instead of presenting a single favourable run as the expected outcome.
Include the operational questions.
Before deployment, review compatibility, required privileges, data handling and network access. Agree on monitoring and on how the environment returns to its baseline state.
Technical benefits and deployment costs belong in the same discussion. A result that requires impractical changes may not be valuable for the intended environment.
The proposed scope, timing, responsibilities and commercial terms should be documented by both teams.
Finish with a reviewable record.
The evaluation report should include the environment, workload, test procedure, raw or appropriately summarised measurements, variability, performance and correctness checks, and known limitations.
Separate observed results from any extrapolation. Explain which additional assumptions would be needed to apply a result to a fleet or facility.
A proposed CARMIX evaluation
Start with a technical discussion. Agree on the workload, the evidence to collect, and the decision the result should support. Pilot details are established after reviewing the environment.