Power and energy answer different questions.
Power describes a rate of energy use. Energy describes the amount consumed over a period. A server can draw less power and still consume more energy to finish a task if that task takes longer.
For a fixed batch, measure the energy used across the run and divide it by the number of valid completed tasks. For a service, define a representative interval and account for completed requests, load and performance.
= measured energy ÷ valid completed units
The unit needs to mean something: a completed job, a validated transaction or a request at a defined service level. A comparison is only useful when that definition is consistent.
State the measurement boundary.
A CPU package counter and a rack power meter describe different parts of the system. Neither should silently stand in for the other.
Google’s published work on AI inference considers host resources and infrastructure overhead alongside accelerator activity. It illustrates why a broader accounting boundary can change the interpretation of a result. Its particular workload mix is not a default for every operator.
Read Google’s measurement approach
Keep the comparison matched.
Record hardware, firmware, operating system, runtime, input data and concurrency. Establish warm-up conditions, include the intervention’s resource use, and repeat the baseline and candidate runs.
Report the result together with throughput, relevant latency percentiles, correctness checks and the spread across runs. That makes trade-offs and uncertainty visible.
When the optimisation depends on a repeated workload or warmed state, make that condition explicit. The test should represent how the system will actually be used.
Apply the same standard to CARMIX.
CARMIX is at the internal benchmarking stage. A proposed evaluation should agree on the workload, instrumentation and acceptance criteria before testing.
The useful question
How much energy does this configuration use to complete the same valid work at the required performance level?