Discussion on Option 3: different functional units

[caohongliu]

Scope of CO2 Emissions in the AI Product Lifecycle
The CO2 emissions associated with AI products occur at various stages of their lifecycle. These stages include:

1. Data Collection: Gathering raw data from various sources.
2. Data Engineering: Processing and preparing data for model training.
3. Model Training: Developing AI models using computational resources.
4. Model Testing: Evaluating the performance of AI models.
5. Model Deployment: Implementing AI models in real-world applications.
6. Model Monitoring: Continuously observing model performance and making necessary adjustments.
7. Model Retraining: Updating models with new data to maintain accuracy and relevance.

Sources of CO2 Emissions
The primary sources of CO2 emissions across the AI product lifecycle include:

• Computational Resources: Energy consumption by CPUs, GPUs, and other hardware during data processing, model training, and inference.
• Storage: Energy required to store large datasets and model parameters.
• Data Transfer: Emissions from transferring data across networks and cloud services.
• Embodied Emissions: CO2 emissions from the production and disposal of hardware used in AI development.

Target Users and Benefits
The proposed standard will serve multiple user groups, each benefiting in distinct ways:

1. AI Product Companies: Companies offering AI products can use CO2e measurements to enhance their sustainability image and reduce operational costs through more efficient resource utilization.
2. AI Developers, Researchers, and Engineers: These professionals can leverage CO2e measurements to design and implement more sustainable AI solutions, optimizing their workflows to minimize environmental impact.
3. End Users: Individuals using AI applications (e.g., OpenAI apps on smartphones) can make informed decisions to reduce their personal carbon footprint by choosing more sustainable AI products.

Functional Units for CO2 Emission Measurement
Recognizing that a single functional unit may not suit all users, we propose the adoption of multiple functional units to measure CO2 emissions. These units could include:

1. Per Model Training

   • Definition: The total CO2 emissions generated from the beginning to the end of a single model training session.
   • Measurement:
     • Compute: Energy consumption of CPUs/GPUs during training.
     • Storage: Energy used for storing training data and intermediate results.
     • Embodied Emissions: Emissions from the manufacturing of hardware used.
   • Adoption: This unit is particularly useful for AI developers and researchers to evaluate and optimize the efficiency of their training processes.

2. Per Inference

   • Definition: The CO2 emissions generated per single inference or prediction made by the AI model.
   • Measurement:
     • Compute: Energy consumed during the inference process.
     • Storage: Energy used to store the model and retrieve it for inference.
     • Data Transfer: Energy consumption for transferring input data and results.
   • Adoption: This unit is beneficial for companies offering AI products and end users to understand the impact of using AI services.

3. Per Data storage

   • Definition: The CO2 emissions of data stored throughout the AI lifecycle (including training data, usage data, etc.).
   • Measurement:
     • Storage: Energy used to store the data.
     • Data Transfer: Energy consumption for moving data between systems.
   • Adoption: Useful for data engineers and companies to monitor and optimize data storage and handling processes.

4. Per User Session

   • Definition: The CO2 emissions generated during a single user session interacting with an AI product.
   • Measurement:
     • Compute: Energy consumed by servers during the session.
     • Storage: Energy used to retrieve and store user-specific data.
     • Data Transfer: Energy consumption for data exchange between user devices and servers.
   • Adoption: End users and companies can use this unit to understand the environmental impact of user interactions with AI applications.

5. Per Model Lifecycle (More difficult to measure)

   • Definition: The total CO2 emissions from the entire lifecycle of an AI model, from development to decommissioning.
   • Measurement:
     • Compute: Cumulative energy consumption of all lifecycle stages.
     • Storage: Total energy used for storing data and models throughout the lifecycle.
     • Data Transfer: Total energy consumption for data transfers across all stages.
     • Embodied Emissions: Emissions from hardware manufacturing and disposal.
   • Adoption: This comprehensive unit is ideal for companies and developers to assess the long-term sustainability of AI models.

[samuelrince]

Thank you, @caohongliu, for this comprehensive initial proposal! I'm largely aligned with your thinking, though I believe we might benefit from some simplifications.

My main concern is the practical challenge future SCI for AI users will face when attempting to report emissions for a model.

As of today (and correct me if I am wrong), we have very limited data on the environmental impacts of storage and networking. The networking aspect is quite challenging - you have outbound traffic for data collection, inbound-only traffic during cluster training, and both inbound and outbound traffic for data replication strategies. This complexity, combined with the scarcity of reliable information on associated energy consumption or GHG emissions, makes accurate estimation of impacts very difficult.

So, I propose:

1. Removing data storage and transfer components from all FUs ;
2. Eliminating FU3 (Per Data storage) entirely ;
3. Removing FU5 (Per Model Lifecycle), as this is essentially a complete LCA. We already have standards for this type of analysis and if a company went for this, they would not need the SCI for AI.

Do you think these simplifications are reasonable, or am I suggesting too radical a reduction in scope? I'm trying to balance being comprehensive with making something that's actually usable in practice.

[caohongliu]

Here are some references for CO2e for storage and networking, among which the most widely adopted is Cloud Carbon Footprint (CCF) [3]:
[1] Joshua Aslan, Kieren Mayers, Jonathan G. Koomey, and Chris France. “Electricity intensity of Internet data transmission: Untangling the estimates.” Journal of Industrial Ecology. doi:10.1111/jiec.12630. August 1, 2017.
[2] The carbon benefits of cloud computing A study on the Microsoft Cloud in partnership with WSP
[3] https://www.cloudcarbonfootprint.org/

I think it is challenging if we want to propose something new. Cheers!

[jawache]

Thanks @caohongliu and @samuelrince agreed very good thoughts here.

If I might give some advice regarding how to proceed. My general recommendation is to propose changes to the spec in small atomic chunks. Very large changes to a spec often get blocked as it takes a lot of effort from people to understand the impact of a large change to a spec. But 4 smaller PRs allow people to absorb the impact of each change quickly and greatly increases the chance of quick consensus.

1. I think the personas are a great idea and a function unit per persona makes a lot of sense (and a good model to use for future decisions around functional units), a section in the spec to define "Personas" would be a good initial PR and would be easy to get consensus on.

2. Then define a Functional Unit per-persona with clarity on which "stages" in the lifecycle should be included in each functional unit. A PR just for that might be a good next step and require more effort to get consensus.

3. I wouldn't "hard-code" at this stage whether to include compute, data and storage (and by omission leave out everything else that has been mentioned in the workshops like monitoring etc...), there is risk that will block a PR that otherwise would have broad consensus.

Just trying to suggest a path forward, it's usually better to just figure out the part of your idea that have consensus, create a PR for that, and then layer content in small batches after that.