GreenComputing-TP4

GreenComputing_TP4

Model Context Protocol (MCP) with Claude Desktop, FastMCP and CodeCarbon

🚀 AI & Data Pipeline Sustainability Benchmark

This project is a detailed benchmark study comparing the Cost, Energy Consumption, and Performance of Large Language Models (LLMs) with the sustainability and efficiency of different data pipeline technologies (File Formats and Processing Frameworks). All energy and CO2 emissions are measured using the CodeCarbon library.

---

1. LLM Performance and Cost Analysis (TP1: Compar'IA)

The goal of this phase is to evaluate LLMs of varying sizes across a range of tasks, focusing on the trade-off between output quality and operational cost (time, energy, and money).

1.1. Models Benchmarked

Size Category	Models Tested (6 total)
Small	[cite_start]Meta LLaMA 3.1 8B, Gemma 8B [cite: 5]
Medium	[cite_start]Mistral Small, GPT-OSS 20B [cite: 6]
Large	[cite_start]GPT-5, DeepSeek R1 [cite: 7]

1.2. Tasks Covered (30 Prompts)

Tasks are categorized by complexity:

• [cite_start]Easy Factual & Rewriting: (1-10) [cite: 31]
• [cite_start]Reasoning & Quantitative: (11-15) (e.g., math problems, conversions) [cite: 42, 43, 47]
• [cite_start]Programming & Debugging: (16-20) (e.g., Python code, Big-O complexity) [cite: 48, 49, 53]
• [cite_start]Harder Knowledge & Reasoning: (21-25) (e.g., complex comparisons, explaining architecture) [cite: 54, 55, 58]
• [cite_start]Advanced/Creative & Multi-step: (26-30) (e.g., project planning, drafting legal disclaimers) [cite: 60, 61, 63]

1.3. Deliverables for TP1

• [cite_start]Spreadsheet: Contains all collected metrics (Quality, Latency, Energy, CO2, Cost) for all 6 models across 30 tasks[cite: 68].
• Dashboard Visualisations:
  • [cite_start]Quality vs Energy graph[cite: 70].
  • [cite_start]Quality vs Cost graph[cite: 71].
  • [cite_start]Latency vs Model size comparison[cite: 72].
  • [cite_start]Final best trade-off ranking table[cite: 73].
• [cite_start]Slide Deck: 2-3 slides summarizing key findings and recommending which model is best for specific use-cases[cite: 74].

---

2. Data Pipeline Efficiency Benchmark

[cite_start]This phase compares different frameworks and file formats for data processing, measuring runtime and emissions for an identical pipeline (merging books and reviews data [cite: 227, 235, 250]).

2.1. Framework Comparison (TP2: Pandas vs. PySpark)

[cite_start]We compare two pipelines performing identical steps (Load, Clean, Join, Compute Metrics, Text Processing, Save) [cite: 273] using Pandas and PySpark.

Metric	[cite_start]Pandas Baseline [cite: 288, 290]	[cite_start]PySpark (Local Mode) [cite: 299, 302]
Duration	[cite_start]Measured with CodeCarbon Tracker [cite: 292]	[cite_start]Measured with CodeCarbon Tracker [cite: 302]
CO2 / Energy	[cite_start]Emissions Tracker output [cite: 296, 297]	[cite_start]Emissions Tracker output [cite: 302]

Analysis Focus:

• [cite_start]Compare runtime and CO2 emissions per framework[cite: 308, 312, 314].
• [cite_start]Discuss the impact of parallelism on sustainability[cite: 315].
• [cite_start]Compute and analyze the Efficiency Ratio (Data Processed / CO2 emitted)[cite: 309, 310].
• [cite_start]Eco-Design Experiment: Implement and re-measure emissions after one optimization (e.g., filtering columns, caching)[cite: 317, 318, 320, 321].

2.2. File Format Comparison (TP3: CSV vs. Parquet)

[cite_start]We compare the entire ETL pipeline when reading/writing CSV files versus Parquet files (using Snappy compression)[cite: 389, 390, 420].

Metric	[cite_start]CSV Pipeline [cite: 404]	[cite_start]Parquet Pipeline (Snappy) [cite: 419]
Performance	[cite_start]pd.read_csv(), df.to_csv() [cite: 411, 416]	[cite_start]pd.read_parquet(), df.to_parquet() [cite: 430, 435]
Storage Footprint	Measured file size	[cite_start]Measured file size (expected $5-10\times$ reduction) [cite: 467]
Sustainability	[cite_start]CO2 and Energy measured via CodeCarbon [cite: 406]	[cite_start]CO2 and Energy measured via CodeCarbon [cite: 427]

Analysis Focus:

• [cite_start]Compare runtime, energy, and CO2 for I/O operations (Load/Save)[cite: 440, 446].
• [cite_start]Quantify storage savings achieved by the columnar format (Parquet)[cite: 449].
• [cite_start]Discuss the trade-offs between readability, performance, and sustainability of the formats[cite: 464].

2.3. Deliverables for TP2 & TP3

• [cite_start]Code: tp_codecarbon_books.ipynb (TP2) and tp_codecarbon_parquet.ipynb (TP3)[cite: 327, 461].
• [cite_start]Logs: emissions_pandas.csv, emissions_spark.csv, emissions_csv.csv, emissions_parquet.csv[cite: 328, 329, 462].
• [cite_start]Figures: Comparison charts for emissions and runtime for both frameworks and file formats[cite: 330, 463].
• [cite_start]Reflection: Short written reflections on the comparison of frameworks (TP2) and file formats (TP3)[cite: 331, 464].

---

3. LLM Tooling and Environmental Cost (TP4: Model Context Protocol)

This final phase investigates the environmental impact of offloading tasks from the LLM's "reasoning" (internal inference) to external Model Context Protocol (MCP) tool calls, instrumented with CodeCarbon.

3.1. Tools to Implement

[cite_start]The server must implement at least three tools using the fastmcp and codecarbon packages[cite: 194, 195]:

Exercise	Tool Description	Tool Goal
A	[cite_start]add(a: int, b: int) [cite: 114]	[cite_start]Compare model internal computation vs. tool call for a large sum[cite: 145, 146].
B	[cite_start]factorial(n: int) [cite: 155]	[cite_start]Compare model internal computation vs. tool call for a recursive math function[cite: 162].
C	[cite_start]analyze_file(path: str) [cite: 172]	[cite_start]Tool for reading a local file, counting words, and returning a JSON summary[cite: 167, 168, 169].

3.2. Metrics & Comparison

You must fill a comparative table for each task, measuring:

• [cite_start]Approach: Model Reasoning (Prompt-only) vs. MCP Tool Call[cite: 178].
• [cite_start]Metrics: Latency (s), Energy (kWh), Emissions (kgCO2eq)[cite: 178].

3.3. Key Analysis Questions (Required in Slide Deck)

• In which scenario does the MCP approach reduce hallucination risk? [cite_start]Why? [cite: 186]
• [cite_start]Does tool latency outweigh energy savings? [cite: 187]
• [cite_start]Which tasks are better done by tools instead of models? [cite: 189]
• [cite_start]If deployed at scale, which approach is more sustainable (model-only vs. tool-assisted)? [cite: 190]

3.4. Deliverables for TP4

• [cite_start]Code: server.py with the MCP server, at least three implemented and instrumented tools, and CodeCarbon integration[cite: 193, 194, 195].
• [cite_start]Spreadsheet: The comparative table with latency, energy, and emissions results[cite: 196, 197, 198].
• [cite_start]Slide Deck (4-6 slides): Includes screenshots of Claude Desktop tool calls, emissions logs, comparison charts, and final recommendations based on both environmental cost and operational reliability[cite: 202, 203, 204, 205, 206, 214, 215].