{"id":"13d34ad5-6ec6-474c-b31d-2a2147d868d4","slug":"codonfm-embed","kind":"skill","name":"codonfm-embed","summary":"Validate coding-sequence CSVs, extract public CodonFM/Encodon embeddings, and choose checkpoints for downstream property modeling.","owner":"NVIDIA","repoUrl":"https://github.com/NVIDIA/skills/tree/HEAD/skills/bionemo-codonfm-embed","license":"Apache-2.0","licenseTier":"permissive","trust":"official","stars":3586,"match":null,"fewStars":false,"format":"skill","createdAt":null,"addedAt":"2026-10-14T05:41:53.637Z","logoUrl":"https://avatars.githubusercontent.com/u/1728152?v=4","linkOnly":false,"collection":{"name":"skills","repo":"nvidia/skills"},"skillPath":"skills/bionemo-codonfm-embed","categories":["developer-tools"],"tags":["agentic-ai","ai-agents","claude-code","codex","coding-agents","cuda","developer-tools","llm"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-14T05:41:53.635Z","sample":false,"url":"https://conduits.dev/tools/codonfm-embed","language":"Python","transports":[],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Extract public Encodon embeddings\n\n## Purpose\n\nExtract one frozen CLS vector per coding sequence with public Encodon v1.\nSupport input validation, command preparation, extraction, and checkpoint\nselection for translation efficiency, expression, or mRNA stability modeling.\nExtraction does not automatically train a downstream regressor.\n\n## Prerequisites\n\n- Validation needs Python 3 standard library only; no GPU, weights, or API key.\n- Execution needs the public CodonFM checkout, its `requirements.txt` environment,\n  a compatible NVIDIA GPU, and local checkpoint weights. A metadata JSON is not\n  a checkpoint. A `.safetensors` file needs its sibling `config.json`; `.ckpt`\n  checkpoints are also supported by the public loader.\n- Run `python -m src.runner` from the CodonFM repository root. In an isolated\n  workspace, use supplied source artifacts; source paths below are relative to\n  that checkout or source archive, not this skill directory.\n\n## Inputs\n\nInput source precedence: explicit user prompt arguments, then supplied\nfiles/checkpoint metadata, then inspected public runner defaults. Resolve\nconflicting model names and checkpoint metadata before execution. Supplied 80M\nmetadata is useful for preparing an 80M command; it does not restrict an\nopen-ended recommendation to that size.\n\nRequired for validation: a CSV. Required for extraction: the CSV, checkpoint,\nmatching model name, and output directory. Optional: context length and batch\nsize overrides. Checkpoint-selection questions can be answered without a CSV.\n\n| Input | Requirement or default |\n| --- | --- |\n| Sequence CSV | Columns `id`, `ref_seq`, `value`, `split`; extra columns allowed |\n| `id` | Nonblank, unique IDs for unambiguous output association |\n| `ref_seq` | Coding sequence, uppercase DNA `A/C/G/T`, length divisible by three; public dataset converts uppercase `U` to `T` |\n| `value` | Numeric label; use `0.0` for new extraction-only data, preserve supplied labels |\n| `split` | Only exact `test` values enter extraction; blank/other values are excluded |\n| Checkpoint and model | Match weights/config to `encodon_80m`, `encodon_600m`, or `encodon_1b` |\n| Context length | Public runner default `2048` tokens, including CLS and SEP |\n| Output directory | A fresh run directory with an empty predictions directory |\n\n## Instructions\n\n1. **Choose the requested workflow.** For a checkpoint/performance question,\n   read [checkpoint selection](references/checkpoint-selection.md) and answer\n   from public benchmark evidence. For the strongest published downstream\n   results, prefer the public **1B random-mask checkpoint** when resources allow;\n   80M is a demonstration or resource-constrained choice. A small labeled set\n   alone does not establish that 80M frozen features are better. Do not download\n   weights or inspect the entire source tree just to make a recommendation.\n2. **Inspect supplied source only where needed.** Confirm runner/config,\n   `src/data/codon_bert_dataset.py`, `src/data/preprocess/codon_sequence.py`,\n   `src/inference/encodon.py`, or `src/utils/pred_writer.py` for the relevant\n   behavior. Read ZIP members with `zipfile.ZipFile.namelist()` and `.read()`;\n   source inspection does not need extraction. If a checkout is needed, use a\n   new directory from `tempfile.mkdtemp()` or `mktemp -d`, without deleting or\n   overwriting an existing directory. For Decodon support questions, inspect\n   runner/config and model/inference modules, cite the inspected files, explain\n   the missing public implementation, and finish there.\n3. **Validate the CSV before running extraction.** Run the bundled checker\n   below with the intended context length. Report per-row verdicts using CSV\n   row numbers as well as IDs, since IDs can repeat. Separate excluded rows,\n   invalid inputs, duplicate-ID warnings, and truncation. Propose fixes without\n   silently rewriting supplied data. The checker is a preflight, not model\n   execution or proof of biological CDS validity.\n4. **Deliver the requested preparation or execution.** For preparation, return\n   a complete command with resolved paths (or clearly identified prerequisites),\n   the test-row count, validation findings, and the output contract below.\n   Include all task/dataset/process flags in the final answer, even if already\n   shown in a tool call. For extraction, reuse/download the chosen checkpoint\n   when needed, execute once resources are ready, and verify the saved arrays.\n   If resources are missing, finish preparation and state what is missing.\n\n## Available Scripts\n\n| Script | Purpose | Arguments |\n| --- | --- | --- |\n| [validate_inputs.py](scripts/validate_inputs.py) | Read-only CSV validation and per-row verdicts | Required CSV path; optional `--context-length` (default `2048`) |\n\nRun the preflight with Python; `CODONFM_SKILL_DIR` is the directory containing this file:\n\n```bash\npython \"$CODONFM_SKILL_DIR/scripts/validate_inputs.py\" \"$CODONFM_DATA_PATH\" \\\n    --context-length 2048\n```\n\nThe checker prints JSON. Exit `0` means no findings, `1` means row findings to\nreview (including exclusions/warnings), and `2` means a file/schema error.\nNeither warnings nor exclusions imply that the public runner will crash.\n\n## Output Format\n\nThe checker emits JSON with `total_rows`, `test_rows`, `excluded_rows`,\n`context_length`, `codon_limit`, `warnings`, and `rows`. Each row records its\none-based data-row number (excluding the header), ID, split, verdict, issues,\nsequence/value validity, and retained/lost codons. A file/schema error emits\n`error` and `csv` instead. These are preflight findings, not generated embeddings.\n\n## Examples\n\nSet `CODONFM_DATA_PATH` to the CSV, `CODONFM_CHECKPOINT_PATH` to the weights,\n`CODONFM_MODEL_NAME` to the matching architecture, and `CODONFM_RUN_DIR` to a\nfresh output directory. Substitute actual paths in a prepared command:\n\n```bash\npython -m src.runner eval \\\n    --task_type embedding_prediction \\\n    --process_item codon_sequence \\\n    --dataset_name CodonBertDataset \\\n    --exp_name embed_extract \\\n    --model_name \"$CODONFM_MODEL_NAME\" \\\n    --checkpoint_path \"$CODONFM_CHECKPOINT_PATH\" \\\n    --data_path \"$CODONFM_DATA_PATH\" \\\n    --context_length 2048 \\\n    --num_nodes 1 \\\n    --num_gpus 1 \\\n    --num_workers 0 \\\n    --val_batch_size 2 \\\n    --out_dir \"$CODONFM_RUN_DIR\" \\\n    --predictions_output_dir \"$CODONFM_RUN_DIR/predictions\"\n```\n\nFor a low-cost demonstration, `encodon_80m` matches\n`nvidia/NV-CodonFM-Encodon-80M-v1`, revision\n`399ca9fe17b57941a7bebc6788033919b417413c`, file\n`NV-CodonFM-Encodon-80M-v1.safetensors` and sibling `config.json`.\n\n## Outputs\n\n- Under `--predictions_output_dir`, `embeddings_merged.npy` contains frozen\n  final-layer CLS vectors, shape `(processed_rows, hidden_size)`.\n- `ids_merged.npy` is index-aligned: embedding row `i` belongs to ID row `i`.\n  Use these IDs to join to the CSV; do not assume every CSV row was retained.\n  Duplicate IDs make that join ambiguous even when extraction succeeds.\n- For the one-GPU example, verify both arrays have the expected test-row count,\n  embeddings are finite, and width matches checkpoint config (`1024` for 80M,\n  `2048` for 600M/1B). Do not fabricate arrays for a preparation-only request.\n\nThe public checkout's downstream-model references are:\n\n- `notebooks/4-EnCodon-Downstream-Task-riboNN.ipynb`\n- `notebooks/5-EnCodon-Downstream-Task-mRFP-expression.ipynb`\n- `notebooks/6-EnCodon-Downstream-Task-mRNA-stability.ipynb`\n\n## Limitations\n\n- Public v1 has no Decodon model/inference implementation, Decodon notebooks,\n  `notebooks/te_predictor.py`, or `notebooks/mfe_predictor.py`.\n- At context length `2048`, retain the first `2046` codons; any remaining\n  3-prime sequence is lost. Increasing the flag does not validate a longer\n  context. Disclose deliberate cropping or a separate chunking/aggregation\n  strategy; neither is equivalent to embedding the complete sequence once.\n- `--dryrun` builds runtime configuration, may create directories, and needs\n  ML dependencies; it reads neither the CSV nor the weights and is not input\n  validation.\n- Do not claim a benchmark-trained regressor generalizes to a new organism,\n  cell type, or assay without new labeled validation data.\n- Do not invoke this skill for a generic expression-prediction request that\n  does not mention CodonFM or Encodon.\n\n## Troubleshooting\n\n| Symptom | Cause and action |\n| --- | --- |\n| Missing `split` column | Eval requests the test split despite the dataset docstring calling this column optional; add an explicit split column to a corrected copy |\n| Fewer output rows | Blank/non-`test` split values are silently filtered; set intended extraction rows to exact `test` in a corrected copy |\n| Repeated output IDs | Duplicate input IDs are not rejected; assign unique IDs while preserving a mapping to the original rows |\n| Oversized sequence | Preprocessing truncates at `context_length - 2` codons; report retained/lost lengths and agree on a sequence-handling strategy |\n| Missing weights or dependencies | Complete validation/command preparation; metadata and `--dryrun` do not substitute for weights |\n| Merge failure on a repeated run | The writer scans `.npy` files; use a fresh predictions directory to avoid stale shards or merged arrays |"}