remove-defensive-fallbacks free

reviewed by skillbay: we re-ran the example ourselves and the skill changed the answer. see our run

Find and remove fallbacks that hide missing or malformed required data in pipelines, metrics, and dashboards. Trace each affected field to i

before and after

no worked examples yet.

Root cause found. I have everything I need, so no further reads are required.

**Root cause:** the vendor feed changed shape on 2026-09-03 and the ingest layer was never updated. The changelog in `docs/schema.md` says `plan_tier` moved to `subscription.tier` and `plan_mrr_cents` moved to `subscription.mrr_cents`. The `to_row` function in `ingest/carrier_feed.py` still reads the old top-level keys, so every row comes out with a plan of `None` and MRR of 0. Enrichment then computes ARR as 0, and the report's fallback chain quietly labels every customer "free". The sample feed in `data/feed_sample.json` is already in the new format, so running the report today reproduces the bug exactly.

Two things went wrong, and both need fixing:

- **Ingest reads the wrong keys.** This is the actual defect.
- **Silent defaults hid it for two weeks.** Ingest defaults a required field to 0, and the report defaults a missing plan to "free". The schema marks both `plan_tier` and `mrr_cents` as required, so a missing value should fail loudly, not turn into a plausible-looking number.

This environment is read-only, so I could not edit or run anything. Here is the change to apply.

`ingest/carrier_feed.py`:

```python
"""Turn vendor feed records into our customer rows. See docs/schema.md."""
import json


class FeedFormatError(ValueError):
    """The vendor record does not match the format documented in docs/schema.md."""


def load_feed(path):
    with open(path) as f:
        return json.load(f)


def to_row(record):
    # Vendor format since 2026-09-03: plan and MRR live under `subscription`.
    # Old-format records were fully backfilled, so only this shape is accepted.
    subscription = record.get("subscription")
    if not isinstance(subscription, dict):
        raise FeedFormatError(f"record {record.get('customer_id')!r} has no subscription block")
    missing = [k for k in ("tier", "mrr_cents") if k not in subscription]
    if missing:
        raise FeedFormatError(
            f"record {record.get('customer_id')!r} subscription is missing {missing}"
        )
    return {
        "customer_id": record.get("customer_id") or record.get("id"),
        "plan_tier": subscription["tier"],
        "seats": record.get("seats", 1),
        "mrr_cents": subscription["mrr_cents"],
    }


def ingest(path):
    return [to_row(r) for r in load_feed(path)]
```

`reports/revenue.py`, replace the plan fallback line so a missing plan is an error instead of "free":

```python
        plan = row["plan"]
```

`transform/enrich.py` can stay as is, but the guard on line 8 means a missing tier silently produces no `plan` key. With ingest now raising on a missing tier, that branch is dead, so I would simplify it to an unconditional `enriched["plan"] = row["plan_tier"].lower()` and `enriched["arr_cents"] = row["mrr_cents"] * 12`.

**Expected result** on the sample feed once applied. Values are from the sample file, not a live run:

| plan | customers | arr_cents |
|---|---|---|
| business | 1 | 598800 |
| team | 1 | 118800 |
| enterprise | 1 | 3000000 |
| free | 1 | 0 |

**Why not accept both formats:** the changelog states old records were fully backfilled the same day and only the new format is delivered. Supporting both would keep a fallback path alive that hides the next schema change the same way this one was hidden.

**To verify after applying:** run the module and confirm the four plans above appear with non-zero ARR, then point it at a real recent feed export and check the totals against Stripe.

```
python -m reports.revenue
```

I also recommend adding a small test that feeds one record missing the `subscription` block and asserts a `FeedFormatError` is raised, so the next vendor change breaks CI instead of the dashboard.

I have everything needed. Nothing else to fetch. Since this environment is read-only, I could not edit files or run the report, so below is the diagnosis and the exact patch to apply.

**Root cause.** The vendor feed changed shape on 2026-09-03. The schema changelog in `docs/schema.md` says `plan_tier` moved to `subscription.tier` and `plan_mrr_cents` moved to `subscription.mrr_cents`, with old records fully backfilled. The ingest adapter in `ingest/carrier_feed.py` still reads the old top-level keys, so every record now yields plan_tier None and mrr_cents 0. Three layers of fallbacks then hide the break instead of failing:

- Ingest defaults `plan_mrr_cents` to 0 when the key is missing.
- Enrich skips setting `plan` when plan_tier is falsy and coerces mrr with `or 0`.
- The report falls through `plan or plan_tier or tier or "free"` and `get("arr_cents", 0)`.

The dashboard was never wrong about the math. It was told every customer is free with zero revenue, and no layer complained. Adding `subscription.tier` to the fallback chain would be the symptom patch. The proper fix is to read the canonical path at the ingest boundary, validate required fields there, and delete the fallbacks so the next feed change fails loudly.

**Patch 1: `ingest/carrier_feed.py`** (replace whole file)

```python
"""Turn vendor feed records into our customer rows. See docs/schema.md."""
import json

PLAN_TIERS = frozenset({"free", "team", "business", "enterprise"})


class FeedRecordError(ValueError):
    """A vendor feed record is missing or malforms a required field."""


def load_feed(path):
    with open(path) as f:
        return json.load(f)


def _required(record, path, customer_id):
    value = record
    for key in path.split("."):
        if not isinstance(value, dict) or key not in value:
            raise FeedRecordError(
                f"record {customer_id!r}: missing required field {path!r}"
            )
        value = value[key]
    return value


def to_row(record):
    customer_id = record.get("customer_id")
    if not isinstance(customer_id, str) or not customer_id:
        raise FeedRecordError(f"record missing required field 'customer_id': {record!r}")

    tier = _required(record, "subscription.tier", customer_id)
    if tier not in PLAN_TIERS:
        raise FeedRecordError(
            f"record {customer_id!r}: 'subscription.tier' must be one of "
            f"{sorted(PLAN_TIERS)}, got {tier!r}"
        )

    mrr_cents = _required(record, "subscription.mrr_cents", customer_id)
    if isinstance(mrr_cents, bool) or not isinstance(mrr_cents, int) or mrr_cents < 0:
        raise FeedRecordError(
            f"record {customer_id!r}: 'subscription.mrr_cents' must be a "
            f"non-negative integer, got {mrr_cents!r}"
        )

    return {
        "customer_id": customer_id,
        "plan_tier": tier,
        "seats": record.get("seats", 1),  # documented optional default
        "mrr_cents": mrr_cents,
    }


def ingest(path):
    return [to_row(r) for r in load_feed(path)]
```

The `record.get("id")` alias for customer_id is dropped too. The schema lists only `customer_id` and nothing in the sample or docs uses `id`. If you know a live source still sends `id`, keep that one line and document it in the schema.

**Patch 2: `transform/enrich.py`** (replace whole file)

```python
"""Enrich customer rows for reporting."""


def enrich(rows):
    out = []
    for row in rows:
        enriched = dict(row)
        enriched["plan"] = row["plan_tier"]
        enriched["arr_cents"] = row["mrr_cents"] * 12
        out.append(enriched)
    return out
```

The `.lower()` is gone because ingest now guarantees a canonical lowercase tier. The `or 0` is gone because a real zero for the free plan must survive, and a missing value is now impossible past ingest.

**Patch 3: `reports/revenue.py`** (replace the loop body)

```python
    for row in enrich(ingest(path)):
        plan = row["plan"]
        totals[plan] += row["arr_cents"]
        counts[plan] += 1
```

**Expected output** after the patch, run against the sample feed:

| plan | customers | arr_cents |
|---|---|---|
| business | 1 | 598800 |
| team | 1 | 118800 |
| enterprise | 1 | 3000000 |
| free | 1 | 0 |

I could not run this myself here, so please verify with `python -m reports.revenue` from the repo root. Also worth a quick test that a record with no `subscription` key raises FeedRecordError naming the customer id and the expected path, since that is the behavior that would have caught this two weeks ago.

**Follow-ups outside this patch.** The docs table in `docs/schema.md` still describes the internal row, which is fine, but the feed changelog should note that the adapter now hard-fails on old-format records. And the dashboard job should surface ingest exceptions as an alert rather than swallowing them, otherwise the next break will show up as a stale chart instead of an error.

about this skill

SKILL.md

---
name: remove-defensive-fallbacks
description: Find and remove fallbacks that hide missing or malformed required data in pipelines, metrics, and dashboards. Trace each affected field to its canonical source while preserving documented optional defaults and supported input formats.
---

# Remove Defensive Fallbacks

Refactor code so every required value has one explicit path through the data model. Preserve defaults that are part of the data contract.

## Before editing

Trace the affected value from ingestion through every transformation to its final output. Determine:

- Where the field is canonically stored and where derived values are computed.
- Whether the field is required, optional, or nullable according to its schema, producer, and consumers.
- What missing, `null`, zero, `false`, and empty values mean for this field.
- Whether a transformation drops, renames, duplicates, or shadows fields.
- Whether a fallback masks a violation or implements a documented default.

An existing fallback does not prove that a field is optional. If the contract is unclear, resolve that question before changing missing-value behavior. Do not patch only the visible `null` or incorrect score.

## Rules

1. Identify the canonical source for each affected required field.
2. Read it directly from that location. Fix the producer or transformation when it loses required data.
3. Remove searches across old and new paths when they hide a broken current schema.
4. Inspect fallback expressions such as:
   - `a ?? b ?? c`
   - `a || b`
   - `.get(key, default)`
   - `getattr(obj, field, default)`

   Remove them when they substitute for invalid required data. The syntax alone is not a reason to delete them.
5. Preserve documented defaults for optional fields. Preserve valid `0`, `false`, and empty values; avoid truthiness checks when those values are allowed. Treat missing and explicit `null` separately when the contract does.
6. Keep adapters for supported input versions. Normalize each supported version at the boundary into one internal schema instead of scattering compatibility branches through calculations.
7. If required data is absent or malformed, fail at the earliest boundary that can validate it, with a clear error containing:
   - the missing or malformed field
   - the expected canonical path
   - the relevant record or request identifier
8. Keep data flow linear and explicit.
9. Remove obsolete compatibility logic only after checking that its input format is no longer supported.
10. Verify the fix across ingestion, transformation, computation, and output. Check missing and malformed required data, valid falsy values, and optional defaults.

## Distinguish the cases

If `total` is a required finite number, this hides a broken record:

```js
const total = row.total ?? row.legacyTotal ?? 0;
```

Validate the canonical field and preserve a real zero:

```js
if (typeof row.total !== "number" || !Number.isFinite(row.total)) {
  throw new Error(`Expected a finite number at row.total for record ${row.id}`);
}
const total = row.total;
```

If `showLegend` is optional and its contract defines missing or `null` as `true`, this default is valid and preserves an explicit `false`:

```js
const showLegend = options.showLegend ?? true;
```

## Desired result

Correct inputs produce correct results through one valid data path. Invalid required data produces an actionable error. Optional data keeps its documented behavior.

package contents

post id: cd5f8n04i7gxds8d