Skip to content

fix(settle_epoch): exit non-zero when settlement fails - #8501

Open
Scottcjn wants to merge 1 commit into
mainfrom
fable/settle-epoch-exit-code-2026-09-21
Open

Scottcjn wants to merge 1 commit into
mainfrom
fable/settle-epoch-exit-code-2026-09-21

Conversation

@Scottcjn

Copy link
Copy Markdown
Owner

Problem

node/settle_epoch.py is run unattended; the scheduler's only signal is the exit code. trigger_settlement() catches every exception and returns None, non-200 returns the response text, and __main__ printed the result and exited 0 in every case. An unreachable node, a malformed /epoch, or a 500 from /rewards/settle all looked like success.

Fix (1 source file + 1 test)

  • main() -> int: exit 0 only when /rewards/settle returned 200 with a JSON object that does not say ok: false; otherwise exit 1 and print SETTLEMENT FAILED to stderr.
  • All existing logging kept (same stdout lines). trigger_settlement()'s return contract is untouched, so tests/test_settle_epoch.py still passes unchanged.
  • NODE_URL can be overridden with RC_NODE_URL (default unchanged) — used by the end-to-end test.

Regression test — tests/test_settle_epoch_exit_code.py

Unit tests for each failure shape plus an end-to-end subprocess run of the real script against a closed port.

On main (fix stashed):

FAILED test_main_exits_zero_on_successful_settlement
FAILED test_main_exits_nonzero_when_node_unreachable
FAILED test_main_exits_nonzero_on_http_error_from_settle
FAILED test_main_exits_nonzero_when_node_reports_ok_false
FAILED test_main_exits_nonzero_on_malformed_epoch_response
FAILED test_script_process_exit_code_nonzero_when_node_down
6 failed, 6 passed in 0.30s      (the 6 passes are the pre-existing tests)

With fix: 12 passed in 0.29s

Existing suite (local, Python 3.13): root pytest tests/ (CI ignores applied) — main: 4006 passed / 47 skipped / 2 xfailed; this branch: 4012 passed, same skips, 0 new failures. node/tests — main: 107–108 failed / 17 errors pre-existing (flaky between identical runs on main: one live-endpoint network test and test_sophia_elya_service.py::test_finalize_epoch_excludes_poisoned_negative_weight_rows); this branch: 108 failed / 1746 passed / 17 errors, diff vs. main by test name = only those two flaky ones. No test that passes on main fails here.

Risk before deploy

  • If a cron/systemd timer wraps this script with something that alerts on non-zero exit, it will start alerting — that is the point, but expect noise on the first run if settlement is currently failing silently.
  • /rewards/settle is admin-rate-limited; this script sends no admin key (pre-existing, not changed here). If it is currently 401/429-ing, this PR makes that visible.

AI-assisted disclosure: this PR was written by Claude (Fable 5.1) working as an engineering agent for Scott / Elyan Labs, in a fresh clone, with the tests run locally as shown above. Please review as you would any contributor PR.

🤖 Generated with Claude Code

trigger_settlement() swallowed every exception and __main__ exited 0 on
None / non-200 text / ok:false, so an unreachable node or a 500 from
/rewards/settle looked like success to cron/systemd.

Add main() -> exit code (0 only for a 200 JSON object without ok:false),
keep all existing logging, add RC_NODE_URL override, and sys.exit(main()).
trigger_settlement()'s return contract is unchanged (existing tests pass).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions github-actions Bot added BCOS-L1 Beacon Certified Open Source tier BCOS-L1 (required for non-doc PRs) BCOS-L2 Beacon Certified Open Source tier BCOS-L2 (required for non-doc PRs) node Node server related tests Test suite changes size/M PR: 51-200 lines labels Sep 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ BCOS v2 Scan Results

Metric Value
Trust Score 36/100
Certificate ID BCOS-e4bb17ea
Tier L1 (not met)

BCOS Badge

What does this mean?

The BCOS (Beacon Certified Open Source) engine scans for:

  • SPDX license header compliance
  • Known CVE vulnerabilities (OSV database)
  • Static analysis findings (Semgrep)
  • SBOM completeness
  • Dependency freshness
  • Test infrastructure evidence
  • Review attestation tier

Full report | What is BCOS?


BCOS v2 Engine - Free & Open Source (MIT) - Elyan Labs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BCOS-L1 Beacon Certified Open Source tier BCOS-L1 (required for non-doc PRs) BCOS-L2 Beacon Certified Open Source tier BCOS-L2 (required for non-doc PRs) node Node server related size/M PR: 51-200 lines tests Test suite changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants