refactor(delphi): Remove legacy poller/system architecture and dead code #2343
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove dead code that was replaced by scripts/job_poller.py + DynamoDB job queue: - polismath/__main__.py (legacy entry point) - polismath/system.py (legacy System/SystemManager) - polismath/poller.py (legacy Poller/PollerManager) - polismath/components/server.py (legacy Server/ServerManager) Updated __init__.py files to remove imports of deleted modules. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove polismath/utils/general.py (unused utility functions - 263 lines)
- Remove unused squareform import from corr.py
- Archive outdated docs to docs/archive/:
- conversion_plan.md (conversion complete)
- NEXT_STEPS.md (outdated)
- project_structure.md (proposed, not actual)
- Update RUNNING_THE_SYSTEM.md: remove SystemManager references, add current CLI usage - Archive architecture_overview.md (describes Clojure, not Python) - Archive summary.md (references deleted poller/server/system components) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Detailed documentation of the dead code identification and removal process, including actual vulture output (98 findings) and recommendations for future cleanup. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit corrects a critical false positive from the dead code cleanup:
1. RESTORED: polismath/utils/general.py (263 lines)
- Was incorrectly marked as dead code
- Actually has 4 active imports in production code:
* postgres.py: postgres_vote_to_delphi()
* run_math_pipeline.py: postgres_vote_to_delphi()
* repness.py: AGREE, DISAGREE constants
* clusters.py: (import existed but unused - see below)
2. REMOVED: Unused imports (5 total)
- clusters.py: weighted_mean, weighted_means
- postgres.py: JSON, QueuePool
- job_poller.py: JSON, QueuePool
3. ADDED: docs/DEAD_CODE_CLEANUP_REPORT_CORRECTED.md
- Documents the false positive
- Provides root cause analysis
- Recommends process improvements
Tests verified: 211 passed, 7 skipped, 2 xfailed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Merge DEAD_CODE_CLEANUP_REPORT.md and DEAD_CODE_CLEANUP_REPORT_CORRECTED.md into a single comprehensive report documenting the entire cleanup series. Changes: - Combined both reports into unified DEAD_CODE_CLEANUP_REPORT.md - Documents false positive (general.py) and correction - Clarifies entry point is run_delphi (not delphi CLI) - Shows net impact: -730 lines from edge branch - All documentation files kept in docs/ (not archived) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add complete output from Vulture dead code analyzer (147 findings): - 90%+ confidence: ~15 unused imports - 60%+ confidence: ~132 potentially unused functions/classes/methods This provides the raw data referenced in DEAD_CODE_CLEANUP_REPORT.md for future dead code cleanup efforts. Command used: vulture . --min-confidence 60 --exclude ".git,__pycache__,*.pyc,.venv,tests" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove all 90%+ confidence unused imports identified by Vulture: Files modified (10): - polismath/benchmarks/bench_repness.py (3 imports) - polismath/components/config.py (Set) - polismath/conversation/conversation.py (Set) - polismath/conversation/manager.py (Set) - polismath/database/postgres.py (Set) - polismath/utils/general.py (itertools, Set) - scripts/delphi_cli.py (Text, rprint) - umap_narrative/801_narrative_report_batch.py (csv, io) - umap_narrative/polismath_commentgraph/core/clustering.py (hdbscan, delayed, Parallel) - umap_narrative/polismath_commentgraph/utils/storage.py (QueuePool) All imports verified as unused by: 1. Vulture static analysis (90% confidence) 2. Manual grep verification 3. Test suite (45 tests passed) This addresses the high-priority cleanup items from vulture_analysis_output.txt. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here's some clean up of dead code: using Claude Code Opus 4.5 to clean up behind Claude Code Sonnet 3.7 from last year ;-)
There are probably still code paths that are unused, and I have not removed the markdown files created during the python port last year -- they'll still serve until we're fully sure of the codebase, I think.
Summary
Systematic cleanup of dead/unused code from the delphi/ folder (ported from Clojure ~1 year ago). Removed 730 net lines of dead code across 6 commits, with all tests passing.
Key Changes
Removed (1,162 lines)
polismath/__main__.py(150 lines) - Legacy entry pointpolismath/system.py(208 lines) - Legacy system managerpolismath/poller.py(507 lines) - PostgreSQL continuous pollingpolismath/components/server.py(297 lines) - Legacy FastAPI serverRemoved (6 unused imports)
squareformfrompca_kmeans_rep/corr.pyweighted_mean,weighted_meansfrompca_kmeans_rep/clusters.pyJSON,QueuePoolfromdatabase/postgres.pyandscripts/job_poller.pyDocumentation
RUNNING_THE_SYSTEM.md(removed SystemManager references)Initially deleted
polismath/utils/general.py(263 lines) due to faulty grep verification, but this file had 4 active imports in production code. It has been restored and tests now pass.Root cause: Grep was likely run from wrong directory. Network issues prevented running tests before initial commit.
Verification
Tests Pass ✅
Legacy System Gone
Entry Points Preserved
Current Delphi entry point (
run_delphi:main) and all other entry points remain functional:run-delphi(Docker/Makefile main entry point)run-math-pipelinerun-umap-pipelinecalculate-extremitycalculate-prioritiesreset-conversationcreate-datamapplotNote: The
delphiCLI script is old and not fully working; Docker usesrun_delphi.Impact
Future Work
Vulture analysis identified 92 additional items (mostly 60% confidence) for future cleanup:
Documentation
Full details in
delphi/docs/DEAD_CODE_CLEANUP_REPORT.mdincluding:Commits:
cebd9d2b6through645894d38Base:
edgeTests: ✅ All passing