add columns to polymarket models #9107

hildobby · 2025-12-09T18:48:04Z

added columns to existing tables (needed for much more efficient queries) and created 3 tables:

one for protocol tvl
one for user balance changes
one for currently open positions (view)

cursor · 2025-12-11T17:59:10Z

PR Summary

Introduces new analytics surfaces and enriches existing Polymarket models for better querying and pricing.

Adds new incremental models: polymarket_polygon_protocol_tvl (hourly TVL/net flow from USDC.e transfers) and polymarket_polygon_users_balance_changes (inflow/outflow/internal per wallet); adds polymarket_polygon_open_positions view (latest positions with latest_price/open_interest)
Enhances polymarket_polygon_market_details with polymarket_link_slug, parsed start/end timestamps, and computed final_price
Extends trades: polymarket_polygon_market_trades_raw and ..._market_trades now include maker_asset_id/taker_asset_id and expose polymarket_link_slug
Upgrades polymarket_polygon_positions to incremental, adds daily pricing join and price/usd_value, and filters late/partial days
Refactors polymarket_polygon_market_prices_daily forward-fill to use utils_days and date bounds
Updates _schema.yml to document new columns/models and add tests

^{Written by Cursor Bugbot for commit d775b77. Configure here.}

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

...ily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_market_prices_daily.sql

...ects/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_protocol_tvl.sql

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_open_positions.sql

hildobby · 2025-12-16T18:36:28Z

@cursor review

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

...ects/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_protocol_tvl.sql

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_open_positions.sql

hildobby · 2025-12-16T19:10:29Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no bugs!

Comment @cursor review or bugbot run to trigger another review on this PR

…values and optimise open positions

…spellbook into polymarket_table_fixes

hildobby · 2025-12-16T19:56:41Z

@cursor review

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_open_positions.sql

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_market_details.sql

hildobby · 2025-12-16T21:01:51Z

https://github.com/cursor review

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

cursor · 2025-12-16T21:12:41Z

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_market_details.sql

+SELECT *,
+CASE WHEN outcome = '50/50' THEN 0.5
+  WHEN LOWER(token_outcome) = outcome THEN 1
+  ELSE 0 END AS price_modifier


Bug: Unresolved markets incorrectly get zero price modifier

The price_modifier calculation returns 0 for markets with outcome = 'unresolved' because neither the '50/50' check nor LOWER(token_outcome) = outcome matches when outcome is 'unresolved'. This differs from the existing behavior in polymarket_polygon_market_prices_daily.sql which preserves the original market price (ELSE ff.price) for unresolved markets. When used in polymarket_polygon_positions.sql, markets past their end time but not yet resolved will incorrectly show positions as worthless (price = 0) instead of using the last traded price.

Additional Locations (1)

dbt_subprojects/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_positions.sql#L21-L27

hildobby · 2026-01-09T11:17:53Z

@cursor review

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

cursor · 2026-01-09T11:24:34Z

...y_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_users_balance_changes.sql

+  , unique_key
+  FROM relevant_transfers
+  WHERE from_polymarket_wallet IS NOT NULL AND to_polymarket_wallet IS NOT NULL
+  AND (from_first_funded_block IS NULL OR from_first_funded_block <= block_number)


Internal transfers create duplicate records for sender wallet

Medium Severity

When a transfer occurs between two polymarket wallets (A→B), the sender wallet A gets recorded twice: once with direction='outflow' (lines 96-98) and again with direction='internal' (lines 117-119). Both records use from_polymarket_wallet AS polymarket_wallet and capture the same amount. The outflow WHERE clause (from_polymarket_wallet IS NOT NULL) doesn't exclude internal transfers, so there's an overlap with the internal clause (from_polymarket_wallet IS NOT NULL AND to_polymarket_wallet IS NOT NULL). This could lead to double-counting in downstream aggregations that sum balance changes by wallet.

cursor · 2026-01-09T11:24:34Z

...y_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_users_balance_changes.sql

+  LEFT JOIN {{ ref('polymarket_polygon_users') }} to_user ON t."to" = to_user.polymarket_wallet
+  LEFT JOIN {{ ref('polymarket_polygon_users') }} from_user ON t."from" = from_user.polymarket_wallet
+  INNER JOIN {{ source('addresses_events_polygon', 'first_funded_by')}} to_ffb ON t."to" = to_ffb.address
+  INNER JOIN {{ source('addresses_events_polygon', 'first_funded_by')}} from_ffb ON t."from" = from_ffb.address


INNER JOINs make NULL checks unreachable dead code

Medium Severity

The INNER JOIN on first_funded_by (lines 34-35) filters out any transfers where either the sender or receiver address doesn't exist in that table. However, the subsequent WHERE clauses include to_first_funded_block IS NULL OR ... and from_first_funded_block IS NULL OR ... checks (lines 77, 98, 119), which can never evaluate to true because the INNER JOIN guarantees these values are non-NULL. This suggests LEFT JOINs were intended, and the current INNER JOINs may silently exclude legitimate transfers involving addresses not yet in first_funded_by.

Additional Locations (1)

dbt_subprojects/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_users_balance_changes.sql#L76-L77

cursor · 2026-01-09T11:24:34Z

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_open_positions.sql

+    FROM {{ ref('polymarket_polygon_positions_raw') }} p
+    INNER JOIN latest_day ld ON p.day = ld.day
+    INNER JOIN {{ ref('polymarket_polygon_market_details') }} mm ON p.token_id = mm.token_id
+    )


Open positions view includes zero-balance closed positions

Medium Severity

The open_positions CTE joins with positions_raw without filtering out positions where balance = 0. Since positions_raw includes all balances (including zero balances from fully exited positions), the view will return closed positions as "open." The schema describes this as "Polymarket: open positions in markets," but users who have completely sold their positions will still appear in the results. A filter like WHERE p.balance > 0 is needed in the open_positions CTE to match the semantic meaning of the table.

add columns to polymarket models

b428969

github-actions bot marked this pull request as draft December 9, 2025 18:48

github-actions bot added WIP work in progress dbt: daily covers the Daily dbt subproject labels Dec 9, 2025

hildobby marked this pull request as ready for review December 9, 2025 18:50

github-actions bot added ready-for-review this PR development is complete, please review and removed WIP work in progress labels Dec 9, 2025

hildobby added 20 commits December 9, 2025 20:04

fix

836fb35

add polymarket link slug

6dbe439

fixes

6621cfa

create open_positions spell

60c35bb

create protocol_tvl spell

8c74bca

create users balance changes spell

f25cda4

fix

0f9c1ab

fix

fe719c6

Merge branch 'main' into polymarket_table_fixes

151497e

fix

9113b9f

fix

01ef68c

fix

f898a34

fix

e9e44e7

fi

11daa3f

add 1 month filter to user balance changes for CI tests

efa32af

fix

7b143e8

fix

85d8be5

fixes

259287a

remove 1 month filter used for tests

351bda9

more fixes

b025811

cursor bot reviewed Dec 11, 2025

View reviewed changes

fixes suggested by cursor - good bot

ea14ab4

hildobby added 10 commits December 12, 2025 13:15

fixes

3be098b

fix

c2ce5a4

fixes

8295e84

Merge branch 'main' into polymarket_table_fixes

c269a9d

fix users spell and add 1 month filter to user balances for CI tests

0c92d76

less restrictive join for open positiosn

b40416f

fix protocol tvl

3fe69d9

fix

c6be77f

Merge branch 'main' into polymarket_table_fixes

1f9bda4

ci tests passed, remove 1 month filter

23f755b

cursor bot reviewed Dec 16, 2025

View reviewed changes

...ects/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_protocol_tvl.sql Outdated Show resolved Hide resolved

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_open_positions.sql Outdated Show resolved Hide resolved

hildobby added 2 commits December 16, 2025 20:09

account for 50/50s and filter properly

35f1eeb

Merge branch 'main' into polymarket_table_fixes

2783ec6

cursor bot reviewed Dec 16, 2025

View reviewed changes

hildobby added 2 commits December 16, 2025 20:53

add price_modifier to market_details, optimise positions and add usd …

6b32759

…values and optimise open positions

Merge branch 'polymarket_table_fixes' of https://github.com/hildobby/…

4185473

…spellbook into polymarket_table_fixes

cursor bot reviewed Dec 16, 2025

View reviewed changes

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_open_positions.sql Show resolved Hide resolved

...ts/daily_spellbook/models/_projects/polymarket/polygon/polymarket_polygon_market_details.sql Show resolved Hide resolved

optimise balance_changes

9592bca

cursor bot reviewed Dec 16, 2025

View reviewed changes

hildobby added 5 commits December 16, 2025 23:11

optimise

3942db8

Merge branch 'main' into polymarket_table_fixes

b2127f5

fix

e69eb26

fix

c45d3aa

Merge branch 'main' into polymarket_table_fixes

d775b77

cursor bot reviewed Jan 9, 2026

View reviewed changes

add columns to polymarket models #9107

Are you sure you want to change the base?

add columns to polymarket models #9107

Conversation

hildobby commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor bot commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hildobby commented Dec 16, 2025

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hildobby commented Dec 16, 2025

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

✅ Bugbot reviewed your changes and found no bugs!

Uh oh!

hildobby commented Dec 16, 2025

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hildobby commented Dec 16, 2025

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Dec 16, 2025

Choose a reason for hiding this comment

Bug: Unresolved markets incorrectly get zero price modifier

Uh oh!

hildobby commented Jan 9, 2026

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Jan 9, 2026

Choose a reason for hiding this comment

Internal transfers create duplicate records for sender wallet

Uh oh!

cursor bot Jan 9, 2026

Choose a reason for hiding this comment

INNER JOINs make NULL checks unreachable dead code

Uh oh!

cursor bot Jan 9, 2026

Choose a reason for hiding this comment

Open positions view includes zero-balance closed positions

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

hildobby commented Dec 9, 2025 •

edited

Loading

cursor bot commented Dec 11, 2025 •

edited

Loading