fix: invalidate index fragment bitmaps after data replacement and stale merge

[wjones127]

This fixes a bug where queries or merge insert could result in the error this fragment does not exist in the dataset.

This would happen because the index referenced row addresses that were no longer in the dataset.

Here's one way this would happen:

• Create a BTREE index covering 2 fragments
• Do a merge insert that rewrites the index column in place. This removes the updated data from the index's fragment bitmap, so data is now hidden.
• Do compaction. This will move the rows to new fragments.
• Optimize the BTREE index: this would build the btree index with the old data, not filtering out the invalidated rows. After doing this the invalidated rows, previously masked out, are part of the index. But since compaction happened, those old invalidated rows moved.

This PR makes sure:

1. When incrementally updating indexes, we only read row ids for data in the fragment bitmap.
2. Make sure we remove fragments from the index fragment bitmap if we do a DataReplacement on those fields.

This rules are described in the spec at https://lance.org/format/table/index/#loading-an-index

Fixes #5321

[github-actions]

Code Review

This PR fixes index fragment bitmap corruption after merge_insert with reordered columns followed by compact_files and optimize_indices. The fix addresses two root causes correctly.

P0: Potential Issue in Scalar Index Rebuild Logic

In rust/lance/src/index/append.rs, lines 140-162, the rebuild-from-scratch path when effective_old_frags.is_empty() passes true for the include_row_id parameter and Some(new_data_stream) for fragments:

super::scalar::build_scalar_index(
    dataset.as_ref(),
    column.name.as_str(),
    &new_uuid.to_string(),
    &params,
    true,  // include_row_id
    None,  // fragments (None means all)
    Some(new_data_stream),  // additional_data_stream
)
.await?

However, the new_data_stream was already prepared from unindexed fragments only (loaded at line 134-142). When rebuilding from scratch with an empty effective bitmap, the intent is to index all current fragments. Passing None for fragments is correct (indexes all), but new_data_stream only contains unindexed fragment data.

Recommendation: Verify that build_scalar_index with fragments=None and additional_data_stream=Some(new_data_stream) produces correct results, or consider passing None for both parameters when rebuilding from scratch.

P1: Replaced Fields Extraction

In rust/lance/src/dataset/transaction.rs, the replaced_fields extraction at lines 2062-2071 only looks at the first data file:

let replaced_fields: Vec<u32> = new_datafiles
    .first()
    .map(|f| {
        f.fields.iter().filter(...).collect()
    })
    .unwrap_or_default();

This is safe given the earlier validation (lines 2037-2058) that ensures all new data files have the same fields. However, adding a comment here would make this dependency explicit.

Tests

The test coverage is thorough:

• test_merge_insert_with_reordered_columns_and_index directly reproduces the reported issue
• test_data_replacement_invalidates_index_bitmap validates the DataReplacement fix

Overall the fix logic is sound and addresses the reported issue correctly.

[codecov]

Codecov Report

❌ Patch coverage is 91.13924% with 7 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance/src/index/append.rs	82.14%	0 Missing and 5 partials ⚠️
rust/lance-index/src/scalar/btree.rs	92.59%	0 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

[westonpace]

Thanks for figuring this out. It makes me wonder if the update path is even worth the complexity it brings at this point. Btree is the only one that really makes use of it.

[westonpace]

So this only matters for btree because it is the only index that scans the old portion of the index instead of rescanning the whole column?

[wjones127]

Actually, bitmap does this too. I need to fix that.

[wjones127]

Fixed bitmap. Other indices should be done as a follow up, but I'm not aware of a code path where they cause a bug yet. Many of them like ZoneMap and BloomFilter given inexact results.

[westonpace]

Minor nit: I've been trying to slowly switch over to using string constants like BTREE_IDS_COLUMN and avoiding integral indexing just because it seems less prone to error in the future.

[wjones127]

I can try that.

[wjones127]

It makes me wonder if the update path is even worth the complexity it brings at this point. Btree is the only one that really makes use of it.

That's a fair point. There's probably not a huge advantage to scanning the index vs scanning the fragments.