Add support for Upserts

[el-yawd]

Description

Related Issues

Closes #

Checklist when merging to main

• No compiler warnings (if applicable)
• Code is formatted with rustfmt
• No useless or dead code (if applicable)
• Code is easy to understand
• Doc comments are used for all functions, enums, structs, and fields (where appropriate)
• All tests pass
• Performance has not regressed (assuming change was not to fix a bug)
• Version number has been updated in helix-cli/Cargo.toml and helixdb/Cargo.toml

Additional Notes

Greptile Overview

Greptile Summary

This PR adds comprehensive support for upsert operations (UpsertN, UpsertE, UpsertV) to HelixDB, enabling update-or-insert semantics for nodes, edges, and vectors. The implementation follows the existing patterns for Add operations and includes changes across the grammar, parser, analyzer, code generator, and runtime engine.

Key Changes

• Grammar: Added UpsertN, UpsertE, and UpsertV syntax matching AddN, AddE, and AddV patterns
• Core Logic: Implemented upsert operations that check if an entity exists in the iterator, merging properties if found or creating new entities if not
• Property Handling: Upsert merges new properties with existing ones, preserving unchanged properties and updating modified ones
• Indexing: Properly maintains secondary indices and BM25 full-text search indices during both update and insert paths
• Type Safety: Full type validation and error checking in the analyzer matching existing Add operations

Observations

• The upsert mechanism relies on checking self.inner.next() to determine if an entity exists, but there's no clear documentation or examples showing how users should pass existing entities to trigger the update path vs the insert path
• Integration tests verify parser compilation but don't test actual runtime upsert behavior (whether updates or inserts actually work correctly)
• The PR checklist shows "Version number has been updated" is unchecked, but Cargo.toml files show version was bumped to 1.1.4
• Secondary index update logic is split between handling changed properties (lines 108-147) and new properties (lines 149-172), which could be consolidated for clarity

Recommendations

1. Add usage documentation and examples demonstrating both update and insert scenarios
2. Add runtime tests that execute actual upsert operations and verify property merging, indexing, and database state
3. Consider clarifying the expected usage pattern (e.g., does upsert require a preceding query step like existing <- N<Person>(id) followed by updated <- existing::UpsertN<Person>({...})?)

Important Files Changed

File Analysis

Filename	Score	Overview
helix-db/src/grammar.pest	5/5	Added grammar rules for UpsertN, UpsertE, and UpsertV operations, mirroring the structure of AddN, AddE, and AddV. Updated step and creation_stmt rules to include upsert operations.
helix-db/src/helix_engine/traversal_core/ops/util/upsert.rs	3/5	Implemented core upsert logic for nodes, edges, and vectors. The implementation checks if an entity exists in the iterator, updates it if found, or creates a new one if not. Handles property merging, secondary indices, and BM25 updates. However, lacks clear mechanism for how existing entities are identified and passed to upsert operations.
helix-db/src/helixc/analyzer/methods/infer_expr_type.rs	4/5	Added type inference logic for UpsertNode, UpsertEdge, and UpsertVector expressions, following the same pattern as Add operations with property validation and type checking.
helix-db/src/helixc/parser/creation_step_parse_methods.rs	5/5	Implemented parser methods for UpsertN, UpsertE, and UpsertV, following the same structure as Add operations. Includes comprehensive unit tests.
helix-db/tests/upsert_integration_test.rs	4/5	Comprehensive integration tests covering basic upsert operations, property handling, and complex scenarios. Tests verify parser compilation but don't test actual runtime upsert behavior (update-or-insert logic).

Sequence Diagram

sequenceDiagram
    participant User
    participant Parser
    participant Analyzer
    participant Generator
    participant Runtime
    participant Storage

    User->>Parser: UpsertN<Person>({name: "Alice"})
    Parser->>Parser: parse_upsert_node()
    Parser->>Analyzer: UpsertNode AST
    
    Analyzer->>Analyzer: Validate node type exists
    Analyzer->>Analyzer: Validate properties & types
    Analyzer->>Generator: Generate UpsertN code
    
    Generator->>Generator: Create upsert_n() call
    Generator->>Runtime: Execute traversal with upsert_n()
    
    Runtime->>Runtime: Check if entity in iterator
    
    alt Entity exists (iterator has item)
        Runtime->>Runtime: Merge properties (old + new)
        Runtime->>Storage: Update secondary indices
        Runtime->>Storage: Update BM25 index
        Runtime->>Storage: Update node in nodes_db
        Runtime-->>User: Return updated node
    else Entity does not exist (iterator empty)
        Runtime->>Runtime: Create new node with UUID
        Runtime->>Storage: Insert secondary indices
        Runtime->>Storage: Insert BM25 index
        Runtime->>Storage: Insert node in nodes_db
        Runtime-->>User: Return new node
    end

Loading

[greptile-apps]

Additional Comments (4)

1. helix-db/src/helix_engine/traversal_core/ops/util/upsert.rs, line 309 (link)

   style: TODO comment indicates error handling needs improvement - operations continue even after errors occur, wasting resources. Consider early returns or using ? operator to propagate errors immediately.

2. helix-db/src/helix_engine/traversal_core/ops/util/upsert.rs, line 62-73 (link)

   style: The upsert logic checks self.inner.next() to determine if an entity exists, but there's no clear mechanism or documentation explaining how users should pass an existing entity to the upsert operation. The tests only verify parser compilation, not actual update behavior. Consider adding documentation/examples showing the update path and runtime tests that verify both insert and update cases.

3. helix-db/tests/upsert_integration_test.rs, line 232-254 (link)

   style: Test only verifies parsing, not runtime behavior. Add runtime tests that: (1) create an entity then upsert it to verify update, (2) upsert a non-existent entity to verify insert, (3) test property merging, (4) verify secondary indices and BM25 updates.

4. helix-db/src/helix_engine/traversal_core/ops/util/upsert.rs, line 108-147 (link)

   style: When updating secondary indices for changed properties, if old.get(k) returns None, you continue without creating the new index. New properties are handled separately in the diff logic (lines 149-172). Consider consolidating this logic for clarity.

_{14 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

[xav-db]

@el-yawd should add some helixql tests in hql-tests and unit tests