Conversation
…ncies in client code (#668) * Initial plan * Fix: Change ModelResult/TypeDefResult imports to type-only imports in ts-schema-generator Co-authored-by: ymc9 <104139426+ymc9@users.noreply.github.com> * Regenerate test schemas with type-only imports Co-authored-by: ymc9 <104139426+ymc9@users.noreply.github.com> * Fix: Update bun runtime models.ts to use type-only imports Co-authored-by: Yiming Cao <ymc9@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: ymc9 <104139426+ymc9@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Yiming Cao <ymc9@users.noreply.github.com>
* feat: initial db pull implementation * fix: generate imports and attributes for zmodel-code-generator * fix: add option to not exclude imports in loadDocument * fix: continue work on db pull * fix: missing import * fix: rewrite model generation generate model from ground up and diff later * feat: add ast factory * fix: ast factory import order * fix: some runtime bugs * fix: lint fix * fix: update zmodel code generator - include imports in output - fix indentaions - include comments in output * feat: add exclude schemas option * feat: implement initial diff update * fix: update format in zmodel code generator * fix: typo * feat: progress on database introspection and syncing * fix: make ignore behave it does in prisma with no index models * fix: lint fix * feat: make all format options configurable * fix: lint fix * feat: Handle the database type mapping * fix: catch up with feature updates - improve code styling - enable schema support for db pull * fix: add sqlite e2e test and fix some bugs * fix: lint fix * fix: formatting for e2e test schemas * test: run db pull e2e test also for postgres * fix: postgres instorspection schema filter * test: update cli tests * feat(cli): Improves database introspection and syncing Enhances the `db pull` command with a spinner for better UX. Adds color-coded logging to highlight important steps. Provides more detailed output on schema changes, including deleted models, enums, added fields, and deleted attributes. Also includes minor improvements to enum mapping and constraint handling. * fix(cli): fixes field casing and sort issues * chore(cli): remove temporary test script Deletes an unused script used for experimenting with URI path resolution. Cleans up the codebase by removing development-only artifacts. * chore: update pnpm-lock.yaml * feat(cli): add MySQL support for schema introspection Introduces a MySQL-specific introspection provider to support pulling existing database schemas into ZenStack. The implementation includes logic for mapping MySQL data types to ZenStack types, handling auto-incrementing fields, and parsing MySQL-specific enum definitions. It utilizes dynamic imports for database drivers to minimize the CLI footprint for users not targeting MySQL. * fix(cli): improve field matching logic during db pull * feat(cli): enhance SQLite introspection with autoincrement support * fix(cli): refine attribute generation in db pull * test(cli): update db pull tests for SQLite specific behavior * refactor(language): export ZModelServices type * fix(cli): improve sqlite introspection for autoincrement and fk names * feat(cli): enhance field matching logic during pull by using relation fields * refactor(cli): refine relation name generation and table syncing * test(cli): update pull tests to reflect improved schema generation * test(cli): add MySQL support to test utility helpers Extends the testing infrastructure to support MySQL databases. Adds MySQL configuration defaults and environment variable overrides. Updates the prelude generation logic to handle MySQL connection strings and provider types, enabling broader database integration testing across the CLI. * fix(cli): omit default constraint names in table sync Avoids explicitly declaring unique constraint names when they match the default database naming convention. This results in cleaner generated schema code by removing redundant mapping arguments. * fix: correctly handle default values for 'text' type in PostgreSQL * fix: sort table indexes to ensure stable schema generation * refactor: dynamically determine supported db providers in CLI * test: fix typo in pull test description * chore(cli): remove debug artifacts and silence test logs Removes hardcoded file system path debugging and unnecessary console logging from the introspector and test suites. Silences CLI command output during tests to provide a cleaner test execution environment. * fix(cli): ensure MySQL column and index ordering Wraps JSON_ARRAYAGG calls in subqueries with explicit ORDER BY clauses to maintain correct metadata ordering. This addresses a limitation in MySQL versions prior to 8.0.21, where ORDER BY is not supported directly within the JSON_ARRAYAGG function, ensuring consistent introspection results across different database versions. * fix(cli): preserve column order during MySQL pull Ensures database columns are sorted by their ordinal position during the introspection process. This maintains the original schema structure and provides a consistent output that matches the physical database layout. * refactor(cli): remove schema fields from MySQL queries Eliminates redundant schema and database name fields from the MySQL introspection query. Since MySQL does not support multi-schema architectures internal to a single connection in this context, removing these fields simplifies the data structure and avoids unnecessary metadata overhead. * fix(cli): improve MySQL introspection and index mapping Refines the database pull process to better handle MySQL-specific patterns. Improves unique constraint detection to prevent redundant mapping attributes when default naming conventions are used. Updates the MySQL introspection logic to correctly identify boolean types, handle timestamp precision in default values, and normalize numeric defaults. Also ensures auto-incrementing columns and primary key indexes are correctly mapped to prevent schema duplication. * test(cli): pass provider to default prelude in tests Ensures that the default schema prelude correctly reflects the database provider specified in test options. This prevents inconsistencies when generating test projects with non-default providers. * fix(cli): improve MySQL introspection for types and defaults Disables NativeEnum support for MySQL to prevent loss of schema-level enums since MySQL enums are column-specific. Refines boolean and numeric type mapping to better handle synthetic boolean types and preserve decimal precision in default values. Updates default value parsing logic to correctly identify unquoted strings and avoid misinterpreting numeric literals as booleans. * fix(cli): improve MySQL default value introspection Refines how default values are handled during database introspection for MySQL by considering the specific field type. This ensures that boolean variants and numeric literals for Float and Decimal types are correctly formatted and preserved. Also clarifies unsupported features in the SQLite provider to improve codebase maintainability. * test(cli): expand and reorganize db pull tests Enhances the test suite for the database pull command by adding comprehensive coverage for common schema features and PostgreSQL-specific functionality. Includes new test cases for: - Restoring complex schemas from scratch, including relations and indexes - Preserving existing imports in multi-file schema setups - Handling PostgreSQL-specific features like multi-schema support and native enums - Verifying schema preservation for field and table mappings The tests are restructured for better clarity across different database providers. * refactor: restructure introspection provider interface and attribute generation * feat: modernize MySQL introspection provider * feat: modernize PostgreSQL introspection provider * feat: modernize SQLite introspection provider * fix: improve relation field naming and default action handling * feat: track imports and auto-format during db pull * test: update pull tests to reflect naming and formatting improvements * fix(cli): refactor PostgreSQL type casting and fix index order Extracts PostgreSQL type casting logic into a reusable helper function to improve maintainability and ensure consistent attribute handling across all field types. Adjusts the table index sorting logic to better preserve the original database creation order while maintaining the priority of unique indexes. * fix(cli): filter out auto-generated MySQL indexes Prevents foreign key indexes created automatically by MySQL from appearing in the introspected schema. This ensures the output reflects manually defined indexes and avoids redundancy in schema definitions. * test(cli): support datasource extras in test utils Enhances the test utility helpers to allow passing extra datasource properties, such as multi-schema configurations for PostgreSQL. Refactors existing database pull tests to use these extra properties, ensuring the generated ZModel schema correctly reflects multi-schema environments while simplifying assertions. * fix: address PR comments * fix: address PR comments * fix: address PR comments * fix: address PR comments * fix(cli): improve file path resolution in pull action * refactor(cli): extract and enhance name casing logic * refactor(cli): consolidate default value normalization * feat(cli): improve enum syncing and relation naming during pull * docs(cli): add documentation comments to SQL introspection queries * test(cli): refactor test utilities and modernize test suites * fix(cli): improve db pull for composite FKs and MySQL uniqueness Enhances database introspection to correctly handle composite foreign keys by mapping columns by position rather than name alone. Improves MySQL introspection by checking statistics tables for single-column unique indexes, ensuring accurate model generation even when column keys are ambiguous. Ensures MySQL synthetic enum names respect requested model casing to prevent unnecessary schema mapping. Adds comprehensive tests for composite relations and database-specific uniqueness detection. * fix: address PR comments * fix(cli): improve SQLite introspection for untyped columns and composite FKs Ensures columns with no declared type are correctly mapped to Bytes following SQLite affinity rules, preventing them from being marked as Unsupported. Updates the DDL parser to correctly identify and map constraint names for composite foreign keys. This ensures that multi-column relations are properly restored during the pull process. Adds regression tests for both untyped columns and composite foreign key restoration. * feat(cli): pull generated/computed columns as Unsupported type Improves database introspection by identifying generated columns in MySQL, PostgreSQL, and SQLite. These columns are now pulled as `Unsupported` types containing their full DDL definition, preventing issues where read-only database fields were incorrectly treated as writable application-level fields. Includes normalization for expression formatting and a fix for string literal escaping in the code generator to ensure stable schema output. Relates to ZModel introspection consistency. * fix(cli): Use parameterized queries for MySQL introspection Switches from template literal interpolation to parameterized queries in MySQL introspection functions. This improves security by preventing potential SQL injection and ensures better handling of database names containing special characters. * fix(cli): use nullish coalescing for precision check
…g, and comments during db pull (#672) * fix(introspection): preserve schema integrity during db pull - Retain data validation attributes (e.g., @email) on fields after introspection (#670) - Preserve original declaration order of enums instead of moving them to the end of the schema file (#669) - Preserve triple-slash comments above enum declarations (#669) Fixes #669, fixes #670 * fix: address PR comments * fix(cli): improve db pull for relations and defaults Prevents field name collisions during introspection by refining the naming strategy for self-referencing relations with multiple foreign keys. Extends support for JSON and Bytes default values across MySQL, PostgreSQL, and SQLite providers to ensure consistent schema restoration. Adds test cases for self-referencing models to verify the avoidance of duplicate fields. * fix: address PR comments
* chore: add warning for mismatched versions during `zen generate` * chore: swap falsy checks for length checks * chore: do not block generation upon error * Add additional new line. * Trigger Build * Trigger Build * fix: addressing PR comments - use `createRequire` instead of dynamic import for better compatibility - use file search to locate nearest package.json file - use longer padding length when formatting warnings * fix: make sure require base dir is absolute path --------- Co-authored-by: ymc9 <104139426+ymc9@users.noreply.github.com>
Contributor
|
Important Review skippedToo many files! This PR contains 288 files, which is 138 over the limit of 150. You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.