From 0730c32ae63e2a5ac5e2c73670f6a5393c6c0e62 Mon Sep 17 00:00:00 2001 From: James Swirhun Date: Sun, 30 Nov 2025 10:03:57 -0700 Subject: [PATCH 1/3] Add language reference and patterns documentation - Restructure Language Reference with nested subsections (Core Concepts, Field Types, Data & Expressions, Query Operations) - Add dimensions.malloynb documentation - Remove deprecated dialect files, connections, imports, sql_sources, fields docs - Add patterns/index.malloynb overview page - Remove patterns/transform.malloynb (moved to user_guides) - Update experiments: remove window.malloynb, update experiments.malloynb and sql_expressions.malloynb - Update table of contents with new structure --- .../experiments/experiments.malloynb | 4 +- .../experiments/sql_expressions.malloynb | 9 +- src/documentation/experiments/window.malloynb | 2 - .../language/aggregates.malloynb | 2 +- .../language/calculations_windows.malloynb | 2 +- src/documentation/language/changelog.malloynb | 4 +- .../language/connections.malloynb | 38 ---- .../language/dialect/bigquery.malloynb | 30 ---- .../language/dialect/duckdb.malloynb | 27 --- .../language/dialect/mysql.malloynb | 34 ---- .../language/dialect/postgres.malloynb | 21 --- .../language/dialect/presto-trino.malloynb | 103 ----------- .../language/dialect/snowflake.malloynb | 28 --- .../language/dimensions.malloynb | 141 +++++++++++++++ .../language/expressions.malloynb | 2 +- src/documentation/language/fields.malloynb | 148 ---------------- .../language/hyperloglog.malloynb | 6 +- src/documentation/language/imports.malloynb | 34 ---- src/documentation/language/order_by.malloynb | 4 +- src/documentation/language/source.malloynb | 50 +++++- .../language/sql_sources.malloynb | 78 --------- src/documentation/language/statement.malloynb | 34 +++- src/documentation/language/tags.malloynb | 2 +- src/documentation/language/views.malloynb | 4 +- src/documentation/patterns/index.malloynb | 44 +++++ src/documentation/patterns/transform.malloynb | 50 ------ src/table_of_contents.json | 163 ++++++++---------- 27 files changed, 350 insertions(+), 714 deletions(-) delete mode 100644 src/documentation/experiments/window.malloynb delete mode 100644 src/documentation/language/connections.malloynb delete mode 100644 src/documentation/language/dialect/bigquery.malloynb delete mode 100644 src/documentation/language/dialect/duckdb.malloynb delete mode 100644 src/documentation/language/dialect/mysql.malloynb delete mode 100644 src/documentation/language/dialect/postgres.malloynb delete mode 100644 src/documentation/language/dialect/presto-trino.malloynb delete mode 100644 src/documentation/language/dialect/snowflake.malloynb create mode 100644 src/documentation/language/dimensions.malloynb delete mode 100644 src/documentation/language/fields.malloynb delete mode 100644 src/documentation/language/imports.malloynb delete mode 100644 src/documentation/language/sql_sources.malloynb create mode 100644 src/documentation/patterns/index.malloynb delete mode 100644 src/documentation/patterns/transform.malloynb diff --git a/src/documentation/experiments/experiments.malloynb b/src/documentation/experiments/experiments.malloynb index 65b8a9ee..2d54ba6f 100644 --- a/src/documentation/experiments/experiments.malloynb +++ b/src/documentation/experiments/experiments.malloynb @@ -5,11 +5,9 @@ Before releasing language features, we like to get a feel for how they work when We value feedback during the experimental phase. Please make sure you let us know what you think. -Below is a list of currently running experiements and how to turn them on. +Below is a list of currently running experiments and how to turn them on. * `##! experimental.join_types` - [Additional Join Type](joins.malloynb) -* `## renderer_next` - [New table and visualization Rendering](renderer.malloynb) -* `##! experimental { function_order_by partition_by aggregate_limit }` - [ordering and partitioning in calculations](window.malloynb) * `##! experimental.sql_functions` - [Write expression in SQL](sql_expressions.malloynb) * `##! experimental.parameters` - [Declare sources with parameters](parameters.malloynb) * `##! experimental.composite_sources` - [Create virtual sources backed by multiple cube tables or source definitions](composite_sources.malloynb) diff --git a/src/documentation/experiments/sql_expressions.malloynb b/src/documentation/experiments/sql_expressions.malloynb index c0a997b2..80b73c45 100644 --- a/src/documentation/experiments/sql_expressions.malloynb +++ b/src/documentation/experiments/sql_expressions.malloynb @@ -1,14 +1,15 @@ >>>markdown ## SQL Expressions -Malloy allows you to call native database functions using `!type`. For example if you wanted to call the duckdb function bit_length that returns a number, in Malloy you couild write `bitlength!number("this string)`. +To enable this feature, add `##! experimental{sql_functions}` at the top of your Malloy file. -Sometimes the SQL expression you want to write can't be expressed this way. For example in DUCKDB, the extract function looks like. +Malloy allows you to call native database functions using `!type`. For example if you wanted to call the duckdb function bit_length that returns a number, in Malloy you could write `bitlength!number("this string")`. - `extract(part from date)` +Sometimes the SQL expression you want to write can't be expressed this way. For example in DuckDB, the extract function looks like: + `extract(part from date)` -In order to make Malloy write an expression like this you can escape to a `sql_` function. In the string parameter you can reference dimensions using the substitution operator `${dimension_name}`. +In order to make Malloy write an expression like this you can escape to a `sql_` function. In the string parameter you can reference dimensions using the substitution operator `${dimension_name}`. SQL functions are diff --git a/src/documentation/experiments/window.malloynb b/src/documentation/experiments/window.malloynb deleted file mode 100644 index 2359eea4..00000000 --- a/src/documentation/experiments/window.malloynb +++ /dev/null @@ -1,2 +0,0 @@ ->>>markdown -## Calculation Partitioning \ No newline at end of file diff --git a/src/documentation/language/aggregates.malloynb b/src/documentation/language/aggregates.malloynb index 1c617045..ecb993ee 100644 --- a/src/documentation/language/aggregates.malloynb +++ b/src/documentation/language/aggregates.malloynb @@ -1,7 +1,7 @@ >>>markdown # Aggregates -Malloy supports the standard aggregate functions `count`, `sum`, `avg`, `min`, and `max`. When these are used in a field's definition, they make that field a [measure](fields.malloynb#measures). +Malloy supports the standard aggregate functions `count`, `sum`, `avg`, `min`, and `max`. When these are used in a field's definition, they make that field a measure. ## Basic Syntax diff --git a/src/documentation/language/calculations_windows.malloynb b/src/documentation/language/calculations_windows.malloynb index 588833c7..2d6f4e4f 100644 --- a/src/documentation/language/calculations_windows.malloynb +++ b/src/documentation/language/calculations_windows.malloynb @@ -1,5 +1,5 @@ >>>markdown -# Calculations and Window Functions +# Calculations Window functions in Malloy are expressed in _calculation_ fields, using the `calculate: ` keyword. Calculation fields operate on the results of a aggregation or selection, while still operating within the same query stage. Logically these calculation operations occur "after" the other operations, so their exact semantics can be challenging to understand. For a full list of the window functions that Malloy supports, visit our [function reference documentation](./functions.malloynb#window-functions). diff --git a/src/documentation/language/changelog.malloynb b/src/documentation/language/changelog.malloynb index ba7038a2..4dd0b8d3 100644 --- a/src/documentation/language/changelog.malloynb +++ b/src/documentation/language/changelog.malloynb @@ -18,7 +18,7 @@ For more information about views, see the [Views](./views.malloynb) section. New syntax for defining sources based on tables, `duckdb.table('data/users.parquet')` has been introduced. The old syntax, `table('duckdb:data/users.parquet')` still works for the time being, but will be deprecated in 4.0. -See the [Connections](./connections.malloynb#table-connection-method) section for more details. +See the [Sources](./source.malloynb) section for more details. ### SQL Source Method @@ -26,7 +26,7 @@ New syntax for defining sources based on SQL queries, `duckdb.sql("""select * .. This makes `from_sql` no longer necessary, and it will be deprecated with the `sql:` statement. -See the [SQL Sources](./sql_sources.malloynb) section for more details. +See the [Sources](./source.malloynb#sources-from-sql-queries) section for more details. ### Source Extensions and Query Refinements diff --git a/src/documentation/language/connections.malloynb b/src/documentation/language/connections.malloynb deleted file mode 100644 index 7fbdcd60..00000000 --- a/src/documentation/language/connections.malloynb +++ /dev/null @@ -1,38 +0,0 @@ ->>>markdown -# Connections - -In Malloy, named connection objects are used to interact with a database, including fetching schemas for SQL queries and tables, as well as actually running queries. - -Connection objects are defined implicitly on their first use when referencing a table or SQL source: ->>>malloy -// This creates a model-level connection definition, `duckdb` -source: users is duckdb.table('../data/users.parquet') - -run: users -> { group_by: first_name; limit: 5 } ->>>markdown - -# Connection Methods - -There are currently two connection methods, `.table()` and `.sql()`. - -## Table Connection Method - -The `.table()` connection method is used to reference a table or view in a database. It accepts a single string representing a table. The exact semantics of how that string is resolved into a table schema depend on the database and application. - -In the official Malloy connection implementations, the behavior is as follows: - -### BigQuery - -In BigQuery, the string passed to the `.table()` connection method can be a two- or three-segment path including the (optional) project ID, dataset ID, and table name, e.g. `bigquery.table('project-id.dataset-id.table-name')` or `bigquery.table('dataset-id.table-name')`. If the project ID is left off, the default project ID for the connection will be used, or else the system default if none is set on the connection. - -### DuckDB - -In DuckDB, the `.table()` method accepts the path (relative to the Malloy file) of CSV, JSON, or Parquet file containing the table data, e.g. `duckdb.table('data/users.csv')` or `duckdb.table('../../users.parquet')`. URLs to such files (or APIs) are also allowed: see [an example here](../patterns/apijson.malloynb). - -### Postgres - -In Postgres, the string passed to the `.table()` connection method can be a two- or three-segment path including the (optional) database ID, schema name, and table name, e.g. `postgres.table('database-id.schema-name.table-name')` or `postgres.table('schema-name.table-name')`. If the database ID is left off, the default database for the connection will be used, or else the system default if none is set on the connection. - -## SQL Connection Method - -The `.sql()` connection method is used to define a source or query based on a SQL query. See the [SQL Sources](./sql_sources.malloynb) section for more information. \ No newline at end of file diff --git a/src/documentation/language/dialect/bigquery.malloynb b/src/documentation/language/dialect/bigquery.malloynb deleted file mode 100644 index 24be3018..00000000 --- a/src/documentation/language/dialect/bigquery.malloynb +++ /dev/null @@ -1,30 +0,0 @@ ->>>markdown -# Google Standard SQL (BigQuery) - -# Functions - -## Useful Functions not in the database function library - - string_agg_distinct - count_approx - -## Database Functions - -Malloy code can, in addition to the [Malloy Standard Functions](../functions.malloynb), reference -any of the listed functions here without needing to use [Raw SQL Functions](../functions.malloynb#raw-sql-functions). - - date_from_unix_date - string_agg - repeat - reverse - -## HyperLogLog Functions - -In addition to the [Malloy Standard HLL Functions](../hyperloglog.malloynb), BigQuery supports -`HLL_COUNT.MERGE`, which both merges sketches and estimates cardinality. - -See the [BigQuery HyperLogLog Documentation](https://cloud.google.com/bigquery/docs/reference/standard-sql/hll_functions). - -# External Resources - -* [BigQuery SQL Reference](https://cloud.google.com/bigquery/docs/introduction-sql) diff --git a/src/documentation/language/dialect/duckdb.malloynb b/src/documentation/language/dialect/duckdb.malloynb deleted file mode 100644 index 6d516bf3..00000000 --- a/src/documentation/language/dialect/duckdb.malloynb +++ /dev/null @@ -1,27 +0,0 @@ ->>>markdown -# DuckDB - -# Functions - -## Useful Functions not in the database function library - - count_approx - string_agg_distinct - -## Database Functions - -Malloy code can, in addition to the [Malloy Standard Functions](../functions.malloynb), reference -any of the listed functions here without needing to use [Raw SQL Functions](../functions.malloynb#raw-sql-functions). - - list_extract - dayname - to_timestamp - string_agg - to_seconds - date_part - repeat - reverse - -# External Resources - -* [DuckB Documentation](https://duckdb.org/docs/) diff --git a/src/documentation/language/dialect/mysql.malloynb b/src/documentation/language/dialect/mysql.malloynb deleted file mode 100644 index 885fab5e..00000000 --- a/src/documentation/language/dialect/mysql.malloynb +++ /dev/null @@ -1,34 +0,0 @@ ->>>markdown -# MySQL - -# Functions - -## Useful Functions not in the database function library - - string_agg - string_agg_distinct - -## Database Functions - -Malloy code can, in addition to the [Malloy Standard Functions](../functions.malloynb), reference -any of the listed functions here without needing to use [Raw SQL Functions](../functions.malloynb#raw-sql-functions). - - repeat - reverse - -# Errata - -## Boolean Filter Expressions - -Becase MySQL doesn't have a true boolean data type, a columns which contains boolean values -will be read as numeric by Malloy. This will make it impossible to use boolean [filter expressions](../filter-expressions.malloynb) -without explicitly casting the value to type boolean - -```malloy -dimension: presentAndNotAccountedFor is present::boolean ~ f'true' and accountedFor::boolean ~ f'false' -``` - -# External Resources - -* [MySQL Documentaion](https://dev.mysql.com/doc/) ->>>markdown diff --git a/src/documentation/language/dialect/postgres.malloynb b/src/documentation/language/dialect/postgres.malloynb deleted file mode 100644 index 1e4abe58..00000000 --- a/src/documentation/language/dialect/postgres.malloynb +++ /dev/null @@ -1,21 +0,0 @@ ->>>markdown -# PostgreSQL - -# Functions - -## Useful Functions not in the database function library - - string_agg_distinct - -## Database Functions - -Malloy code can, in addition to the [Malloy Standard Functions](../functions.malloynb), reference -any of the listed functions here without needing to use [Raw SQL Functions](../functions.malloynb#raw-sql-functions). - - string_agg - repeat - reverse - -# External Resouces - -* [PostgreSQL Documentation](https://www.postgresql.org/docs/) diff --git a/src/documentation/language/dialect/presto-trino.malloynb b/src/documentation/language/dialect/presto-trino.malloynb deleted file mode 100644 index 3796e61b..00000000 --- a/src/documentation/language/dialect/presto-trino.malloynb +++ /dev/null @@ -1,103 +0,0 @@ ->>>markdown -# Presto / Trino Dialect - -Malloy supports Presto and Trino databases with two dialects which are closely related. - -# Functions - -## Useful Functions not in the database function library - - hll_accumulate - hll_combine - hll_estimate - hll_export - hll_import - string_agg - string_agg_distinct - count_approx - -## Database Functions - -Malloy code written for these dialects can, in addition to the [Malloy Standard Functions](../functions.malloynb), reference -any of the listed functions here without needing to use [Raw SQL Functions](../functions.malloynb#raw-sql-functions). - -Refer to the appropriate database documentaion for information on individual functions. - - approx_percentile - arbitrary - array_distinct - array_except - array_intersect - array_join - array_max - array_min - array_normalize - array_remove - array_sort - array_split_into_chunks - array_union - arrays_overlap - bitwise_and - bitwise_and_agg - bitwise_or - bitwise_or_agg - bitwise_xor_agg - bool_and - bool_or - cardinality - combinations - contains - corr - date_format - date_parse - element_at - flatten - from_unixtime - json_extract_scalar - max_by - min_by - ngrams - percent_rank - regexp_like - regexp_replace - regexp_extract - repeat(STRING) - reverse - sequence - shuffle - slice - split - split_part - to_unixtime - trim_array - url_extract_fragment - url_extract_host - url_extract_parameter - url_extract_path - url_extract_port - url_extract_protocol - url_extract_query - variance - - -## Presto Only - -The following functions are only available on Presto connections - - array_average - array_cum_sum - array_duplicates - array_intersect - array_has_duplicates - array_least_frequent - array_position array_sum - array_sort_desc - array_top_n - remove_nulls - reverse(ARRAY) - -# External resources - -* [Presto Documentation](https://prestodb.io/docs/current/) -* [Trino Documentation](https://trino.io/docs/current/index.html) -* [Trino HyperLogLog Documentation](https://trino.io/docs/current/functions/hyperloglog.html) \ No newline at end of file diff --git a/src/documentation/language/dialect/snowflake.malloynb b/src/documentation/language/dialect/snowflake.malloynb deleted file mode 100644 index bd7b6485..00000000 --- a/src/documentation/language/dialect/snowflake.malloynb +++ /dev/null @@ -1,28 +0,0 @@ ->>>markdown -# Snowflake - -# Functions - -## Useful Functions not in the database function library - - string_agg_distinct - string_agg - -## Database Functions - -Malloy code can, in addition to the [Malloy Standard Functions](../functions.malloynb), reference -any of the listed functions here without needing to use [Raw SQL Functions](../functions.malloynb#raw-sql-functions). - - repeat - reverse - -## HyperLogLog Functions - -In addition to the [Malloy Standard HLL Functions](../hyperloglog.malloynb), Snowflake supports -`HLL`, which both initializes a new HLL sketch, and estimates cardinality. - -See the [Snowflake HyperLogLog Documentation](https://docs.snowflake.com/en/user-guide/querying-approximate-cardinality). - -# External Resources - -* [Snowflake Documentation](https://docs.snowflake.com/) diff --git a/src/documentation/language/dimensions.malloynb b/src/documentation/language/dimensions.malloynb new file mode 100644 index 00000000..b8f1c587 --- /dev/null +++ b/src/documentation/language/dimensions.malloynb @@ -0,0 +1,141 @@ +>>>markdown +# Dimensions + +Dimensions are scalar fields that represent attributes of your data. They're used to categorize, group, and filter data in queries. Unlike [measures](aggregates.malloynb), dimensions don't aggregate—they represent individual values. + +## Defining Dimensions + +Dimensions are defined in a source using the `dimension:` keyword. +>>>malloy +source: users is duckdb.table('../data/users.parquet') extend { + dimension: full_name is concat(first_name, ' ', last_name) + dimension: age_group is + pick 'minor' when age < 18 + pick 'adult' when age < 65 + else 'senior' +} +>>>markdown + +All columns from the underlying table are automatically available as dimensions—you only need to explicitly define computed dimensions. + +## Using Dimensions in Queries + +Dimensions can be used in several contexts: + +### Grouping with `group_by:` + +Use dimensions to group aggregated results: +>>>malloy +run: users -> { + group_by: full_name + aggregate: count_users is count() + limit: 5 +} +>>>markdown + +### Selecting with `select:` + +Use dimensions in projection queries (no aggregation): +>>>malloy +run: users -> { + select: full_name, age + limit: 5 +} +>>>markdown + +### Inline Dimensions + +Dimensions can be defined inline in a query: +>>>malloy +run: duckdb.table('../data/users.parquet') -> { + group_by: + name_length is length(first_name) + aggregate: user_count is count() + limit: 5 +} +>>>markdown + +## Computed Dimensions + +Dimensions can be computed from other fields using expressions. + +### String Operations +>>>malloy +source: users2 is duckdb.table('../data/users.parquet') extend { + dimension: initials is concat(substr(first_name, 1, 1), substr(last_name, 1, 1)) + dimension: email_domain is substr(email, strpos(email, '@') + 1) +} +>>>markdown + +### Date/Time Extractions +>>>malloy +##(docs) hidden +source: flights is duckdb.table('../data/flights.parquet') extend { + measure: flight_count is count() +} +>>>malloy +run: flights -> { + group_by: + dep_year is dep_time.year + dep_quarter is dep_time.quarter + dep_month is dep_time.month + aggregate: flight_count + limit: 5 +} +>>>markdown + +### Conditional Logic with `pick` +>>>malloy +source: flights2 is duckdb.table('../data/flights.parquet') extend { + dimension: distance_category is + pick 'short' when distance < 500 + pick 'medium' when distance < 1500 + else 'long' + measure: flight_count is count() +} + +run: flights2 -> { + group_by: distance_category + aggregate: flight_count +} +>>>markdown + +## Dimensions from Joins + +When sources are joined, dimensions from joined sources are accessible using dot notation: +>>>malloy +source: carriers is duckdb.table('../data/carriers.parquet') extend { + primary_key: code +} + +source: flights3 is duckdb.table('../data/flights.parquet') extend { + join_one: carriers with carrier + measure: flight_count is count() +} + +run: flights3 -> { + group_by: carriers.nickname + aggregate: flight_count + limit: 5 +} +>>>markdown + +## Dimension Naming + +Dimension names must start with a letter or underscore and contain only letters, numbers, and underscores. Names that conflict with Malloy keywords must be enclosed in backticks: + +```malloy +dimension: `year` is dep_time.year +dimension: `order` is sort_order +``` + +## Dimensions vs Measures + +| Aspect | Dimensions | Measures | +|--------|------------|----------| +| Definition | Scalar calculations | Aggregate calculations | +| Used in | `group_by:`, `select:` | `aggregate:` | +| Purpose | Categorize/group data | Compute statistics | +| Example | `carrier`, `dep_time.year` | `count()`, `sum(distance)` | + +For aggregate calculations, see [Aggregates](aggregates.malloynb). diff --git a/src/documentation/language/expressions.malloynb b/src/documentation/language/expressions.malloynb index bc51b522..74f97f8a 100644 --- a/src/documentation/language/expressions.malloynb +++ b/src/documentation/language/expressions.malloynb @@ -104,7 +104,7 @@ Many functions available in SQL are available unchanged in Malloy. Malloy provid ## Aggregation -Aggregations may included in an expression to create [measures](fields.malloynb#measures), e.g. `count()`, `sum(distance)`, or `aircraft_models.seats.avg()`. For detailed information, see the [Aggregates](aggregates.malloynb) section. +Aggregations may be included in an expression to create measures, e.g. `count()`, `sum(distance)`, or `aircraft_models.seats.avg()`. For detailed information, see the [Aggregates](aggregates.malloynb) section. diff --git a/src/documentation/language/fields.malloynb b/src/documentation/language/fields.malloynb deleted file mode 100644 index 63fb4c54..00000000 --- a/src/documentation/language/fields.malloynb +++ /dev/null @@ -1,148 +0,0 @@ ->>>markdown -# Fields - -Fields constitute all kinds of data in Malloy. They -can represent dimensional attributes sourced directly from -tables in a database, constant values to be used in later analysis, computed metrics derived from other fields, or even nested structures created from aggregating subqueries. - -## Defining Fields - -Fields defined in sources are reusable. A field is a *dimension*, *measure* or *view*, or *calculation*. When these are used in a query, these fields are invoked with `select:`, `group_by:`, `aggregate:`, `nest:`, or `calculate:`. Their definitions are syntactically identical whether defined in a source or a view (with the exception of calculations, which can only be defined in a view, and not in a source). In either case, they are defined using the `is` keyword. - -**In a source** ->>>malloy -source: users is duckdb.table('../data/users.parquet') extend { - dimension: age_in_dog_years is age * 7 -} ->>>markdown - -**In a query** ->>>malloy -run: users -> { - group_by: age_in_dog_years is age * 7 -} ->>>markdown - -The right hand side of this kind of definition can be any -field expression. See the [Expressions](expressions.malloynb) -section for more information. - -Like dimensions and measures, views can also be defined as -part of a source or in a query's view. When a view is used or defined in another view, it is known as a "nested view" and produces an "aggregating -subquery." See the [Nesting](nesting.malloynb) section for a -detailed discussion of nested views. ->>>malloy -##(docs) hidden -import "flights.malloy" ->>>malloy -run: flights -> { - group_by: carrier - nest: by_month is { - group_by: departure_month is dep_time.month - aggregate: flight_count is count() - limit: 3 - } -} ->>>markdown - -## Field Names - -Field names generally must start with a letter or underscore, and can only contain letters, numbers, and underscores. Field names which don't follow these rules, or which conflict with a Malloy keyword, must be enclosed in back ticks, e.g. `` `year` is dep_time.year``. - -## Kinds of Fields - -Malloy has four different kinds of fields: _dimensions_, _measures_, _views_, and _calculations_. - -### Dimensions - -Dimensions are fields representing scalar values. All fields -inherited directly from a table are dimensions. - -Dimensions are defined using expressions that contain no -aggregate functions. ->>>malloy -source: users2 is duckdb.table('../data/users.parquet') extend { - dimension: full_name is concat(first_name, ' ', last_name) -} ->>>markdown - -Dimensions may be used in both reductions and projections. ->>>malloy -// Show the top 10 full names by number of occurrences -run: users2 -> { - limit: 10 - group_by: full_name - aggregate: occurrences is count() -} - -// Show 10 users' full names -run: users2 -> { - select: full_name - limit: 10 -} ->>>markdown - -### Measures - -Measures are fields representing aggregated data over multiple records. Measures may not be used in projections (i.e., `select:` queries), they can only be used in "group by" or "aggregating" queries. ->>>malloy -source: flights2 is duckdb.table('../data/flights.parquet') extend { - measure: - -- define the measure in the source: - total_flight_time is sum(flight_time) -} - -run: flights2 -> { - group_by: carrier - -- use the previously defined measure in a query: - aggregate: total_flight_time -} ->>>markdown - -### Views - -A view is a query that has been saved to the semantic model. By adding a view to a Source, you are indicating that this is an important query for the dataset. ->>>markdown -```malloy -source: flights is duckdb.table('../data/flights.parquet') extend { - view: by_carrier is { - group_by: carrier - aggregate: flight_count is count() - } -} -``` ->>>markdown -A view can always begin with another view from the same source. In the following example, `top_carriers` is a view that starts with the `by_carrier` view, selects only the `carrier` column, and sets a limit of 5 rows. ->>>markdown -```malloy -source: flights is duckdb.table('../data/flights.parquet') extend { - ... - view: top_carriers is by_carrier -> { - select: carrier - limit: 5 - } -} -``` ->>>markdown - - - -Views are composable building blocks, which you can use for complex analysis. The following query nests two different views: the `by_carrier` view computes `flight_count` and `destination_count` for each airline in the dataset. Nesting the `by_manufacturer` view shows the aircraft count, broken out by manufacturer within each of the airlines. ->>>malloy -run: flights -> by_carrier + { - limit: 5 - - nest: by_manufacturer -} ->>>markdown -See the [Nesting](nesting.malloynb) section for more details about nested views. - -### Calculations (Window Functions) - -*Calculations* are fields based off of groupings and aggregate values in a view, and therefore can only be created in a view with `calculate:` and can not be predefined in a source. See the [Calculations and Window Functions](./calculations_windows.malloynb) section for details. ->>>malloy -run: flights -> { - group_by: carrier - aggregate: flight_count - calculate: flight_count_rank is rank() -} \ No newline at end of file diff --git a/src/documentation/language/hyperloglog.malloynb b/src/documentation/language/hyperloglog.malloynb index f0d4ad1f..8e8437fb 100644 --- a/src/documentation/language/hyperloglog.malloynb +++ b/src/documentation/language/hyperloglog.malloynb @@ -27,11 +27,11 @@ In Malloy, you can use the HyperLogLog algorithm to efficiently estimate the car Malloy HyperLogLog functions *can only be used with the following databases*: -* **Presto-Trino** See the [Malloy Presto-Trino Page](dialect/presto-trino.malloynb). +* **Presto-Trino** -* **BigQuery**. See the [Malloy BigQuery Page](dialect/bigquery.malloynb). +* **BigQuery** -* **Snowflake**. See the [Malloy Snowflake Page](dialect/snowflake.malloynb). +* **Snowflake** ## HLL Functions diff --git a/src/documentation/language/imports.malloynb b/src/documentation/language/imports.malloynb deleted file mode 100644 index e1cd710c..00000000 --- a/src/documentation/language/imports.malloynb +++ /dev/null @@ -1,34 +0,0 @@ ->>>markdown -# Imports - -In order to reuse or extend a source from another file, you can include all the -exported sources from another file using `import "path/to/some/file.malloy"`. - -For example, if you wanted to create a file flights_by_carrier.malloy with a query from the `flights` source, you could write: ->>>malloy -import "flights.malloy" - -run: flights -> { limit: 5; group_by: carrier; aggregate: flight_count } ->>>markdown - -## Import Locations - -Imported files may be specified with relative or absolute URLs. - -| Import Statement | Meaning from `"file:///f1/a.malloy"` | -| ---------------- | --------| -| `import "b.malloy"` | `"file:///f1/b.malloy"` | -| `import "./c.malloy"` | `"file:///f1/c.malloy"` | -| `import "/f1/d.malloy"` | `"file:///f1/d.malloy"` | -| `import "file:///f1/e.malloy"` | `"file:///f1/e.malloy"` | -| `import "../f2/f.malloy"` | `"file:///f2/f.malloy"` | -| `import "https://example.com/g.malloy"` | `"https://example.com/g.malloy"` | - -## Selective Imports - -The default is to import all objects from the referenced file. You can also use `{} from` to select (and optionally rename) specific objects to be imported. ->>>malloy -import { airports, spaceports is airports } from "airports.malloy" - -run: airports -> { aggregate: airport_count is count() } -run: spaceports -> { aggregate: spaceport_count is count() } \ No newline at end of file diff --git a/src/documentation/language/order_by.malloynb b/src/documentation/language/order_by.malloynb index 3a6d18f8..54debb44 100644 --- a/src/documentation/language/order_by.malloynb +++ b/src/documentation/language/order_by.malloynb @@ -7,7 +7,7 @@ Often when querying data the amount of data returned to look at is much smaller ## Implicit Ordering ### Rule 1: Newest first -If a query stage has a [dimensional](fields.malloynb#dimensions) column that represents a point in time, it is usually the most +If a query stage has a [dimensional](dimensions.malloynb) column that represents a point in time, it is usually the most important concept in the query. Because the most recent data is usually the most relevant, Malloy sorts the newest data first. >>>malloy ##(docs) hidden @@ -20,7 +20,7 @@ run: flights -> { >>>markdown ### Rule 2: Largest first -If there is a [measure](fields.malloynb#measures) involved, Malloy sorts larger values first. +If there is a [measure](aggregates.malloynb) involved, Malloy sorts larger values first. In the following example, Rule 1 doesn't apply, so the default behavior is to sort by first aggregate, `flight_count` with the largest values first. >>>malloy diff --git a/src/documentation/language/source.malloynb b/src/documentation/language/source.malloynb index e99d0d31..bfd6bbec 100644 --- a/src/documentation/language/source.malloynb +++ b/src/documentation/language/source.malloynb @@ -111,7 +111,7 @@ run: carrier_facts2 -> { ### Sources from SQL Queries -Sources can be created from a SQL query, e.g. +Sources can be created from a SQL query using the `.sql()` connection method: >>>malloy source: limited_users is duckdb.sql(""" SELECT @@ -130,6 +130,52 @@ run: limited_users -> { Like with `duckdb.table('data/users.parquet')`, Malloy fetches the schema from the database to make columns of the resulting table accessible in computations. +Unlike other sources, SQL sources can be run directly as queries: +>>>malloy +run: duckdb.sql("select 1 as one") +>>>markdown + +They can also be defined as named queries: +>>>malloy +query: my_sql_query is duckdb.sql("select 1 as one") +run: my_sql_query +>>>markdown + +SQL sources can be extended like any other source: +>>>malloy +source: users_extended is duckdb.sql(""" + SELECT first_name, last_name, gender + FROM '../data/users.parquet' + LIMIT 10 +""") extend { + measure: user_count is count() +} + +run: users_extended -> { aggregate: user_count } +>>>markdown + +#### Embedding Malloy in SQL ("Turducken") + +Malloy queries can be embedded inside SQL blocks. When `%{` and `}` appear inside a `"""` quoted string, the Malloy query between the brackets is compiled and replaced with the generated SQL: +>>>malloy +source: users is duckdb.table('../data/users.parquet') + +source: malloy_in_sql is duckdb.sql(""" + SELECT * FROM + (%{ // Malloy query starts with %{ + users -> { + limit: 10 group_by: first_name, last_name, gender + aggregate: n_with_this_name is count() + } + }) -- Malloy query ends after } + WHERE n_with_this_name > 10 +""") + +run: malloy_in_sql -> { select: * } +>>>markdown + +This technique is nicknamed ["Turducken"](https://en.wikipedia.org/wiki/Turducken) because you wrap Malloy inside SQL, then wrap that SQL back into a Malloy source. + ## Source Extensions Any source can be extended to add filters, specify a primary key, add fields and joins, rename fields, or limit which fields are available. @@ -187,7 +233,7 @@ run: airports2 -> { limit: 2 } >>>markdown -For more information about fields and how to define them, see the [Fields](./fields.malloynb) section, or for information on views specifically, see the [Views](./views.malloynb) section. +For more information, see [Dimensions](./dimensions.malloynb), [Aggregates](./aggregates.malloynb), and [Views](./views.malloynb). ### Filtering Sources diff --git a/src/documentation/language/sql_sources.malloynb b/src/documentation/language/sql_sources.malloynb deleted file mode 100644 index b86a03b4..00000000 --- a/src/documentation/language/sql_sources.malloynb +++ /dev/null @@ -1,78 +0,0 @@ ->>>markdown -# SQL Sources - -_SQL sources, introduced in Malloy version 0.56, replace the previous method of including SQL queries in a Malloy model, [SQL blocks](./sql_blocks.malloynb)._ - -Sometimes it can be useful to base Malloy models off of SQL queries. You can do so by using the `.sql()` [connection method](./connections.malloynb#connection-methods). ->>>malloy -source: my_sql_source is duckdb.sql(""" - SELECT - first_name, - last_name, - gender - FROM '../data/users.parquet' - LIMIT 10 -""") ->>>markdown - -These SQL sources can be used any place a table source can be used: ->>>malloy -run: my_sql_source -> { - group_by: first_name - aggregate: user_count is count() -} ->>>markdown - -Unlike other kinds of sources, SQL sources can be used like a query in some cases. - -They can be run directly: ->>>malloy -run: duckdb.sql("select 1 as one") ->>>markdown - -And they can also be defined as a query: ->>>malloy -query: my_sql_query is duckdb.sql("select 1 as one") -run: my_sql_query ->>>markdown - -_Note: you can only run a SQL source as a query when it is defined as a `query:` or included directly in a `run:` statement._ - -## Extending SQL Sources - -Like other kinds of source, SQL sources can be extended to add reusable computations. ->>>malloy -source: limited_users is duckdb.sql(""" - SELECT - first_name, - last_name, - gender - FROM '../data/users.parquet' - LIMIT 10 -""") extend { - measure: user_count is count() -} - -run: limited_users -> { aggregate: user_count } ->>>markdown - -## Embedding Malloy Queries in an SQL Block (A.K.A. "Turducken") - -Malloy queries can be embedded in SQL blocks as well. When `%{` and `}` appear inside a `"""` quoted (but not a `"` or `'` quoted) string of an SQL source, the Malloy query between the brackets is compiled and replaced with the SELECT statement generated from the query. ->>>malloy -source: users is duckdb.table('../data/users.parquet') - -source: malloy_in_sql_query is duckdb.sql(""" - SELECT * FROM - (%{ // Malloy query starts with the %{ - users -> { - limit: 10 group_by: first_name, last_name, gender - aggregate: n_with_this_name is count() - } - }) -- Malloy query ends after the } - WHERE n_with_this_name > 10 -""") - -run: malloy_in_sql_query -> { select: * } ->>>markdown -_We have referred to this feature as ["Turducken"](https://en.wikipedia.org/wiki/Turducken) because you then take the SQL block and wrap it in an SQL source. It isn't the perfect name for infinite nesting, but it is amusing_ diff --git a/src/documentation/language/statement.malloynb b/src/documentation/language/statement.malloynb index 6a0768ce..3052bc55 100644 --- a/src/documentation/language/statement.malloynb +++ b/src/documentation/language/statement.malloynb @@ -1,14 +1,13 @@ >>>markdown -# Models +# File Structure -Malloy recognizes modeling as a key aspect of data analytics and provides tools that allow for modularity and reusability of definitions. Whereas in SQL, queries generally define all metrics inline, requiring useful snippets to be saved and managed separately, in Malloy, -_dimensions_, _measures_, and _views_ can be saved and attached to a modeled source, which itself is defined as part of a Malloy document, often referred to as a _model_. +A Malloy file (`.malloy`) is a collection of [statements](#statements), [comments](#comments), and [tags](#tags). Statements can be separated with an optional semi-colon for clarity, which helps readability when statements appear on the same line. -A Malloy document is a collection of [statements](#statements), [comments](#comments), and [tags](#tags). Statements can be separated with an optional semi-colon for clarity, which helps readability when statements appear on the same line. +Malloy recognizes modeling as a key aspect of data analytics and provides tools for modularity and reusability. Whereas SQL queries generally define all metrics inline, in Malloy, _dimensions_, _measures_, and _views_ can be saved and attached to a modeled source, which itself is defined as part of a Malloy file. ## Statements -There are four kinds of statements that can appear in a Malloy model: +There are four kinds of statements that can appear in a Malloy file: * [Import Statements](#import-statements) * [Query Statements](#query-statements) @@ -18,12 +17,33 @@ There are four kinds of statements that can appear in a Malloy model: ### Import Statements -Import statements allow you to import sources from another .malloy file. +Import statements allow you to import sources and queries from another `.malloy` file. >>>malloy import "flights.malloy" >>>markdown -See the [Imports](imports.malloynb) section for more information. +#### Import Locations + +Imported files may be specified with relative or absolute URLs. + +| Import Statement | Meaning from `"file:///f1/a.malloy"` | +| ---------------- | --------| +| `import "b.malloy"` | `"file:///f1/b.malloy"` | +| `import "./c.malloy"` | `"file:///f1/c.malloy"` | +| `import "/f1/d.malloy"` | `"file:///f1/d.malloy"` | +| `import "file:///f1/e.malloy"` | `"file:///f1/e.malloy"` | +| `import "../f2/f.malloy"` | `"file:///f2/f.malloy"` | +| `import "https://example.com/g.malloy"` | `"https://example.com/g.malloy"` | + +#### Selective Imports + +By default, all objects from the referenced file are imported. You can use `{} from` to select (and optionally rename) specific objects: +>>>malloy +import { airports, spaceports is airports } from "airports.malloy" + +run: airports -> { aggregate: airport_count is count() } +run: spaceports -> { aggregate: spaceport_count is count() } +>>>markdown ### Query Statements diff --git a/src/documentation/language/tags.malloynb b/src/documentation/language/tags.malloynb index 7b942298..c64693f9 100644 --- a/src/documentation/language/tags.malloynb +++ b/src/documentation/language/tags.malloynb @@ -52,7 +52,7 @@ None of these are render tags, even though they use the tag property language. * `##! disableWarnings` tags are parsed by the compiler and interpreted as compiler flags * `#(myApp) custom="application" values="here"` something like this could be used by an app to write custom tags -For a thorough list of available Renderer Tags, refer to the [Render Tags Documentation](https://github.com/malloydata/malloy/blob/main/packages/malloy-render/docs/renderer_tags_overview.md) in the Malloy GitHub project. +For a thorough list of available Renderer Tags, refer to the [Render Tags Documentation](https://github.com/malloydata/malloy/blob/main/packages/malloy-render/docs/renderer_tags_overview.md) in the Malloy GitHub project, or see the [Rendering Results](../visualizations/overview.malloynb) section for usage examples and visualization guides. ## Tag Property Language diff --git a/src/documentation/language/views.malloynb b/src/documentation/language/views.malloynb index 5d6f8525..e7a7c7e2 100644 --- a/src/documentation/language/views.malloynb +++ b/src/documentation/language/views.malloynb @@ -58,9 +58,7 @@ Dimensions are included with `group_by` (or `select` in a projection), measures When referencing existing fields in a `select:` clause, wildcard expressions like `*`, or `some_join.*` may be used. -See the [Fields](fields.malloynb) section for more information -about the different kinds of fields and how they can be -defined. +See [Dimensions](dimensions.malloynb) and [Aggregates](aggregates.malloynb) for more information about field types and how they can be defined. ### Filters diff --git a/src/documentation/patterns/index.malloynb b/src/documentation/patterns/index.malloynb new file mode 100644 index 00000000..0e055ffc --- /dev/null +++ b/src/documentation/patterns/index.malloynb @@ -0,0 +1,44 @@ +>>>markdown +# Common Patterns + +Reusable patterns for common data analysis tasks in Malloy. + +--- + +## Time & Trends + +| Pattern | Description | +|---------|-------------| +| [Comparing Timeframes](yoy.malloynb) | How do I compare metrics across time periods? | +| [Moving Average](moving_avg.malloynb) | How do I smooth out trends with moving averages? | +| [Cohort Analysis](cohorts.malloynb) | How do I track cohorts over time? | + +--- + +## Aggregation + +| Pattern | Description | +|---------|-------------| +| [Percent of Total](percent_of_total.malloynb) | How do I calculate percentages of totals? | +| [Foreign Sums and Averages](foreign_sums.malloynb) | How do I aggregate across joined tables? | +| [Nested Subtotals](nested_subtotals.malloynb) | How do I add subtotals to nested views? | + +--- + +## Data Shaping + +| Pattern | Description | +|---------|-------------| +| [Bucketing with 'Other'](other.malloynb) | How do I group rare values into 'Other'? | +| [Auto-binning Histograms](autobin.malloynb) | How do I create histograms automatically? | +| [Reading Nested Data](reading_nested.malloynb) | How do I work with nested/array data? | +| [Sessionize - Map/Reduce](sessionize.malloynb) | How do I sessionize event data? | + +--- + +## Utility + +| Pattern | Description | +|---------|-------------| +| [Dimensional Indexes](dim_index.malloynb) | How do I build dimensional indexes? | + diff --git a/src/documentation/patterns/transform.malloynb b/src/documentation/patterns/transform.malloynb deleted file mode 100644 index 66ad2cee..00000000 --- a/src/documentation/patterns/transform.malloynb +++ /dev/null @@ -1,50 +0,0 @@ ->>>markdown -# Transforming Data - -Malloy can be used to for data transformation. Files with the extension .malloysql are interpreted as a combination of both Malloy and SQL. Each language is blocked by a preceeding `>>>`. Each block can do anything that is appropriate in the language. In Malloy, for example, you can import other Malloy files. In SQL, you can execute any DDL command. - -In the example below, we create a simple semantic model for the table `airports`. There are two SQL sections, one creates a CSV file of the major airports, the other creates a view in the database with a list of states, the airport count in each state, and the count of each of the facility types for each state. - -This mechanism can be used to create governed datasets for use in other tooling: the transformation code for these tables can be governed with source control, and the queries take advantage of Malloy's reusibility. - -File airports.malloysql: - ```malloysql - >>>malloy - source: airports is duckdb.table('data/airports.parquet') extend { - measure: airport_count is count() - - view: major_airports is { - where: major = 'Y' - select: * - } - - view: by_state is { - group_by: state - aggregate: airport_count - nest: by_fac_type is { - group_by: fac_type - aggregate: airport_count - } - } - } - - >>>sql connection:duckdb - - -- create a table using a Malloy query - - COPY ( - %{ - airports -> major_airports - }% - ) TO 'major_airports.csv' (HEADER) - - >>>sql - - -- create a view using a Malloy query - - CREATE OR REPLACE VIEW by_state as ( - %{ - airports -> by_state - }% - ) - ``` \ No newline at end of file diff --git a/src/table_of_contents.json b/src/table_of_contents.json index 640e91ac..dcaa6a42 100644 --- a/src/table_of_contents.json +++ b/src/table_of_contents.json @@ -55,113 +55,102 @@ "title": "Language Reference", "items": [ { - "title": "Models", - "link": "/language/statement.malloynb" - }, - { - "title": "Sources", - "link": "/language/source.malloynb" - }, - { - "title": "Queries", - "link": "/language/query.malloynb" - }, - { - "title": "Views", - "link": "/language/views.malloynb" - }, - { - "title": "Data Types", - "link": "/language/datatypes.malloynb" - }, - { - "title": "Fields", - "link": "/language/fields.malloynb" - }, - { - "title": "Aggregates", - "link": "/language/aggregates.malloynb" - }, - { - "title": "Expressions", - "link": "/language/expressions.malloynb" - }, - { - "title": "Functions", - "link": "/language/functions.malloynb" - }, - { - "title": "Filters", - "link": "/language/filters.malloynb" - }, - { - "title": "Calculations (window functions)", - "link": "/language/calculations_windows.malloynb" - }, - { - "title": "Ordering and Limiting", - "link": "/language/order_by.malloynb" - }, - { - "title": "Joins", - "link": "/language/join.malloynb" - }, - { - "title": "Nested Views", - "link": "/language/nesting.malloynb" - }, - { - "title": "SQL Sources", - "link": "/language/sql_sources.malloynb" - }, - { - "title": "Imports", - "link": "/language/imports.malloynb" - }, - { - "title": "Connections", - "link": "/language/connections.malloynb" + "title": "Quick Reference", + "link": "/language/quick_reference.html" }, { - "title": "Tags", - "link": "/language/tags.malloynb" + "title": "Core Concepts", + "items": [ + { + "title": "File Structure", + "link": "/language/statement.malloynb" + }, + { + "title": "Sources", + "link": "/language/source.malloynb" + }, + { + "title": "Queries", + "link": "/language/query.malloynb" + }, + { + "title": "Views", + "link": "/language/views.malloynb" + } + ] }, { - "title": "Quick Reference", - "link": "/language/quick_reference.html" + "title": "Field Types", + "items": [ + { + "title": "Dimensions", + "link": "/language/dimensions.malloynb" + }, + { + "title": "Aggregates", + "link": "/language/aggregates.malloynb" + }, + { + "title": "Calculations", + "link": "/language/calculations_windows.malloynb" + } + ] }, { - "title": "Change Log", - "link": "/language/changelog.malloynb" + "title": "Data & Expressions", + "items": [ + { + "title": "Data Types", + "link": "/language/datatypes.malloynb" + }, + { + "title": "Expressions", + "link": "/language/expressions.malloynb" + }, + { + "title": "Functions", + "link": "/language/functions.malloynb" + } + ] }, { - "title": "Dialects", + "title": "Query Operations", "items": [ { - "title": "BigQuery", - "link": "/language/dialect/bigquery.malloynb" + "title": "Filters", + "link": "/language/filters.malloynb" }, { - "title": "Presto / Trino", - "link": "/language/dialect/presto-trino.malloynb" + "title": "Joins", + "link": "/language/join.malloynb" }, { - "title": "Snowflake", - "link": "/language/dialect/snowflake.malloynb" + "title": "Nested Views", + "link": "/language/nesting.malloynb" }, - { "title": "MySQL", "link": "/language/dialect/mysql.malloynb" }, - { "title": "DuckDB", "link": "/language/dialect/duckdb.malloynb" }, { - "title": "PostgreSQL", - "link": "/language/dialect/postgres.malloynb" + "title": "Ordering and Limiting", + "link": "/language/order_by.malloynb" } ] + }, + { + "title": "Tags", + "link": "/language/tags.malloynb" + }, + { + "title": "Change Log", + "link": "/language/changelog.malloynb" } ] }, { "title": "Common Patterns", "items": [ + { + "title": "Patterns Overview", + "link": "/patterns/index.malloynb" + }, { "title": "Comparing Timeframes", "link": "/patterns/yoy.malloynb" @@ -198,10 +187,6 @@ "title": "Moving Average", "link": "/patterns/moving_avg.malloynb" }, - { - "title": "Transform Data", - "link": "/patterns/transform.malloynb" - }, { "title": "Sessionize - Map/Reduce", "link": "/patterns/sessionize.malloynb" @@ -324,10 +309,6 @@ "title": "SQL Expressions", "link": "/experiments/sql_expressions.malloynb" }, - { - "title": "Window Partitions", - "link": "/experiments/window.malloynb" - }, { "title": "Parameters", "link": "/experiments/parameters.malloynb" From b03a21debbde783eb84d10803277d23a2c0f11d4 Mon Sep 17 00:00:00 2001 From: James Swirhun Date: Sun, 30 Nov 2025 14:14:59 -0700 Subject: [PATCH 2/3] Fix broken links to renamed documentation files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - sql_sources.malloynb → sql_blocks.malloynb - imports.malloynb → statement.malloynb#import-statements - fields.malloynb → dimensions.malloynb --- src/blog/2023-10-03-malloy-four/index.malloynb | 6 +++--- src/documentation/user_guides/malloy_by_example.malloynb | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/src/blog/2023-10-03-malloy-four/index.malloynb b/src/blog/2023-10-03-malloy-four/index.malloynb index 66101cf3..a361ec4c 100644 --- a/src/blog/2023-10-03-malloy-four/index.malloynb +++ b/src/blog/2023-10-03-malloy-four/index.malloynb @@ -51,11 +51,11 @@ the [4.0 Messages FAQ](../../documentation/language/m4warnings.malloynb) documen * The `declare:` statement has been removed, use `dimension:` or `measure:` * `join_:` statements in a query must be in the `extend: {}` block * The new [run statement](../../documentation/language/statement.malloynb#run-statements), `run:` replaces `query:` with no name -* The `sql:` statement is replaced by [SQL sources](../../documentation/language/sql_sources.malloynb), e.g. `duckdb.sql("SELECT ...")` +* The `sql:` statement is replaced by [SQL blocks](../../documentation/language/sql_blocks.malloynb), e.g. `duckdb.sql("SELECT ...")` * [Table sources](../../documentation/language/source.malloynb#sources-from-tables-or-views) have new syntax: `connection_name.table('table_path')` instead of `table('connection_name:table_path')` * [Projections](../../documentation/language/views.malloynb#projection) are performed with `select:` instead of `project:` * The `from()` function to create a source from a query is no longer needed -* When [nesting Malloy code inside a SQL string](../../documentation/language/sql_sources.malloynb#embedding-malloy-queries-in-an-sql-block-a-k-a-turducken-), the syntax +* When [nesting Malloy code inside a SQL string](../../documentation/language/sql_blocks.malloynb#embedding-malloy-queries-in-an-sql-block--turducken-), the syntax is `%{ <> }` and not `%{ <> }%` * [Explicit aggregate locality](../../documentation/language/aggregates.malloynb#required-explicit-aggregate-locality) (using `source.`) is now required in some cases for `sum()` and `avg()` * The filter shortcut `{? }` has been removed @@ -79,6 +79,6 @@ these will continue to work. They are listed here to provide a complete record o * New [safe cast operator](../../documentation/language/expressions.malloynb#safe-type-cast) `:::` * Addition of [annotations and tags](../../documentation/language/tags.malloynb) * New [null-coalescing operator](../../documentation/language/expressions.malloynb#null-operations) `??` -* [Selective imports](../../documentation/language/imports.malloynb#selective-imports) of objects from other files +* [Selective imports](../../documentation/language/statement.malloynb#selective-imports) of objects from other files * [Analytic/window functions](../../documentation/language/calculations_windows.malloynb) using `calculate:` >>>markdown diff --git a/src/documentation/user_guides/malloy_by_example.malloynb b/src/documentation/user_guides/malloy_by_example.malloynb index d19fb112..3c8d2726 100644 --- a/src/documentation/user_guides/malloy_by_example.malloynb +++ b/src/documentation/user_guides/malloy_by_example.malloynb @@ -59,7 +59,7 @@ run: duckdb.table('../data/airports.parquet') -> { Malloy separates a query's view from the source of the data. A source can be thought of as a table and a collection of computations and relationships which are relevant to that table. ([Source Documentation](../language/source.malloynb)). -[Fields](../language/fields.malloynb) can be defined as part of a source. +[Dimensions](../language/dimensions.malloynb) can be defined as part of a source. * A `measure:` is a declared aggregate calculation (think function that operates across the table) which can be used in `aggregate:` elements in a query stage @@ -559,7 +559,7 @@ run: airport_facts -> flights_by_origin ### SQL Sources -See the [SQL Sources](../language/sql_sources.malloynb) section. +See the [SQL Blocks](../language/sql_blocks.malloynb) section. ### Embedding Malloy queries in SQL ([SQL Block Documentation](../language/sql_blocks.malloynb#embedding-malloy-queries-in-an-sql-block-turducken-)) @@ -571,6 +571,6 @@ See the [SQL Sources](../language/sql_sources.malloynb) section. ### Nested data and Symmetric aggregates ([Aggregates Documentation](../language/aggregates.malloynb)) -### Import ([Import Documentation](../language/imports.malloynb)) +### Import ([Import Documentation](../language/statement.malloynb#import-statements)) ### Data styles and rendering ([Rendering Documentation](../visualizations/dashboards.malloynb)) From 14879e62675782c3b2dbc89d278d65e949efe741 Mon Sep 17 00:00:00 2001 From: James Swirhun Date: Sun, 30 Nov 2025 14:28:32 -0700 Subject: [PATCH 3/3] DCO Remediation Commit for James Swirhun I, James Swirhun , hereby add my Signed-off-by to this commit: 0730c32ae63e2a5ac5e2c73670f6a5393c6c0e62 I, James Swirhun , hereby add my Signed-off-by to this commit: b03a21debbde783eb84d10803277d23a2c0f11d4 Signed-off-by: James Swirhun