rowwise(): Detect and reject non-atomic, non-list column values with clear error message #7250

venom1204 · 2025-08-12T21:17:07Z

This PR enhances rowwise() to detect all non-atomic, non-list column values (e.g., language objects, expressions, pairlists, environments, S4 objects) instead of only functions. It adds a clear error message with the offending column name and type, plus guidance to wrap values in list(...) if intended. Includes updated tests for both allowed and rejected cases.

hi @tdhock @MichaelChirico @aitap can you please have a look when you got time .

thanks.

codecov · 2025-08-12T21:28:25Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.02%. Comparing base (0912a66) to head (8f1bd4f).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #7250   +/-   ##
=======================================
  Coverage   99.02%   99.02%           
=======================================
  Files          87       87           
  Lines       16791    16798    +7     
=======================================
+ Hits        16628    16635    +7     
  Misses        163      163

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-08-12T21:35:04Z

HEAD=issue_7219 stopped early for DT[by,verbose=TRUE] improved in #6296

Generated via commit 1371a79

Download link for the artifact containing the test results: ↓ atime-results.zip

Task	Duration
R setup and installing dependencies	5 minutes and 16 seconds
Installing different package versions	11 minutes and 17 seconds
Running and plotting the test cases	4 minutes and 8 seconds

tdhock · 2025-08-14T12:06:10Z

R/rowwiseDT.R

  nrows = length(body) %/% ncols
  if (length(body) != nrows * ncols)
    stopf("There are %d columns but the number of cells is %d, which is not an integer multiple of the columns", ncols, length(body))
+


please undo addition of empty lines

tdhock

please fix

…into issue_7219

venom1204 · 2025-08-24T11:32:36Z

hi @tdhock I did the modifications can you please have a look when you got time.
thanks

ben-schwen · 2025-08-25T06:50:28Z

R/rowwiseDT.R

    stopf("There are %d columns but the number of cells is %d, which is not an integer multiple of the columns", ncols, length(body))
+  is_problematic = vapply(
+    body,
+    function(v) !is.atomic(v) && !is.null(v) && typeof(v) != "list",


is this check really what @aitap wrote about catering for that is.atomic(NULL)=TRUE on older R versions?

it should also be faster to check !(is.atomic(v) || is.null(v) || typeof(v) == "list")

Note that we have our internal versions of vapply e.g. vapply_1b

…into issue_7219

ben-schwen · 2025-08-26T08:10:37Z

R/rowwiseDT.R

+    first_problem_idx = which(is_problematic)[1L]
+    col_idx = (first_problem_idx - 1L) %% ncols + 1L
+    col_name = header[col_idx]
+    obj_type = typeof(body[[first_problem_idx]])


Not sure if typeof is the right choice here. For the problem raised in #7219 class would be a better choice since class would be "function" but typeof would be still "closure".

ben-schwen · 2025-08-26T08:16:00Z

R/rowwiseDT.R

+    col_name = header[col_idx]
+    obj_type = typeof(body[[first_problem_idx]])
+    stopf(
+      "In column '%s', received an object of type '%s'.\nComplex objects (like functions, models, etc.) must be wrapped in list() to be stored in a data.table column.\nPlease use `list(...)` for this value.",


Is this really true? Many models are indeed lists, e.g. lm(mpg~., data=mtcars).

Other thing on top of my head would be expressions which we do not fully support. Storing environments into a data.table seems uncommon to me.

ben-schwen

We also still miss a NEWS item!

…o issue_7219

ben-schwen · 2026-01-04T17:06:02Z

R/rowwiseDT.R

+    col_name = header[col_idx]
+    obj_type = class1(body[[first_problem_idx]])
+    stopf(
+      "In column '%s', received an object of type '%s'.\nComplex objects (like functions, formulas, or calls) must be wrapped in list() to be stored in a data.table column.\nPlease use `list(...)` for this value.",


The indent is off, see e.g.

> rowwiseDT(x =, func =, 1, \(x) x + 1) Error in rowwiseDT(x = , func = , 1, function(x) x + 1) : In column 'func', received an object of type 'function'. Complex objects (like functions, formulas, or calls) must be wrapped in list() to be stored in a data.table column. Please use `list(...)` for this value.

Maybe also something more in the direction of

stopf("Column '%s' is type '%s'. Non-atomic, non-list objects must be wrapped in list(), e.g., list(f) instead of f", col_name, obj_type)

ben-schwen · 2026-01-04T17:09:31Z

R/rowwiseDT.R

+    first_problem_idx = which(is_problematic)[1L]
+    col_idx = (first_problem_idx - 1L) %% ncols + 1L
+    col_name = header[col_idx]
+    obj_type = class1(body[[first_problem_idx]])


Suggested change

first_problem_idx = which(is_problematic)[1L]

col_idx = (first_problem_idx - 1L) %% ncols + 1L

col_name = header[col_idx]

obj_type = class1(body[[first_problem_idx]])

idx = which(is_problematic)[1L]

col_idx = (idx - 1L) %% ncols + 1L

col_name = header[col_idx]

obj_type = class1(body[[idx]])

Would feel more consistent to our codebase/

ben-schwen · 2026-01-04T17:09:43Z

NEWS.md


 19. Ellipsis elements like `..1` are correctly excluded when searching for variables in "up-a-level" syntax inside `[`, [#5460](https://github.com/Rdatatable/data.table/issues/5460). Thanks @ggrothendieck for the report and @MichaelChirico for the fix.

+20. `rowwiseDT()` now provides a helpful error message when a complex object that is not a list (e.g., a function) is provided as a cell value, instructing the user to wrap it in `list()`. [#7219](https://github.com/Rdatatable/data.table/issues/7219). Thanks @kylebutts for the report and @venom1204 for the fix.


Suggested change

20. `rowwiseDT()` now provides a helpful error message when a complex object that is not a list (e.g., a function) is provided as a cell value, instructing the user to wrap it in `list()`. [#7219](https://github.com/Rdatatable/data.table/issues/7219). Thanks @kylebutts for the report and @venom1204 for the fix.

20. `rowwiseDT()` now provides a helpful error message when a complex object that is not a list (e.g., a function) is provided as a cell value, instructing the user to wrap it in `list()`, [#7219](https://github.com/Rdatatable/data.table/issues/7219). Thanks @kylebutts for the report and @venom1204 for the fix.

Ofc, it must also be moved to the 1.18.99 NEWS section

ben-schwen

Sorry for the late review. Some minor things, but overall close to finish.

…o issue_7219

venom1204 · 2026-01-11T05:29:39Z

Sorry for the late review. Some minor things, but overall close to finish.

hi @ben-schwen
sorry for the late replies, was engaged somewhere else .
Have a look and let me know if any changes are needed.

R/rowwiseDT.R

ben-schwen · 2026-01-11T16:19:11Z

TY. Good work!

added check

aaa09aa

venom1204 requested a review from MichaelChirico as a code owner August 12, 2025 21:17

..

2110d98

Merge branch 'master' into issue_7219

6b271bb

tdhock reviewed Aug 14, 2025

View reviewed changes

tdhock requested changes Aug 14, 2025

View reviewed changes

venom1204 added 3 commits August 15, 2025 07:04

removed empty lines

2f777d5

Merge branch 'master' into issue_7219

d19bc2a

Merge branch 'issue_7219' of https://github.com/Rdatatable/data.table …

dd36cf7

…into issue_7219

venom1204 requested a review from tdhock August 15, 2025 07:06

Merge branch 'master' into issue_7219

08fe92e

ben-schwen reviewed Aug 25, 2025

View reviewed changes

venom1204 added 2 commits August 25, 2025 09:48

vapply_1b

669f6ad

Merge branch 'issue_7219' of https://github.com/Rdatatable/data.table …

3ece463

…into issue_7219

venom1204 requested a review from ben-schwen August 25, 2025 10:01

ben-schwen reviewed Aug 26, 2025

View reviewed changes

venom1204 added 3 commits August 30, 2025 10:43

type-> class

e2de9c0

Merge branch 'master' of https://github.com/Rdatatable/data.table int…

dddcedb

…o issue_7219

lintr + error

dafc34f

venom1204 requested a review from ben-schwen August 30, 2025 11:43

ben-schwen reviewed Jan 4, 2026

View reviewed changes

ben-schwen requested changes Jan 4, 2026

View reviewed changes

Merge branch 'master' of https://github.com/Rdatatable/data.table int…

6cb20c8

…o issue_7219

venom1204 added 2 commits January 11, 2026 05:03

updated things

f3e484b

updated news

1371a79

ben-schwen reviewed Jan 11, 2026

View reviewed changes

R/rowwiseDT.R Outdated Show resolved Hide resolved

Update R/rowwiseDT.R

8f1bd4f

ben-schwen merged commit fd2d84a into master Jan 11, 2026
9 of 10 checks passed

ben-schwen deleted the issue_7219 branch January 11, 2026 16:18


		19. Ellipsis elements like `..1` are correctly excluded when searching for variables in "up-a-level" syntax inside `[`, [#5460](https://github.com/Rdatatable/data.table/issues/5460). Thanks @ggrothendieck for the report and @MichaelChirico for the fix.

		20. `rowwiseDT()` now provides a helpful error message when a complex object that is not a list (e.g., a function) is provided as a cell value, instructing the user to wrap it in `list()`. [#7219](https://github.com/Rdatatable/data.table/issues/7219). Thanks @kylebutts for the report and @venom1204 for the fix.

rowwise(): Detect and reject non-atomic, non-list column values with clear error message #7250

rowwise(): Detect and reject non-atomic, non-list column values with clear error message #7250

Conversation

venom1204 commented Aug 12, 2025

Uh oh!

codecov bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tdhock left a comment

Choose a reason for hiding this comment

Uh oh!

venom1204 commented Aug 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ben-schwen left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ben-schwen Jan 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ben-schwen left a comment

Choose a reason for hiding this comment

Uh oh!

venom1204 commented Jan 11, 2026

Uh oh!

Uh oh!

Uh oh!

ben-schwen commented Jan 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Aug 12, 2025 •

edited

Loading

github-actions bot commented Aug 12, 2025 •

edited

Loading

ben-schwen left a comment •

edited

Loading

ben-schwen Jan 4, 2026 •

edited

Loading