Skip to content

Conversation

neutrinoceros
Copy link
Member

@neutrinoceros neutrinoceros commented Jul 23, 2025

PR Summary

Rationale:
I find the mandatory signature of derived field functions counter-intuitive and harmful: the field argument is rarely used in user-defined functions, but it's still the first argument on the mandatory signature, so I regularly find myself defining stuff as

def myfield(data, field):
    ...

which is currently a footgun, and the motivation for #4028
I would love this to be actually valid, and ideally, the field argument, if unused, should be optional. However, for a long time I couldn't see a backward-compatible path forward, so I abstained from proposing this change.
However, I recently came to the realization that it should actually be possible to allow this without breaking backward compatibility, on the condition that we stop passing data and field arguments positionally within yt, which is the starting point of the PR. Provided nothing explodes in CI, I'll work on making user field validation more flexible, as promised in the title.

PR Checklist

  • New features are documented, with docstrings and narrative docs
  • Adds a test for any bugs fixed. Adds tests for new features.

@neutrinoceros neutrinoceros added enhancement Making something better api-consistency naming conventions, code deduplication, informative error messages, code smells... labels Jul 23, 2025
@neutrinoceros neutrinoceros force-pushed the enh/allow_missing_field_arg branch 7 times, most recently from 1263328 to eb0d059 Compare July 23, 2025 14:15
@neutrinoceros
Copy link
Member Author

Almost there. I note that this comes with an extremely minor breaking change: the following signatures, which work as of yt 4.4.1, are not accepted anymore

def field(field, /, data):
    ...
    
def bar(field, data, /):
    ...

though I seriously doubt anyone would actually use them in the wild, there is one reasonable case where supportin keyword-argument cannot be done directly, i.e., if the field function is defined in native code. But even if this exceedingly unlikely scenario, the robust solution is trivial: wrap your native function inside a Python one that does nothing else than take field and data as keyword and pass them down as positionals.

@neutrinoceros neutrinoceros force-pushed the enh/allow_missing_field_arg branch from 5cf8f6d to 708ae96 Compare July 23, 2025 15:13
@neutrinoceros neutrinoceros added this to the 4.5.0 milestone Jul 23, 2025
@neutrinoceros
Copy link
Member Author

the bulk of the diff now is in the last few commits where I removed the field parameters everywhere it wasn't used. I'm happy to split this into its own follow up PR if it helps reviewing.

@neutrinoceros neutrinoceros force-pushed the enh/allow_missing_field_arg branch from 6613fb4 to 2f77dec Compare July 23, 2025 17:12
@neutrinoceros neutrinoceros marked this pull request as ready for review July 23, 2025 17:37
@neutrinoceros neutrinoceros changed the title ENH: allow missing field argument in derived field functions ENH: allow omitting the field argument in derived field functions Jul 23, 2025
@jzuhone
Copy link
Contributor

jzuhone commented Jul 24, 2025

Can someone remind me why we originally had field in the field function signature? What was it used for? (@matthewturk @brittonsmith)

@neutrinoceros
Copy link
Member Author

it's used in some places to retrieve field.units and use it in the return value. You can find all remaining use cases by searching for (field, data) within this branch !

@matthewturk
Copy link
Member

I think we also used it, maybe previously to get the field name to consolidate things like species fractions.

@cphyc
Copy link
Member

cphyc commented Jul 26, 2025

It's also used when defining fields for particles, where one doesn't want to hardcode which particle type the field is operating on.

@matthewturk matthewturk mentioned this pull request Jul 31, 2025
2 tasks
@brittonsmith
Copy link
Member

Maybe I'm restating what @matthewturk said, but it's used frequently for generic field functions that change behavior based on the specific field it's being used for. This is usually done with field.name.

@neutrinoceros
Copy link
Member Author

I definitely has some use cases, but is there a consensus that it's not always needed, and as such, should neither be mandatory nor have to be the first argument ?

@brittonsmith
Copy link
Member

I would go so far as to say it is usually not needed. From a user perspective, this is a friendly change. As long we retain support for the field, data function signature, I think this is a net positive. I would also feel a lot better if accepting field wasn't totally eliminated from the documentation. When this is needed, it's very useful and we run the risk of losing this knowledge.

@neutrinoceros
Copy link
Member Author

I just removed it everywhere it was not used, but I don't think I eliminated it. The diff obviously doesn't show every example I left alone :)

@neutrinoceros
Copy link
Member Author

(Or if I did, it means we don't have any actual, good example showing why this argument is sometimes useful)

@brittonsmith
Copy link
Member

I've checked and there is no mention in the creating derived fields section of the documentation. There was no mention previously in the narrative docs, but the examples showed it, so the user would at least know it had to be there even if they didn't know why. That was an oversight that I would argue we were getting away with because one could poke around the source to look for example usage. However, now one would not know to look for this and may only be further confused by the occasional appearance of field functions with the field, data signature. In short, the prior situation was suboptimal but it maintained a connection to knowledge (and functionality) we will lose if we do not add mention of it.

@neutrinoceros
Copy link
Member Author

Thanks for checking ! I'll add a couple examples for it then.

jzuhone
jzuhone previously approved these changes Aug 21, 2025
@neutrinoceros
Copy link
Member Author

Whoops, looks like I forgot about this for a while. I just added the promised examples. Let me know what you think, @brittonsmith !

Copy link
Member

@brittonsmith brittonsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@neutrinoceros I like the doc updates for discussing the field argument. I'm happy with all this. This is very nice.

@jzuhone jzuhone merged commit d909952 into yt-project:main Sep 12, 2025
12 of 14 checks passed
@neutrinoceros neutrinoceros deleted the enh/allow_missing_field_arg branch September 12, 2025 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-consistency naming conventions, code deduplication, informative error messages, code smells... docs enhancement Making something better
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants