From a3cd084b7b0ce195e748f9ad35ef2255547bbe9c Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Fri, 20 Sep 2024 18:47:12 -0700 Subject: [PATCH 01/34] Declarative `macro_rules!` derive macros --- text/3698-declarative-derive-macros.md | 110 +++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 text/3698-declarative-derive-macros.md diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md new file mode 100644 index 00000000000..e11454083cb --- /dev/null +++ b/text/3698-declarative-derive-macros.md @@ -0,0 +1,110 @@ +- Feature Name: `declarative_derive_macros` +- Start Date: 2024-09-20 +- RFC PR: [rust-lang/rfcs#3698](https://github.com/rust-lang/rfcs/pull/3698) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Support implementing `derive(Trait)` via a `macro_rules!` macro. + +# Motivation +[motivation]: #motivation + +Many crates support deriving their traits with `derive(Trait)`. Today, this +requires defining proc macros, in a separate crate, typically with several +additional dependencies adding substantial compilation time, and typically +guarded by a feature that users need to remember to enable. + +However, many common cases of derives don't require any more power than an +ordinary `macro_rules!` macro. Supporting these common cases would allow many +crates to avoid defining proc macros, reduce dependencies and compilation time, +and provide these macros unconditionally without requiring the user to enable a +feature. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +You can define a macro to implement `derive(MyTrait)` by defining a +`macro_rules!` macro with the `#[macro_derive]` attribute. Such a macro can +create new items based on a struct, enum, or union. Note that the macro can +only append new items; it cannot modify the item it was applied to. + +For instance, the following macro will ignore the item it is attached to, and +append a function `answer()`: + +```rust +#[macro_derive] +macro_rules! AnswerFn { + ($_:tt) => { fn answer() -> u32 { 42 } }; +} + +#[derive(AnswerFn)] +struct Struct; + +fn main() { + assert_eq!(42, answer()); +} +``` + +Derive macros defined using `macro_rules!` follow the same scoping rules as +any other macro, and may be invoked by any path that resolves to them. + +A derive macro may share the same path as a trait of the same name. For +instance, the name `mycrate::MyTrait` can refer to both the `MyTrait` trait and +the macro for `derive(MyTrait)`. This is consistent with existing derive +macros. + +A derive macro may also define *helper attributes*. These attributes are +[inert](https://doc.rust-lang.org/reference/attributes.html#active-and-inert-attributes), +and their only purpose is to be fed into the derive macro that defined them. +That said, they can be seen by all macros. + +To define helper attributes, put an attributes key in the `macro_derive` +attribute, with a comma-separated list of identifiers for helper attributes: +`#[macro_derive(attributes(helper))]`. The derive macro can process the +`#[helper]` attribute, along with any arguments to it, as part of the item the +derive macro was applied to. + +If a derive macro mistakenly emits the token stream it was applied to +(resulting in a duplicate item definition), the error the compiler emits for +the duplicate item should hint to the user that the macro was defined +incorrectly, and remind the user that derive macros only append new items. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +Adding this feature will allow many crates in the ecosystem to drop their proc +macro crates and corresponding dependencies, and decrease their build times. + +Crates could instead define `macro_rules!` macros and encourage users to invoke +them using existing syntax like `macroname! { ... }`, rather than using +derives. This would provide the same functionality, but would not support the +same syntax people are accustomed to, and could not maintain semver +compatibility with an existing proc-macro-based derive. In addition, this would +not preserve the property derive macros normally have that they cannot change +the item they are applied to. + +A mechanism to define attribute macros would let people write attributes like +`#[derive_mytrait]`, but that would not provide compatibility with existing +derive syntax. + +We could allow `macro_rules!` derive macros to emit a replacement token stream; +however, that would be inconsistent with the restriction preventing proc macros +from doing the same. + +# Prior art +[prior-art]: #prior-art + +We have had proc-macro-based derive macros for a long time, and the ecosystem +makes extensive use of them. + +The [`macro_rules_attribute`](https://crates.io/crates/macro_rules_attribute) +crate defines proc macros that allow invoking declarative macros as derives, +demonstrating a demand for this. This feature would allow defining such derives +without requiring proc macros at all, and would support the same invocation +syntax as a proc macro. + +The `macro_derive` attribute and its `attributes` syntax are based on the +[existing `proc_macro_derive` attribute for proc +macros](https://doc.rust-lang.org/reference/procedural-macros.html#derive-macros). From 567411b0c9ae6f03fdd79be8c001387955e47189 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sat, 21 Sep 2024 01:15:37 -0700 Subject: [PATCH 02/34] Give a more realistic example --- text/3698-declarative-derive-macros.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index e11454083cb..6b1ae40c4d5 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -30,20 +30,27 @@ You can define a macro to implement `derive(MyTrait)` by defining a create new items based on a struct, enum, or union. Note that the macro can only append new items; it cannot modify the item it was applied to. -For instance, the following macro will ignore the item it is attached to, and -append a function `answer()`: +For example: ```rust +trait Answer { fn answer(&self) -> u32; } + #[macro_derive] -macro_rules! AnswerFn { - ($_:tt) => { fn answer() -> u32 { 42 } }; +macro_rules! Answer { + // Simplified for this example + (struct $n:ident $_:tt) => { + impl Answer for $n { + fn answer(&self) -> u32 { 42 } + } + }; } -#[derive(AnswerFn)] +#[derive(Answer)] struct Struct; fn main() { - assert_eq!(42, answer()); + let s = Struct; + assert_eq!(42, s.answer()); } ``` From f3f5de92076e6da49ad37e8bc9ea736b8a8d309c Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sat, 21 Sep 2024 14:44:25 -0700 Subject: [PATCH 03/34] Add alternative about allowing direct invocation --- text/3698-declarative-derive-macros.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 6b1ae40c4d5..45fabbc3e6d 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -100,6 +100,11 @@ We could allow `macro_rules!` derive macros to emit a replacement token stream; however, that would be inconsistent with the restriction preventing proc macros from doing the same. +We could allow directly invoking a `macro_rules!` derive macro as a +function-like macro. This has the potential for confusion, given the +append-only nature of derive macros versus the behavior of normal function-like +macros. It might potentially be useful for code reuse, however. + # Prior art [prior-art]: #prior-art From 65b1053600129cafd1e6c4e17554042ff9b2308b Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 24 Sep 2024 09:39:53 -0700 Subject: [PATCH 04/34] Mention `$crate` --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 45fabbc3e6d..32134beb43a 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -84,6 +84,10 @@ incorrectly, and remind the user that derive macros only append new items. Adding this feature will allow many crates in the ecosystem to drop their proc macro crates and corresponding dependencies, and decrease their build times. +This will also give derive macros access to the `$crate` mechanism to refer to +the defining crate, which is simpler than mechanisms currently used in proc +macros to achieve the same goal. + Crates could instead define `macro_rules!` macros and encourage users to invoke them using existing syntax like `macroname! { ... }`, rather than using derives. This would provide the same functionality, but would not support the From 923325fa3f4214d473453e0d419beca60689cd9a Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 24 Sep 2024 09:41:53 -0700 Subject: [PATCH 05/34] Caching --- text/3698-declarative-derive-macros.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 32134beb43a..4dda7a4d63f 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -88,6 +88,9 @@ This will also give derive macros access to the `$crate` mechanism to refer to the defining crate, which is simpler than mechanisms currently used in proc macros to achieve the same goal. +Macros defined this way can more easily support caching, as they cannot depend +on arbitrary unspecified inputs. + Crates could instead define `macro_rules!` macros and encourage users to invoke them using existing syntax like `macroname! { ... }`, rather than using derives. This would provide the same functionality, but would not support the From ba775482bbdb7308a86006c72c56fe0912a10994 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 24 Sep 2024 09:46:53 -0700 Subject: [PATCH 06/34] Clarify text about helper attributes --- text/3698-declarative-derive-macros.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 4dda7a4d63f..5f5e4aedc72 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -64,8 +64,9 @@ macros. A derive macro may also define *helper attributes*. These attributes are [inert](https://doc.rust-lang.org/reference/attributes.html#active-and-inert-attributes), -and their only purpose is to be fed into the derive macro that defined them. -That said, they can be seen by all macros. +and exist for the derive macro to parse and act upon. Note that +they're visible to *all* macros, not just the one that defined them; macros +should ignore any attributes not meant for them. To define helper attributes, put an attributes key in the `macro_derive` attribute, with a comma-separated list of identifiers for helper attributes: From 8881b6837bab27993e8dfb5a1984f0b81aebd3a1 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 25 Sep 2024 20:38:37 -0700 Subject: [PATCH 07/34] Future possibilities: Better error reporting --- text/3698-declarative-derive-macros.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 5f5e4aedc72..b73c6ba6739 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -128,3 +128,9 @@ syntax as a proc macro. The `macro_derive` attribute and its `attributes` syntax are based on the [existing `proc_macro_derive` attribute for proc macros](https://doc.rust-lang.org/reference/procedural-macros.html#derive-macros). + +# Future possibilities +[future-possibilities]: #future-possibilities + +We should provide a way for `macro_rules!` macros to provide better error +reporting, with spans, rather than just pointing to the macro. From f195edf9be243ce7f407f32bb7e1cefdd9c46d08 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 25 Sep 2024 20:42:06 -0700 Subject: [PATCH 08/34] Mention automatically_derived --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index b73c6ba6739..6e4b6e44c20 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -74,6 +74,10 @@ attribute, with a comma-separated list of identifiers for helper attributes: `#[helper]` attribute, along with any arguments to it, as part of the item the derive macro was applied to. +If a derive macro emits a trait impl for the type, it may want to add the +[`#[automatically_derived]`](https://doc.rust-lang.org/reference/attributes/derive.html#the-automatically_derived-attribute) +attribute, for the benefit of diagnostics. + If a derive macro mistakenly emits the token stream it was applied to (resulting in a duplicate item definition), the error the compiler emits for the duplicate item should hint to the user that the macro was defined From c4b185b90f016fab2cf159364ddf7932db14976b Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 25 Sep 2024 20:43:41 -0700 Subject: [PATCH 09/34] Future possibilities: Error recovery --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 6e4b6e44c20..2d782eb29ce 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -138,3 +138,7 @@ macros](https://doc.rust-lang.org/reference/procedural-macros.html#derive-macros We should provide a way for `macro_rules!` macros to provide better error reporting, with spans, rather than just pointing to the macro. + +We may want to support error recovery, so that a derive can produce an error +but still provide enough for the remainder of the compilation to proceed far +enough to usefully report further errors. From 7bcdf3b9f1fcf12cf82c9e0150f32f521d23716f Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 30 Sep 2024 13:22:14 -0700 Subject: [PATCH 10/34] Add drawbacks section mentioning impact on crate maintainers --- text/3698-declarative-derive-macros.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 2d782eb29ce..8e8379b82be 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -83,6 +83,20 @@ If a derive macro mistakenly emits the token stream it was applied to the duplicate item should hint to the user that the macro was defined incorrectly, and remind the user that derive macros only append new items. +# Drawbacks +[drawbacks]: #drawbacks + +This feature will not be sufficient for *all* uses of proc macros in the +ecosystem, and its existence may create social pressure for crate maintainers +to switch even if the result is harder to maintain. + +Before stabilizing this feature, we should receive feedback from crate +maintainers, and potentially make further improvements to `macro_rules` to make +it easier to use for their use cases. This feature will provide motivation to +evaluate many new use cases that previously weren't written using +`macro_rules`, and we should consider quality-of-life improvements to better +support those use cases. + # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From 7e1d5171fd165e4c4fb90fc0fbffff448ebf95b3 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 30 Sep 2024 13:38:40 -0700 Subject: [PATCH 11/34] Expand future work --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 8e8379b82be..9a115abcb08 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -156,3 +156,7 @@ reporting, with spans, rather than just pointing to the macro. We may want to support error recovery, so that a derive can produce an error but still provide enough for the remainder of the compilation to proceed far enough to usefully report further errors. + +As people test this feature and run into limitations of `macro_rules!` parsing, +we should consider additional features to make this easier to use for various +use cases. From bf67d21413138caa0a6087b11c1936c345825af3 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 30 Sep 2024 14:02:26 -0700 Subject: [PATCH 12/34] Future work: Namespacing helper attributes --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 9a115abcb08..a8c9e1f4ecc 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -160,3 +160,7 @@ enough to usefully report further errors. As people test this feature and run into limitations of `macro_rules!` parsing, we should consider additional features to make this easier to use for various use cases. + +We may want to provide a means to namespace helper attributes or detect +collisions between them. This would apply to both proc macros and +`macro_rules!` macros. From 32d91b67a23ff98b5356a031d79ce8a87fb2071e Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 30 Sep 2024 14:04:10 -0700 Subject: [PATCH 13/34] Future work: helpers for `where` bounds --- text/3698-declarative-derive-macros.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index a8c9e1f4ecc..6459233ac59 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -164,3 +164,6 @@ use cases. We may want to provide a means to namespace helper attributes or detect collisions between them. This would apply to both proc macros and `macro_rules!` macros. + +We may want to provide simple helpers for generating/propagating `where` +bounds, which would otherwise be complex to do in a `macro_rules!` macro. From 9faa8f41217c271daa5bbb1441a3f3e4a973c327 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 30 Sep 2024 14:15:58 -0700 Subject: [PATCH 14/34] Add unresolved question about avoid cascading errors due to missing impls --- text/3698-declarative-derive-macros.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 6459233ac59..30adf564925 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -147,6 +147,14 @@ The `macro_derive` attribute and its `attributes` syntax are based on the [existing `proc_macro_derive` attribute for proc macros](https://doc.rust-lang.org/reference/procedural-macros.html#derive-macros). +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +Before stabilizing this feature, we should ensure there's a mechanism macros +can use to ensure that an error when producing an impl does not result in a +cascade of additional errors caused by a missing impl. This may take the form +of a fallback impl, for instance. + # Future possibilities [future-possibilities]: #future-possibilities From 5ee8fe04dab8b288f999d0996b51ddb488c54ee3 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 2 Oct 2024 10:13:06 -0700 Subject: [PATCH 15/34] Add unresolved question to make sure we don't have awful error messages --- text/3698-declarative-derive-macros.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 30adf564925..0024e4cd3a2 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -155,6 +155,9 @@ can use to ensure that an error when producing an impl does not result in a cascade of additional errors caused by a missing impl. This may take the form of a fallback impl, for instance. +Before stabilizing this feature, we should make sure it doesn't produce wildly +worse error messages in common cases. + # Future possibilities [future-possibilities]: #future-possibilities From 8352b1c725cb6893bc83fc649fef564296ba4356 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 2 Oct 2024 14:05:16 -0700 Subject: [PATCH 16/34] Future work: helpers for higher-level concepts like struct fields --- text/3698-declarative-derive-macros.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 0024e4cd3a2..b2dbecd25b2 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -176,5 +176,12 @@ We may want to provide a means to namespace helper attributes or detect collisions between them. This would apply to both proc macros and `macro_rules!` macros. +We could provide a maco matcher to match an entire struct field, along with +syntax (based on macro metavariable expressions) to extract the field name or +type (e.g. `${f.name}`). This would simplify many common cases by leveraging +the compiler's own parser. + +We could do the same for various other high-level constructs. + We may want to provide simple helpers for generating/propagating `where` bounds, which would otherwise be complex to do in a `macro_rules!` macro. From f797596f54190f8cfc3add09045c094c9ad0119c Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 21 Oct 2024 14:30:01 +0800 Subject: [PATCH 17/34] Add further steps about averting pressure on crate maintainers --- text/3698-declarative-derive-macros.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index b2dbecd25b2..d095c264823 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -88,7 +88,9 @@ incorrectly, and remind the user that derive macros only append new items. This feature will not be sufficient for *all* uses of proc macros in the ecosystem, and its existence may create social pressure for crate maintainers -to switch even if the result is harder to maintain. +to switch even if the result is harder to maintain. We can and should attempt +to avert and such pressure, such as by providing a post with guidance that +crate maintainers can link to when responding to such requests. Before stabilizing this feature, we should receive feedback from crate maintainers, and potentially make further improvements to `macro_rules` to make @@ -158,6 +160,11 @@ of a fallback impl, for instance. Before stabilizing this feature, we should make sure it doesn't produce wildly worse error messages in common cases. +Before stabilizing this feature, we should have clear public guidance +recommending against pressuring crate maintainers to adopt this feature +rapidly, and encourage crate maintainers to link to that guidance if such +requests arise. + # Future possibilities [future-possibilities]: #future-possibilities From 3526d7fd4b2fac1c3a325500317a92e70e3f7a46 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Mon, 21 Oct 2024 23:32:42 +0800 Subject: [PATCH 18/34] Copy a drawback to the unresolved questions section --- text/3698-declarative-derive-macros.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index d095c264823..6f56c323fa7 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -160,6 +160,13 @@ of a fallback impl, for instance. Before stabilizing this feature, we should make sure it doesn't produce wildly worse error messages in common cases. +Before stabilizing this feature, we should receive feedback from crate +maintainers, and potentially make further improvements to `macro_rules` to make +it easier to use for their use cases. This feature will provide motivation to +evaluate many new use cases that previously weren't written using +`macro_rules`, and we should consider quality-of-life improvements to better +support those use cases. + Before stabilizing this feature, we should have clear public guidance recommending against pressuring crate maintainers to adopt this feature rapidly, and encourage crate maintainers to link to that guidance if such From ba7effc36c22288ce508b54604e5dd889e7a1773 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 22 Oct 2024 05:55:32 -0700 Subject: [PATCH 19/34] Fix typo --- text/3698-declarative-derive-macros.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 6f56c323fa7..074f80b5954 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -190,7 +190,7 @@ We may want to provide a means to namespace helper attributes or detect collisions between them. This would apply to both proc macros and `macro_rules!` macros. -We could provide a maco matcher to match an entire struct field, along with +We could provide a macro matcher to match an entire struct field, along with syntax (based on macro metavariable expressions) to extract the field name or type (e.g. `${f.name}`). This would simplify many common cases by leveraging the compiler's own parser. From d4702b840e9d4b412ced21396e2ff5739f947b54 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 22 Oct 2024 05:58:11 -0700 Subject: [PATCH 20/34] Future work: unsafe derives --- text/3698-declarative-derive-macros.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 074f80b5954..68a8186c40a 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -175,6 +175,9 @@ requests arise. # Future possibilities [future-possibilities]: #future-possibilities +We should provide a way for derive macros to declare themselves `unsafe` to +invoke, requiring an unsafe attribute syntax to invoke. + We should provide a way for `macro_rules!` macros to provide better error reporting, with spans, rather than just pointing to the macro. From aaf98601a5b94e61ba69a83afda229988e798330 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Tue, 22 Oct 2024 06:05:14 -0700 Subject: [PATCH 21/34] Future possibilities: parameters --- text/3698-declarative-derive-macros.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 68a8186c40a..a8d195e590e 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -178,6 +178,9 @@ requests arise. We should provide a way for derive macros to declare themselves `unsafe` to invoke, requiring an unsafe attribute syntax to invoke. +We could support passing parameters to derive macros (e.g. +`#[derive(Trait(params), OtherTrait(other, params))]`). + We should provide a way for `macro_rules!` macros to provide better error reporting, with spans, rather than just pointing to the macro. From c6a2d35ab7d1345cbec052017eea17f2e1c010e1 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Sun, 27 Oct 2024 20:08:00 -0700 Subject: [PATCH 22/34] Example --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index a8d195e590e..bcb165bef68 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -149,6 +149,10 @@ The `macro_derive` attribute and its `attributes` syntax are based on the [existing `proc_macro_derive` attribute for proc macros](https://doc.rust-lang.org/reference/procedural-macros.html#derive-macros). +Some crates in the ecosystem already implement the equivalent of derives using +declarative macros; for instance, see +[merde](https://github.com/bearcove/merde). + # Unresolved questions [unresolved-questions]: #unresolved-questions From a71fbf7a4f8b529a024868bd918771226f8055cb Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Thu, 31 Oct 2024 23:55:47 -0700 Subject: [PATCH 23/34] Elaborate on an alternative --- text/3698-declarative-derive-macros.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index bcb165bef68..9971ef19054 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -124,9 +124,11 @@ A mechanism to define attribute macros would let people write attributes like `#[derive_mytrait]`, but that would not provide compatibility with existing derive syntax. -We could allow `macro_rules!` derive macros to emit a replacement token stream; -however, that would be inconsistent with the restriction preventing proc macros -from doing the same. +We could allow `macro_rules!` derive macros to emit a replacement token stream. +That would be inconsistent with the restriction preventing proc macros from +doing the same, but it would give macros more capabilities, and simplify some +use cases. Notably, that would make it easy for derive macros to re-emit a +structure with another `derive` attached to it. We could allow directly invoking a `macro_rules!` derive macro as a function-like macro. This has the potential for confusion, given the From cf8e13e87b97fab1df9d37aaa168b92da735e314 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Thu, 31 Oct 2024 23:57:17 -0700 Subject: [PATCH 24/34] Add more future possibilities --- text/3698-declarative-derive-macros.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 9971ef19054..8caeb5a47e0 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -184,6 +184,11 @@ requests arise. We should provide a way for derive macros to declare themselves `unsafe` to invoke, requiring an unsafe attribute syntax to invoke. +We should provide a way for derive macros to invoke other derive macros. + +We should provide a means to perform a `derive` on a struct without being +directly attached to that struct. + We could support passing parameters to derive macros (e.g. `#[derive(Trait(params), OtherTrait(other, params))]`). From d50dbffcaa9de47823d7d64b74ac31cd8def8164 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 02:54:49 -0800 Subject: [PATCH 25/34] Further discussion on future possibility of derives with parameters --- text/3698-declarative-derive-macros.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 8caeb5a47e0..a5b45a5a966 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -190,7 +190,9 @@ We should provide a means to perform a `derive` on a struct without being directly attached to that struct. We could support passing parameters to derive macros (e.g. -`#[derive(Trait(params), OtherTrait(other, params))]`). +`#[derive(Trait(params), OtherTrait(other, params))]`). This may benefit from +having `derive(...)` rules inside the `macro_rules!` macro declaration, similar +to the `attr(...)` rules proposed in RFC 3697. We should provide a way for `macro_rules!` macros to provide better error reporting, with spans, rather than just pointing to the macro. From c32347393671bdd490d527ee79717d1fa6894ac5 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 02:55:04 -0800 Subject: [PATCH 26/34] Discuss syntax alternative: `derive(...)` rules, like RFC 3697's `attr(...)` --- text/3698-declarative-derive-macros.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index a5b45a5a966..c65d01e52b1 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -135,6 +135,20 @@ function-like macro. This has the potential for confusion, given the append-only nature of derive macros versus the behavior of normal function-like macros. It might potentially be useful for code reuse, however. +## Syntax alternatives + +Rather than using `#[macro_derive]`, we could have `macro_rules!` macros +provide `derive(...)` rules, similar to the `attr(...)` rules proposed by RFC +3697. + +It seems less important to allow a single macro to be both a derive and +non-derive macro, not least of which because of the different naming +conventions (`MyTrait` vs `my_macro`). + +However, using `derive(...)` syntax would make it easier to add parameterized +derives in the future (e.g. `derive(MyTrait(params))`). `derive(...)` syntax +would also be more consistent with the proposed declarative attribute macros. + # Prior art [prior-art]: #prior-art From e088d55b149ba33a84785caa252eb0e684fae549 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 04:51:00 -0800 Subject: [PATCH 27/34] Unresolved question: helper attribute namespacing and hygiene --- text/3698-declarative-derive-macros.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index c65d01e52b1..f5c5ebb9105 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -192,6 +192,26 @@ recommending against pressuring crate maintainers to adopt this feature rapidly, and encourage crate maintainers to link to that guidance if such requests arise. +## Helper attribute namespacing and hygiene + +Should we have a namespacing and hygiene mechanism for helper attributes? +Adding new helper attributes is currently a compatibility hazard, because +they're not namespaced and can conflict. + +For instance, could we have `pub macro_helper_attr! skip` in the standard +library, namespaced under `core::derives` or similar? Could we let macros parse +that in a way that matches it in a namespaced fashion, so that: +- If you write `#[core::derives::skip]`, the macro matches it +- If you `use core::derives::skip;` and `write #[skip]`, the macro matches it +- If you `use elsewhere::skip` (or no import at all) and write `#[skip]`, the + macro *doesn't* match it. + +We already have *some* interaction between macros and name resolution, in order +to have namespaced `macro_rules!` macros. Would something like this be feasible? + +(We would still need to specify the exact mechanism by which macros match these +helper attributes.) + # Future possibilities [future-possibilities]: #future-possibilities From 319ca10149460f3630b7a00516e4d7d83a86f9a0 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 10:45:41 -0800 Subject: [PATCH 28/34] Elaborate a future work item --- text/3698-declarative-derive-macros.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index f5c5ebb9105..1c302558543 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -221,7 +221,8 @@ invoke, requiring an unsafe attribute syntax to invoke. We should provide a way for derive macros to invoke other derive macros. We should provide a means to perform a `derive` on a struct without being -directly attached to that struct. +directly attached to that struct. (This would also potentially require +something like a compile-time reflection mechanism.) We could support passing parameters to derive macros (e.g. `#[derive(Trait(params), OtherTrait(other, params))]`). This may benefit from From 5be851f9fa118bbb6c4adcee06e7adb6c3eebfc9 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 10:49:00 -0800 Subject: [PATCH 29/34] Future possibilities: const Trait and similar --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 1c302558543..2e887b980d3 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -229,6 +229,10 @@ We could support passing parameters to derive macros (e.g. having `derive(...)` rules inside the `macro_rules!` macro declaration, similar to the `attr(...)` rules proposed in RFC 3697. +In the future, if we support something like `const Trait` or similar trait +modifiers, we'll want to support `derive(const Trait)`, and define how a +`macro_rules!` derive handles that. + We should provide a way for `macro_rules!` macros to provide better error reporting, with spans, rather than just pointing to the macro. From 07f297d4147ecb8b27a0ddfe0393d890adfbe2d6 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 23:58:22 -0800 Subject: [PATCH 30/34] Updates from design meeting Switch syntax to use `derive()`, and defer helper attributes to a future RFC. --- text/3698-declarative-derive-macros.md | 41 +++++++------------------- 1 file changed, 10 insertions(+), 31 deletions(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 2e887b980d3..2882cf9feaa 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -26,19 +26,18 @@ feature. [guide-level-explanation]: #guide-level-explanation You can define a macro to implement `derive(MyTrait)` by defining a -`macro_rules!` macro with the `#[macro_derive]` attribute. Such a macro can -create new items based on a struct, enum, or union. Note that the macro can -only append new items; it cannot modify the item it was applied to. +`macro_rules!` macro with one or more `derive()` rules. Such a macro can create +new items based on a struct, enum, or union. Note that the macro can only +append new items; it cannot modify the item it was applied to. For example: ```rust trait Answer { fn answer(&self) -> u32; } -#[macro_derive] macro_rules! Answer { // Simplified for this example - (struct $n:ident $_:tt) => { + derive() (struct $n:ident $_:tt) => { impl Answer for $n { fn answer(&self) -> u32 { 42 } } @@ -62,18 +61,6 @@ instance, the name `mycrate::MyTrait` can refer to both the `MyTrait` trait and the macro for `derive(MyTrait)`. This is consistent with existing derive macros. -A derive macro may also define *helper attributes*. These attributes are -[inert](https://doc.rust-lang.org/reference/attributes.html#active-and-inert-attributes), -and exist for the derive macro to parse and act upon. Note that -they're visible to *all* macros, not just the one that defined them; macros -should ignore any attributes not meant for them. - -To define helper attributes, put an attributes key in the `macro_derive` -attribute, with a comma-separated list of identifiers for helper attributes: -`#[macro_derive(attributes(helper))]`. The derive macro can process the -`#[helper]` attribute, along with any arguments to it, as part of the item the -derive macro was applied to. - If a derive macro emits a trait impl for the type, it may want to add the [`#[automatically_derived]`](https://doc.rust-lang.org/reference/attributes/derive.html#the-automatically_derived-attribute) attribute, for the benefit of diagnostics. @@ -137,17 +124,13 @@ macros. It might potentially be useful for code reuse, however. ## Syntax alternatives -Rather than using `#[macro_derive]`, we could have `macro_rules!` macros -provide `derive(...)` rules, similar to the `attr(...)` rules proposed by RFC -3697. +Rather than using `derive()` rules, we could have `macro_rules!` macros use a +`#[macro_derive]` attribute, similar to the `#[proc_macro_derive]` attribute +used for proc macros. -It seems less important to allow a single macro to be both a derive and -non-derive macro, not least of which because of the different naming -conventions (`MyTrait` vs `my_macro`). - -However, using `derive(...)` syntax would make it easier to add parameterized -derives in the future (e.g. `derive(MyTrait(params))`). `derive(...)` syntax -would also be more consistent with the proposed declarative attribute macros. +However, this would be inconsistent with `attr()` rules as defined in RFC 3697. +This would also make it harder to add parameterized derives in the future (e.g. +`derive(MyTrait(params))`). # Prior art [prior-art]: #prior-art @@ -161,10 +144,6 @@ demonstrating a demand for this. This feature would allow defining such derives without requiring proc macros at all, and would support the same invocation syntax as a proc macro. -The `macro_derive` attribute and its `attributes` syntax are based on the -[existing `proc_macro_derive` attribute for proc -macros](https://doc.rust-lang.org/reference/procedural-macros.html#derive-macros). - Some crates in the ecosystem already implement the equivalent of derives using declarative macros; for instance, see [merde](https://github.com/bearcove/merde). From 0bb3ea4eba7ee5a4d6cb7dc25e8812537999c898 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Wed, 20 Nov 2024 23:59:58 -0800 Subject: [PATCH 31/34] Future work: talk about namespaced, hygienic helper attributes --- text/3698-declarative-derive-macros.md | 48 +++++++++++++------------- 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 2882cf9feaa..12c443a081d 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -171,26 +171,6 @@ recommending against pressuring crate maintainers to adopt this feature rapidly, and encourage crate maintainers to link to that guidance if such requests arise. -## Helper attribute namespacing and hygiene - -Should we have a namespacing and hygiene mechanism for helper attributes? -Adding new helper attributes is currently a compatibility hazard, because -they're not namespaced and can conflict. - -For instance, could we have `pub macro_helper_attr! skip` in the standard -library, namespaced under `core::derives` or similar? Could we let macros parse -that in a way that matches it in a namespaced fashion, so that: -- If you write `#[core::derives::skip]`, the macro matches it -- If you `use core::derives::skip;` and `write #[skip]`, the macro matches it -- If you `use elsewhere::skip` (or no import at all) and write `#[skip]`, the - macro *doesn't* match it. - -We already have *some* interaction between macros and name resolution, in order -to have namespaced `macro_rules!` macros. Would something like this be feasible? - -(We would still need to specify the exact mechanism by which macros match these -helper attributes.) - # Future possibilities [future-possibilities]: #future-possibilities @@ -223,10 +203,6 @@ As people test this feature and run into limitations of `macro_rules!` parsing, we should consider additional features to make this easier to use for various use cases. -We may want to provide a means to namespace helper attributes or detect -collisions between them. This would apply to both proc macros and -`macro_rules!` macros. - We could provide a macro matcher to match an entire struct field, along with syntax (based on macro metavariable expressions) to extract the field name or type (e.g. `${f.name}`). This would simplify many common cases by leveraging @@ -236,3 +212,27 @@ We could do the same for various other high-level constructs. We may want to provide simple helpers for generating/propagating `where` bounds, which would otherwise be complex to do in a `macro_rules!` macro. + +## Helper attribute namespacing and hygiene + +We should provide a way for derive macros to define helper attributes ([inert +attributes](https://doc.rust-lang.org/reference/attributes.html#active-and-inert-attributes) +that exist for the derive macro to parse and act upon). Such attributes are +supported by proc macro derives; however, such attributes have no namespacing, +and thus currently represent compatibility hazards because they can conflict. +We should provide a namespaced, hygienic mechanism for defining and using +helper attributes. + +For instance, could we have `pub macro_helper_attr! skip` in the standard +library, namespaced under `core::derives` or similar? Could we let macros parse +that in a way that matches it in a namespaced fashion, so that: +- If you write `#[core::derives::skip]`, the macro matches it +- If you `use core::derives::skip;` and `write #[skip]`, the macro matches it +- If you `use elsewhere::skip` (or no import at all) and write `#[skip]`, the + macro *doesn't* match it. + +We already have *some* interaction between macros and name resolution, in order +to have namespaced `macro_rules!` macros. Would something like this be feasible? + +(We would still need to specify the exact mechanism by which macros match these +helper attributes.) From d38892f04763a051012be5f619579d9de1d15e1a Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Thu, 21 Nov 2024 00:00:33 -0800 Subject: [PATCH 32/34] Future work: suggest a possible naming lint --- text/3698-declarative-derive-macros.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 12c443a081d..9feb96be456 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -213,6 +213,10 @@ We could do the same for various other high-level constructs. We may want to provide simple helpers for generating/propagating `where` bounds, which would otherwise be complex to do in a `macro_rules!` macro. +We may want to add a lint for macro names, encouraging macros with derive rules +to use `CamelCase` names, and encouraging macros without derive rules to use +`snake_case` names. + ## Helper attribute namespacing and hygiene We should provide a way for derive macros to define helper attributes ([inert From cae86ff4d9c2fbd39af1d772ec90326ade043664 Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Thu, 21 Nov 2024 00:01:02 -0800 Subject: [PATCH 33/34] Add unsafe derive rules and corresponding syntax This matches what RFC 3715 defines for proc macro derives. --- text/3698-declarative-derive-macros.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 9feb96be456..82054d9643b 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -70,6 +70,18 @@ If a derive macro mistakenly emits the token stream it was applied to the duplicate item should hint to the user that the macro was defined incorrectly, and remind the user that derive macros only append new items. +A `derive()` rule can be marked as `unsafe`: `unsafe derive() (...) +=> { ... }`. Invoking such a derive using a rule marked as unsafe +requires unsafe derive syntax: either +`#[unsafe(derive(DangerousTrait))]` or +`#[derive(unsafe(DangerousTrait))]`. (The latter syntax allows +isolating the `unsafe` to a single derive within a list of +derives.) Invoking an unsafe derive rule without the unsafe derive +syntax will produce a compiler error. Using the unsafe derive +syntax without an unsafe derive will trigger an "unused unsafe" +lint. (RFC 3715 defines the equivalent mechanism for proc macro +derives.) + # Drawbacks [drawbacks]: #drawbacks From 98aca066794b1891cc5a36ec437a1ebef0a7893e Mon Sep 17 00:00:00 2001 From: Josh Triplett Date: Thu, 21 Nov 2024 00:01:38 -0800 Subject: [PATCH 34/34] Add reference-level explanation with grammar additions --- text/3698-declarative-derive-macros.md | 34 ++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/text/3698-declarative-derive-macros.md b/text/3698-declarative-derive-macros.md index 82054d9643b..648557d969b 100644 --- a/text/3698-declarative-derive-macros.md +++ b/text/3698-declarative-derive-macros.md @@ -82,6 +82,40 @@ syntax without an unsafe derive will trigger an "unused unsafe" lint. (RFC 3715 defines the equivalent mechanism for proc macro derives.) +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +The grammar for macros is extended as follows: + +> _MacroRule_ :\ +>    ( `unsafe`? `derive` `(` `)` )? _MacroMatcher_ `=>` _MacroTranscriber_ + +The _MacroMatcher_ matches the entire construct the attribute was +applied to, receiving precisely what a proc-macro-based attribute +would in the same place. + +(The empty parentheses after `derive` reserve future syntax space +for derives accepting arguments, at which time they'll be replaced +by a second _MacroMatcher_ that matches the arguments.) + +A derive invocation that uses an `unsafe derive` rule will produce +an error if invoked without using the `unsafe` derive syntax. A +derive invocation that uses an `derive` rule (without `unsafe`) +will trigger the "unused unsafe" lint if invoked using the `unsafe` +derive syntax. A single derive macro may have both `derive` and +`unsafe derive` rules, such as if only some invocations are unsafe. + +This grammar addition is backwards compatible: previously, a _MacroRule_ could +only start with `(`, `[`, or `{`, so the parser can easily distinguish rules +that start with `derive` or `unsafe`. + +Adding `derive` rules to an existing macro is a semver-compatible change, +though in practice, it will likely be uncommon. + +If a user invokes a macro as a derive and that macro does not have any `derive` +rules, the compiler should give a clear error stating that the macro is not +usable as a derive because it does not have any `derive` rules. + # Drawbacks [drawbacks]: #drawbacks