Stabilize generic associated types by jackh726 · Pull Request #96709 · rust-lang...
source link: https://github.com/rust-lang/rust/pull/96709
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Closes #44265
I'm going to spend the next couple days writing/working on external documentation, but I want this to open this to get FCPs started. If we're lucky, we can get this in 1.62 (if we're not, oh well).
Status of the discussion
- There have been several serious concerns raised, summarized here.
- There has also been a deep-dive comment explaining some of the "patterns of code" that are enabled by GATs, based on use-cases posted to this thread or on the tracking issue.
- We have modeled some aspects of GATs in a-mir-formality to give better confidence in how they will be resolved in the future. You can read a write-up here.
- The major points of the discussion have been summarized on the GAT initiative repository.
- FCP has been proposed and we are awaiting final decisions and discussion amidst the relevant team members.
Stabilization proposal
This PR proposes the stabilization of #![feature(generic_associated_types)]. While there a number of future additions to be made and bugs to be fixed (both discussed below), properly doing these will require significant language design and will ultimately likely be backwards-compatible. Given the overwhelming desire to have some form of generic associated types (GATs) available on stable and the stability of the "simple" uses, stabilizing the current subset of GAT features is almost certainly the correct next step.
Tracking issue: #44265
Initiative: https://rust-lang.github.io/generic-associated-types-initiative/
RFC: https://github.com/rust-lang/rfcs/blob/master/text/1598-generic_associated_types.md
Version: 1.62 (2022-05-19 => beta, 2022-06-30 => stable).
[ ] Reference documentation
[ ] Book chapter
Motivation
There are a myriad of potential use cases for GATs. Stabilization unblocks probable future language features (e.g. async functions in traits), potential future standard library features (e.g. a LendingIterator or some form of Iterator with a lifetime generic), and a plethora of user use cases (some of which can be seen just by scrolling through the tracking issue and looking at all the issues linking to it).
There are a myriad of potential use cases for GATs. First, there are many users that have chosen to not use GATs primarily because they are not stable (some of which can be seen just by scrolling through the tracking issue and looking at all the issues linking to it). Second, while language feature desugaring isn't blocked on stabilization, it gives more confidence on using the feature. Likewise, library features like LendingIterator are not necessarily blocked on stabilization to be implemented unstably; however few, if any, public-facing APIs actually use unstable features.
This feature has a long history of design, discussion, and developement - the RFC was first introduced roughly 6 years ago. While there are still a number of features left to implement and bugs left to fix, it's clear that it's unlikely those will have backwards-incompatibility concerns. Additionally, the bugs that do exist do not strongly impede the most-common use cases.
What is stabilized
The primary language feature stabilized here is the ability to have generics on associated types, as so. Additionally, where clauses on associated types will now be accepted, regardless if the associated type is generic or not.
trait ATraitWithGATs {
type Assoc<'a, T> where T: 'a;
}
trait ATraitWithoutGATs<'a, T> {
type Assoc where T: 'a;
}When adding an impl for a trait with generic associated types, the generics for the associated type are copied as well. Note that where clauses are allowed both after the specified type and before the equals sign; however, the latter is a warn-by-default deprecation.
struct X;
struct Y;
impl ATraitWithGATs for X {
type Assoc<'a, T> = &'a T
where T: 'a;
}
impl ATraitWithGATs for Y {
type Assoc<'a, T>
where T: 'a
= &'a T;
}To use a GAT in a function, generics are specified on the associated type, as if it was a struct or enum. GATs can also be specified in trait bounds:
fn accepts_gat<'a, T>(t: &'a T) -> T::Assoc<'a, T>
where for<'x> T: ATraitWithGATs<Assoc<'a, T> = &'a T> {
...
}GATs can also appear in trait methods. However, depending on how they are used, they may confer where clauses on the associated type definition. More information can be found here. Briefly, where clauses are required when those bounds can be proven in the methods that construct the GAT or other associated types that use the GAT in the trait. This allows impls to have maximum flexibility in the types defined for the associated type.
To take a relatively simple example:
trait Iterable {
type Item<'a>;
type Iterator<'a>: Iterator<Item = Self::Item<'a>>;
fn iter<'x>(&'x self) -> Self::Iterator<'x>;
//^ We know that `Self: 'a` for `Iterator<'a>`, so we require that bound on `Iterator`
// `Iterator` uses `Self::Item`, so we also require a `Self: 'a` on `Item` too
}A couple well-explained examples are available in a previous blog post.
What isn't stabilized/implemented
Universal type/const quantification
Currently, you can write a bound like X: for<'a> Trait<Assoc<'a> = &'a ()>. However, you cannot currently write for<T> X: Trait<Assoc<T> = T> or for<const N> X: Trait<Assoc<N> = [usize; N]>.
Here is an example where this is needed:
trait Foo {}
trait Trait {
type Assoc<F: Foo>;
}
trait Trait2: Sized {
fn foo<F: Foo, T: Trait<Assoc<F> = F>>(_t: T);
}In the above example, the caller must specify F, which is likely not what is desired.
Object-safe GATs
Unlike non-generic associated types, traits with GATs are not currently object-safe. In other words the following are not allowed:
trait Trait {
type Assoc<'a>;
}
fn foo(t: &dyn for<'a> Trait<Assoc<'a> = &'a ()>) {}
//^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not allowed
let ty: Box<dyn for<'a> Trait<Assoc<'a> = &'a ()>>;
//^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not allowedHigher-kinded types
You cannot write currently (and there are no current plans to implement this):
struct Struct<'a> {}
fn foo(s: for<'a> Struct<'a>) {}Tests
There are many tests covering GATs that can be found in src/test/ui/generic-associated-types. Here, I'll list (in alphanumeric order) tests highlight some important behavior or contain important patterns.
./parse/*: Parsing of GATs in traits and impls, and the trait path with GATs./collections-project-default.rs: Interaction with associated type defaults./collections.rs: TheCollectionpattern./const-generics-gat-in-trait-return-type-*.rs: Const parameters./constraint-assoc-type-suggestion.rs: Emit correct syntax in suggestion./cross-crate-bounds.rs: Ensure we handles bounds across crates the same./elided-in-expr-position.rs: Disallow lifetime elision in return position./gat-in-trait-path-undeclared-lifetime.rs: Ensure we error on undeclared lifetime in trait path./gat-in-trait-path.rs: Base trait path case./gat-trait-path-generic-type-arg.rs: Don't allow shadowing of parameters./gat-trait-path-parenthesised-args.rs: Don't allow paranthesized args in trait path./generic-associated-types-where.rs: Ensure that we require where clauses from trait to be met on impl./impl_bounds.rs: Check that the bounds on GATs in an impl are checked./issue-76826.rs:Windowspattern./issue-78113-lifetime-mismatch-dyn-trait-box.rs: Implicit 'static diagnostics./issue-84931.rs: Ensure that we have a where clause on GAT to ensure trait parameter lives long enough./issue-87258_a.rs: Unconstrained opaque type with TAITs./issue-87429-2.rs: Ensure we can use bound vars in the bounds./issue-87429-associated-type-default.rs: Ensure bounds hold with associated type defaults, for both trait and impl./issue-87429-specialization.rs: Check that bounds hold under specialization./issue-88595.rs: Under the outlives lint, we require a bound for both trait and GAT lifetime when trait lifetime is used in function./issue-90014.rs: Lifetime bounds are checked with TAITs./issue-91139.rs: Under migrate mode, but not NLL, we don't capture implied bounds from HRTB lifetimes used in a function and GATs./issue-91762.rs: We used to too eagerly pick param env candidates when normalizing with GATs. We now require explicit parameters specified../issue-95305.rs: Disallow lifetime elision in trait paths./iterable.rs:Iterablepattern./method-unsatified-assoc-type-predicate.rs: Print predicates with GATs correctly in method resolve error./missing_lifetime_const.rs: Ensure we must specify lifetime args (not elidable)./missing-where-clause-on-trait.rs: Ensure we don't allow stricter bounds on impl than trait./parameter_number_and_kind_impl.rs: Ensure paramters on GAT in impl match GAT in trait./pointer_family.rs:PointerFamilypattern./projection-bound-cycle.rs: Don't allow invalid cycles to prove bounds./self-outlives-lint.rs: Ensures that an e.g.Self: 'ais written on the traits GAT if that bound can be implied from the GAT usage in the trait./shadowing.rs: Don't allow lifetime shadowing in params./streaming_iterator.rs:StreamingIterator(LendingIterator) pattern./trait-objects.rs: Disallow trait objects for traits with GATs./variance_constraints.rs: Require that GAT substs be invariant
Remaining bugs and open issues
A full list of remaining open issues can be found at: F-generic_associated_types `#![feature(generic_associated_types)]` a.k.a. GATs
There are some known-bug tests in-tree at src/test/ui/generic-associated-types/bugs.
Here I'll categorize most of those that GAT bugs (or involve a pattern found more with GATs), but not those that include GATs but not a GAT issue in and of itself. (I also won't include issues directly for things listed elsewhere here.)
Using the concrete type of a GAT instead of the projection type can give errors, since lifetimes are chosen to be early-bound vs late-bound.
Where clause bounds from associated types don't add implied bounds to functions. This means that using concrete types vs projection types can give unsatisfied lifetime bound errors.
In certain cases, we can run into cycle or overflow errors. This is more generally a problem with associated types.
Bounds on an associatd type need to be proven by an impl, but where clauses need to be proven by the usage. This can lead to confusion when users write one when they mean the other.
We sometimes can't normalize closure signatures fully. Really an asociated types issue, but might happen a bit more frequently with GATs, since more obvious place for HRTB lifetimes.
When calling a function, we assign types to parameters "too late", after we already try (and fail) to normalize projections. Another associated types issue that might pop up more with GATs.
We don't fully have implied bounds for lifetimes appearing in GAT trait paths, which can lead to unconstrained type errors.
Suggestion for adding lifetime bounds can suggest unhelpful fixes (T: 'a instead of Self: 'a), but the next compiler error after making the suggested change is helpful.
We can end up requiring that for<'a> I: 'a when we really want for<'a where I: 'a> I: 'a. This can leave unhelpful errors than effectively can't be satisfied unless I: 'static. Requires bigger changes and not only GATs.
Unlike with non-generic associated types, we don't eagerly normalize with param env candidates. This is intended behavior (for now), to avoid accidentaly stabilizing picking arbitrary impls.
Some Iterator adapter patterns (namely filter) require Polonius or unsafe to work.
Potential Future work
Universal type/const quantification
No work has been done to implement this. There are also some questions around implied bounds.
Object-safe GATs
The intention is to make traits with GATs object-safe. There are some design work to be done around well-formedness rules and general implementation.
GATified std lib types
It would be helpful to either introduce new std lib traits (like LendingIterator) or to modify existing ones (adding a 'a generic to Iterator::Item). There also a number of other candidates, like Index/IndexMut and Fn/FnMut/FnOnce.
Reduce the need for for<'a>
Seen here. One possible syntax:
trait Iterable {
type Iter<'a>: Iterator<Item = Self::Item<'a>>;
}
fn foo<T>() where T: Iterable, T::Item<let 'a>: Display { } //note the `let`!Better implied bounds on higher-ranked things
Currently if we have a type Item<'a> where self: 'a, and a for<'a> T: Iterator<Item<'a> = &'a (), this requires for<'a> Self: 'a. Really, we want for<'a where T: 'a> ...
There was some mentions of this all the back in the RFC thread here.
Alternatives
Make generics on associated type in bounds a binder
Imagine the bound for<'a> T: Trait<Item<'a>= &'a ()>. It might be that for<'a> is "too large" and it should instead be T: Trait<for<'a> Item<'a>= &'a ()>. Brought up in RFC thread here and in a few places since.
Another related question: Is for<'a> the right syntax? Maybe where<'a>? Also originally found in RFC thread here.
Stabilize lifetime GATs first
This has been brought up a few times. The idea is to only allow GATs with lifetime parameters to in initial stabilization. This was probably most useful prior to actual implementation. At this point, lifetimes, types, and consts are all implemented and work. It feels like an arbitrary split without strong reason.
History
- On 2016-04-30, RFC opened
- On 2017-09-02, RFC merged and tracking issue opened
- On 2017-10-23, Move Generics from MethodSig to TraitItem and ImplItem
- On 2017-12-01, Generic Associated Types Parsing & Name Resolution
- On 2017-12-15, Lifetime Resolution for Generic Associated Types #46706
- On 2018-04-23, Feature gate where clauses on associated types
- On 2018-05-10, Extend tests for RFC1598 (GAT)
- On 2018-05-24, Finish implementing GATs (Chalk)
- On 2019-12-21, Make GATs less ICE-prone
- On 2020-02-13, fix lifetime shadowing check in GATs
- On 2020-06-20, Projection bound validation
- On 2020-10-06, Separate projection bounds and predicates
- On 2021-02-05, Generic associated types in trait paths
- On 2021-02-06, Trait objects do not work with generic associated types
- On 2021-04-28, Make traits with GATs not object safe
- On 2021-05-11, Improve diagnostics for GATs
- On 2021-07-16, Make GATs no longer an incomplete feature
- On 2021-07-16, Replace associated item bound vars with placeholders when projecting
- On 2021-07-26, GATs: Decide whether to have defaults for
where Self: 'a - On 2021-08-25, Normalize projections under binders
- On 2021-08-03, The push for GATs stabilization
- On 2021-08-12, Detect stricter constraints on gats where clauses in impls vs trait
- On 2021-09-20, Proposal: Change syntax of where clauses on type aliases
- On 2021-11-06, Implementation of GATs outlives lint
- On 2021-12-29. Parse and suggest moving where clauses after equals for type aliases
- On 2022-01-15, Ignore static lifetimes for GATs outlives lint
- On 2022-02-08, Don't constrain projection predicates with inference vars in GAT substs
- On 2022-02-15, Rework GAT where clause check
- On 2022-02-19, Only mark projection as ambiguous if GAT substs are constrained
- On 2022-03-03, Support GATs in Rustdoc
- On 2022-03-06, Change location of where clause on GATs
- On 2022-05-04, A shiny future with GATs blog post
- On 2022-05-04, Stabilization PR
added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
labels
Collaborator
rust-highfive commented on May 4
|
Some changes occurred in src/tools/rustfmt. |
added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label
|
|
The latest upstream changes (presumably |
I don't think we should stabilise GATs now (and I'm not convinced we should stabilise GATs at all). I'm sorry this is going to be a negative post, I'll try and be as positive as possible. I really appreciate the work that has gone into this feature and for help in answering my questions along the way. I think it is very important to make a strong argument that not only can we add GATs to the language, but that we should. This is an important decision. GATs are probably the largest change to the type systems since associated types (pre 1.0), and certainly the largest change since 1.0. They're also the largest addition in complexity since way before 1.0 and the language feature with the largest possibility to change the character of the language since before 1.0. I think that a decision on GATs is a fairly straightforward trade-off between complexity and expressivity. I'll address both sides of the trade-off. ComplexityRust is often criticised for being an overly complex language. From last year's annual survey, 33% of users said their biggest worry for the future of Rust was that it would become "too complex" (the second highest ranked answer). Languages which support GATs, HKTs, or similar typically have even worse reputations than Rust for complexity and learning curve. GATs nearly always increase complexity. They introduce new syntax and new semantics to the type system. There are a few use cases for lifetime-GATs which simplify types, but mostly GATs are useful for expressing new abstractions which are inherently difficult for many programmers to understand. GATs' complexity is not a 'zero-cost abstraction' (in the sense that you only pay the price for a feature if you use it): GATs will primarily be used by library authors and are part of the API, thus programmers will not get a choice to avoid them. They will be part of libraries and if programmers want to use those libraries, they must learn about GATs (compare to async, where if a programmer is not doing async programming, they don't need to know about async or await). GATs' complexity is not restricted to advanced programmers. Since GATs are used in APIs, they cannot be hidden only in implementations where only advanced programmers (as library authors are likely to be) need to care about them. They are exposed to all programmers (compare to unsafe coding features, which can be completely encapsulated). GATs are a feature which appeal to language geeks and compiler hackers and can even be fairly intuitive to us, but which are terrifying for most programmers. Note how popular Haskell is with PL/compiler people, but how it is largely shunned by industry. ExpressivityGATs clearly increase expressivity, but I think that a solid argument that the expressivity is useful has not been made. Furthermore, the increased expressivity changes the character of Rust significantly. There are use cases for GATs, but very few of them have been proved out. I don't know of any use cases which have been implemented and demonstrated to be significantly useful. I realise there is a reluctance to use unstable features, and this is a high bar. But the bar should be high - this is a huge change to the language. There are numerous cases of small bugs or small gaps in expressivity which have prevented people using GATs for the use cases they want to use them for (see e.g., the blog post linked from the OP, or this reddit thread). These are the sort of things which must be addressed before stabilisation so that we can be sure that they are in fact small and not hiding insurmountable issues. GATs have a strong use case inside the compiler as part of the implementation of async methods or impl Trait in traits. However, there is no requirement that GATs need to be exposed to the user to facilitate this usage. I think using GATs as a principled internal representation is fantastic, but that does not require exposing them to users. Most use cases (and certainly most of the compelling use cases) are for lifetime GATs. I think that we must separately justify lifetime and type GATs, and we could add one to the language without adding the other (e.g., we have HRTBs for lifetimes but not for types). Note also that lifetime GATs are categorically simpler than type GATs because the grammar of types is structural and recursive, whereas the grammar of lifetimes is simple. Furthermore, lifetimes are part of what makes Rust unique and thus expressivity at the cost of complexity is more essential to the language in the lifetime case. Finally, GATs increase expressivity but in a way which takes Rust in a new direction. GATs and HKTs more generally are a fine way of building abstractions, but they are not the Rust way. We use concrete types like |
|
@nrc I agree with the sentiment of much of your post, but disagree on the specifics. Rather, I think the problem of teaching and complexity is a pervasive problem across Rust as a whole, and I don't think that holding back features is a particularly effective way to solve the problem. As-is, I've seen many APIs in the wild that are unnecessarily complex because the author didn't have access to GATs, requiring them to create absurd abstraction towers like custom type family traits in order to achieve similar expressivity. GATs don't really enable any capability that didn't exist before, but they definitely have the power to simplify many of the more weird cases of type astronomy.
To this, I'd like to provide a specific and solid example of GATs (and in particular, type GATs) proving extremely useful. I work on I strongly suspect that many similar use-cases will appear in the future, given the possibilities GATs open up to specialise the implementation of functions, predicated upon a generic type. As far as I'm aware, no other languages that support GATs (or adjacent features like HKTs) have the same monomorphisation and performance promises that Rust has, so this space remains mostly unexplored. But, without stabilisation, this space is not open for exploration by API authors that care about performance. |
|
Parser combinators are parsers composed of smaller parsers, similar to how There are a number of cases where it's necessary to parse a pattern without actually evaluating its output. For example, This can be very wasteful though, particularly if the creation of With GATs, we can specialise the invocation of each parser's parse function with a type that controls whether an output value gets actually generated or not at compile-time (in effect, a restricted form of the monad pattern) that uses a GAT to work across whatever types the implementation of the parser cares for. The beautiful part is that this has no impact on the user-facing API, but 'magically' speeds up the parser by statically guaranteeing that unnecessary work will be skipped, allowing something like Although the details of this case are quite specific to chumsky, I believe this general pattern - GATs as a way to generically define an operation with types known only to the implementation - is generally useful for a lot of code, as is visible in existing languages with GATs/HKTs. Where Rust really hits the ball out of the park is that it guarantees monomorphisation, allowing these patterns to be truly zero-cost. It's difficult to overstate just how powerful that is. |
overlisted commented on May 5
|
@nrc I don't think that libraries exposing complex stuff in their public interface is necessarily a bad thing. It should be up to the library author to make it clear how to use it. Serde has a short explanation of HRTBs right in their docs. |
wild alternative number 3: only stabilize type GATswhile I don't know how possible this is in practice, (since Item is a supertype of Item<&'a T> after all) but to me it seems like the vast majority of the footguns and undesired complexity that can make dealing with gats suprisingly unergonomic is around LendingIterator and friends, but not around "simple" generic types like in zesterer's example. At the very least, moving the marketing of this feature away from LendingIterator and lifetime GATs in general and towards concrete ergonomic and performance wins with "simple types" in the short term could be useful in reducing complexity for the end user to understand |
Member
BurntSushi commented on May 5
|
I think I might be one of the people who has wanted GATs the longest. My desire for them predates my lame attempt at working around their absence many moons ago. The first time I realized the With all that said, I do unfortunately tend to agree with quite a bit of what @nrc is saying.
I'm quite sympathetic to GATs exploding the complexity budget of Rust, but I think this is one of the more compelling points in terms of not stabilizing them as-is. I understand the idea of making incremental progress, but as a total outsider to lang development, this feature looks like it has way too many papercuts and limitations to stabilize right now. Not being able to write a basic In terms of making incremental progress, what does it look like to use GATs as an implementation detail to make async traits work? Is there a reason why we shouldn't start there? I do appreciate that there is a chicken-and-egg problem here. It's hard to get the experience I think we need with people actually using GATs before stabilizing them. I'm not sure how to approach that problem other than taking a more conservative incremental approach here. |
|
In response to @nrc (and making every section collapsible to not take up 2 vertical screen widths): "GATs nearly always increase complexity" GATs' complexity is in the public API "if a programmer is not doing async programming, they don't need to know about async or await""Haskell is [...] largely shunned by industry" Now for the parts where I agree with you, because while I don't agree with your conclusion, I absolutely found every point you raised to be well-constructed and at minimum worth consideration. "abstractions like 'collection' [...] are not good fits for most""Most use cases [...] are for lifetime GATs" In the end though, I think at minimum we should really be stabilizing lifetime GATs. I personally am not too worried about type GATs, however I think this cost/benefit of holding them back is far more agreeable than lifetime GATs. So even if the route is "don't stabilize everything" out of an abundance of caution, I would still heavily implore those making the final call to at minimum shoot for lifetime GATs being stabilized this cycle. edits Response to BurntSushi Also this part from @zesterer describes my needs quite nicely:
This is a pattern that I find myself stubbing my toe on quite often (as someone who writes a lot of libraries focused on ease-of-use, not as a normal user). Very well put. |
Yeah, I wanted to say that too. A function such as: is not a zero-cost abstraction either, both in terms of implementation and cognitive complexity (someone who doesn’t know async programming in Rust would assume the function returns So GAT requires learning… like pretty much everything a typed language provides. Making a point on the tradeoffs here using something that is already a pretty big tradeoff (and distracting, because the types are harder to read with
Yeah, I struggle a lot with that in |
|
@phaazon "is not a zero-cost abstraction" zero cost abstraction doesn't mean zero cost, it's mean "you would have the same result doing it by "hand"". So you use here is wrong, async being zero cost abstraction mean that if you want do the same feature that offer async by hand you would not gain speed. (thus I don't talk about implementation here) What does 'Zero Cost Abstraction' mean? |
I am also quite sympathetic to the complexity argument here too. And I think I see the complexity as very different than you, based on what you've written. That GATs make the language more consistent in some corners does not really hold much sway with me in terms of "complexity" here. What I think of, and what I think @nrc is thinking, is the emergent complexity of abstractions that are enabled by GATs. I don't really know how to untie the knot here in terms of teasing these different sorts of complexities apart, but they are very different ideas that unfortunately can both be reasonably described as "complexity." If someone wanted to start a Zulip stream on this topic, I'd be happy to try and explain this viewpoint a bit more because I think it's a huge topic that has the potential to derail this thread.
I have absolutely no idea. I have no tools for how to even think about an answer to this question. :-/
OK so thank you for mentioning this, because this is not what I had in mind. If everyone involved in Rust's type system really believes that all outstanding issues are not only resolvable, but are resolvable in a backwards compatible way, then I'm happy to trust that. They're the experts, not me. I might raise an eyebrow in surprise of such confidence given how many outstanding issues there are today personally, but no, ultimately I'd trust them. The issue I have with GAT's incompleteness is actually about the user experience. The const analogy is a good one, because it demonstrates just how different things are here. In my understanding, it is very easy to articulate and apply the limitations of what is allowed in a const context. Just about anyone can understand things like:
Even better than that, the limitations are so clear and crisp, that the compiler can recognize it and tell you exactly what's wrong. That's a reasonable failure mode. "They just haven't gotten to it today so I can't do that." OK. Great. Now I can move on to some other solution to my problem or whatever. Compare that with trying to write a This is a really important point. Because if I try to write that My advice to folks is to try and look at this feature and the experience it gives to users through beginner eyes. Heck, I'm no Rust beginner (nor even someone completely ignorant of type theory) and a lot of the failure modes I've seen with the GAT feature are really inscrutable. I think we should do one of three things:
|
@BurntSushi The problem of lifetime borrow that the filter example show is not unique to GaTs, and I saw a lot of question on stackoverflow about this problem. Always solve by polonius borrow checker. I don't think this is a good argument on this case, it apply to every Rust code. But you are right everytime the user is very confuse why the code don't compile. Returning a reference from a HashMap or Vec causes a borrow to last beyond the scope it's in? for example (there are several linked question). Notice the question is from 2016 |
Member
BurntSushi commented on May 5
|
@Stargateur That's beside the point for a few reasons, and I don't think is really addressing the substance of my concerns. Firstly I acknowledged that the Secondly, the |
Contributor
jam1garner commented on May 5
|
@BurntSushi very fair points! And apologies on the phrasing, didn't draw enough of a line between where I was riffing off you and where I was stating my own opinion. While I'm sure this isn't the greatest solution, I wonder if something like "GATs without trait bounds" is a possible path forward? (Or maybe slightly broader—no constraining lifetime GATs with trait bounds?) I know for many that's functionally useless, it just happens to cover roughly a third of my usecases while (to my knowledge) having a good biglt fewer edge cases for {borrow, type, trait bound} checking to go wrong. Not sure how representative my usecase is though so maybe for everyone else that'd be more frustrating than anything :) And you're right I definitely was (somewhat intentionally) construing different definitions (feature cognitive overhead vs, as you said, emergent complexity of the abstractions it enables). I'd like to think I somewhat covered how I feel about the kind you/nrc are talking about (and how I think in practice I'm not sure beginners would actually need to think about it, similar to iterator adapter trait bounds, assuming "collection traits" and the likes don't become idiomatic), however that is of course a bit handwavey as my argument isn't all that concrete. And yeah the UX could be better :( I can't speak for everyone but as someone who tries to make diagnostics PRs when I hit issues and have the time/energy, I personally can't really kick the tires if I can't use the feature in my libraries, and thus don't have a very natural path to find pain points to. I've converted a library of mine twice (once to real GATs, once to lifetime GATs emulation via HRTBs) but it's a lot of refactoring effort (big crate, w/ derive macro, etc) only to not get too far out of trivial usage territory. Part of me is tempted to say "if diagnostics are the blocker, stabilize and improvements will roll in", as I believe someone else said upthread it's sorta a chicken-and-egg issue. Since it's mainly a feature aimed at sufficiently complex libraries I don't think my situation (prohibitively high cost to give any real-world testing, only for branch staling to make that effort difficult to keep useful) is that uncommon. Regardless, I'm very grateful for the work being done here. The boundaries are fuzzy and the interactions with lifetimes are pretty novel. Even if GATs get held back indefinitely I'll be happy, whatever decision is made will probably be the one I agree with in 2 year's time anyways :P |
|
it's far from fair to try and imply a single aspect of Haskell (the degree of abstraction) is the sole cause... I agree with that. We cannot simply attribute it to HKT, and HKT is rather intuitive (especially in Haskell) for me. The imagination of HKT in the blog post looks even attractive since it seems actually a simplification for me. Anyway, thanks everyone for pushing Rust into a better language! |
Contributor
CraftSpider commented on May 5
|
I agree with the statement that HKTs are not, on their own, fundamentally unintuitive. Also, as an outside observer: why are these points being raised now, and not before many posts (including official Rust blog posts) about GATs approaching stabilization? I've been expecting them as a user as a non-controversial addition for a while now. This isn't meant as a criticism of the points, just curiosity about what happened that they only came up after the work was done. |
|
@CraftSpider Theoretically, we already decide to include GaTs like describe in RFC 1598, thus it's allowed to make small change or even cancel everything before it's land to stable. "why are these points being raised now" there two factors here, first it's very hard to predict how thing as complex as GaTs will go. Unexpected things happen. Secondly, the stabilisation request is the "last time" where people can raise concern. Specially the concern here is precisely about the stabilisation could be premature. It's not surprising that a feature like GaTs that could change everything in Rust attract concern like this. Then these concerns can be look by the team associate with the stabilisation, here I think it's the Lang team that will have the last word about this. |
|
I'll address some technical points later, but for now...
This is partly a process failure, but also this is exceptional work both in scope, and in time between RFC and implementation (it's been 6 years since the RFC was proposed and 5 years since it was accepted). In our process for creating new features, there is no formal place for registering objections between RFC discussion and stabilisation discussion, that is usually OK, but here the time period was exceptionally long. Personally, I have registered concerns about this feature privately and publicly, but like I say there is nowhere to do that officially. The project has changed a lot since the RFC was proposed, both technically (the type system has got more complex in other ways) and non-technically (the language is much more mature now, we have many more users, and are attracting new users at a much higher rate), also people change - in six years people leave and new people arrive, and people's opinions change. Also, it is expected that during implementation and once an implementation is available for use, we gain experience and that informs our decision making and can change people's opinions. Accepting an RFC is never a guarantee that a feature will be stabilized, and in fact it is quite rare that a feature is stabilised in exactly the same form as described in an RFC. |
|
As an outside observer: Having so many points raised against the stabilization while trying to stabilize this is a bit concerning. The main take-away I get from scrolling through this thread is that GATs are not ready yet (missing/limiting parts) or that it's actually unknown whether GATs in this form is production ready / the papercuts and limitations will have bad/limiting consequences in the future (missing experience). I totally get the chicken-and-egg problem here. Extending @BurntSushi idea to "Sidestep the above two problems by starting with a more conservative stabilization": I remember the async stabilization being driven and motivated very much by (positive) experience from nightly production usage (fuchsia team). Maybe something like that is needed for GATs as well to assess that the current approach is ready. Don't the generators for As a closing note, I am really baffled by the commitment of all involved. For me personally GATs are first of all the "things needed" to get async fns in traits and I am regularly surprised of how much work this actually requires. Hats off to you folks! |
Contributor
PoignardAzur commented on May 6
My personal take: at first I thought positively of GATs, and my reaction was essentially "I'm looking forwards to the day this is available with stable Rust". Then as we got closer to the deadline, more and more blog posts came out with examples of GAT code and at that point my reaction became "Wait, is that what GATs look like in practice? I don't understand any of this code. Am I going to have to read code like this everywhere soon?", hence why I'd now agree with BurntSushi that the feature still needs some baking. For instance, reading code like this is making me very nervous: |
Member
joshtriplett commented on Aug 8
|
I read through the stabilization report and the documentation in the GATs repository, and this looks good to me. @rfcbot reviewed |
Member
nrc commented 29 days ago
@joshtriplett there are a bunch of reasonable objections in this thread, do you have any kind of response to those? Could you explain why "this looks good to me"? You're co-lead of the language team and a prominent member of the libs team, this is arguably the largest change to Rust's type system since 1.0. I'd really like to understand a bit more of your thought process here. |
|
@nrc I've followed some of this thread, as well as some summaries of the objections, and similar discussions on Zulip. Leaving aside @nikomatsakis's summary post (which as stated looks reasonable to me), my thought process is, roughly:
I absolutely do think this is a feature with the potential for a high complexity surface area. In signing off on this, I'm not ignoring or discounting the complexity; rather, I'm evaluating the trade-off between that complexity and the value it provides. |
Contributor
Author
jackh726 commented 29 days ago
|
Hi all, I haven't really posted much here. As I said before, I've wanted to try to stay out of the discussion and let the discussion continue without my influence. However, given that @nikomatsakis has proposed FCP on this, I figured I'd put on my types team hat and give my reasonings for why my box is checked. I'd also like to go ahead and give my more general opinions on this feature and its stabilization. First, I'd like to say that I just put up a PR that slightly changes the rules for the self-outlives error (#100382). Particularly, GATs in the inputs to trait methods now factor into the lint. I'll update the initiative repo sometime soon to reflect this. (This is relevant to one of the questions for the types team.) Second, I'd like to also note that I'm planning to try to improve errors you might get when using higher-ranked trait bounds with GATs today. These will eventually be allowed, but that will come later. (I will discuss this below.) Answering questions for T-typesAre we confident we can address the known shortcomings without backwards incompatible changes?I am confident, yes. To me, there are a few shortcomings to the current implementation and design:
Are we confident that we have the right rules for required bounds?With my above-mentioned PR, yes. I think GATs in the arguments are a fine extension to the current rules. As I also mention above, I think not extending to where clauses also makes sense given the lack of implied-bounds for types appearing in where clauses. General thoughts on stabilizationFirst off, I'm biased here. But, I'm biased in the best way possible: I've been working on this feature for over a year now. I know the bugs; I know the limitations. But I also know how much excitment and readiness for the feature there is. And I know that people use GATs on nightly, despite the limitations listed above. If the problem for stabilization is that the feature is too complex for Rust, then I think that's maybe an incomplete argument. It is indeed a complex feature, but with that complexity comes power. And I have yet to see a comprehensive response for an alternative to GATs to solve the problems that GATs solve. It's very disheartening to see this argument continue to be made without a concrete counter solution in place. If the problem for stabilization is the design of certain parts of the feature: I get this. As I list above, I know they exist. However, the design today is in a place where we follow existing Rust-isms most closely (No new syntax position for the If the problem for stabilization is the bugs: I also get this. But I think the feature is worth stabilizing despite these. The biggest bug, I think, is the HRTB Finally, if the concern is that, if we stabilize, we're going to forget about GATs and leave them in the current state, I think that's maybe a little misguided. Sure, there are bugs (several unsound bugs, to add) that have been sitting in the issue tracker for years, so some might be wary of adding more surface area to the Rust langugage - and implementation. However, this area, the type and trait system, is precisely what the types team was formed to oversee and work on. Some of the limitations above - HRTB implied bounds, the borrow checker - are questions the types team is actively working on as part of efforts to "formalize" the type checker and trait solving. The GATs initiative isn't going away either - we will continue to fix bugs and consider design improvements (such as the trait bound syntax). I personally feel confident that progress will continue to move forward. Final thoughtsAll this being said: Let's stabilize GATs. Yes, there are limitations currently. But they solve many problems Rust users have - despite those limitations. And, there are multiple people that are working together to remove the limitations that do exist now. Letting GATs sit unstable doesn't help anyone. People that are on nightly because of GATs must stay on nightly. People who can't use GATs because they aren't stable still can't use them. The feature isn't going to change in backwards-incompatible ways. And the feature doesn't move any faster than if it were stable. |
Member
nrc commented 28 days ago
@joshtriplett thanks for expanding! |
Member
nrc commented 28 days ago
I'm glad there has been some progress here. Points 2, 3, and 5 still sound like there are significant unknowns though and I'd much rather these were addressed before stabilisation.
I think this is the wrong way to approach this question. It is assuming that we must address the expressivity question in some way and that we need to decide how to address it. I think the question ought to be whether we need to address the expressivity question at all. In any language there are going to be use cases which are hard/ugly/impossible to express. The only 'solution' to that is to grow the language until it is infinitely large, which has obvious downsides. We have to draw a line somewhere, maybe that line includes GATs (obviously I don't personally think so, but it does seem a reasonable opinion that it should) but I'd love for us to be talking explicitly about where we draw the line and why, rather than pretending that there is no line (or no need for a line).
I don't think there is an issue with having some well-defined work still to do (we often stabilise features with known further work items) or that there are some minor bugs (it's software!), but rather that there are more issues, and that these issues seem larger and to be more unknown than we usually expect when stabilising a feature. That makes it feel like stabilisation is risky - we could be trapping ourselves out of a better solution. It might be that you are more certain about things than the write up suggests. I get the impression you are trying to be exhaustive and unbiased in your descriptions of the remaining issues. If you think that there what we currently have is the best we could have and the alternatives are just worse, rather than needing more work to decide if they are worse or better, then that would actually make me feel better about the stabilisation, because it means there is a much smaller space for exploration going forward. |
Contributor
c410-f3r commented 28 days ago
|
Kindly asking as an user that wants to use a feature: Is it possible for the remaining team members to cast their votes? With 160 comments and 3 months of discussions, I guess there are already enough arguments from both sides to at least reach a decision based on majority. |
Contributor
eggyal commented 28 days ago
Perhaps I'm speaking out of turn, but just because you aren't seeing activity on this issue doesn't mean that it isn't being actively discussed amongst the team. On Zulip, for example, there have been active conversations including with those who have not yet cast a vote. This is a big change to the language (perhaps the biggest since 1.0), and once stabilised there's really no going back. I think it's right and proper that people are given the time and space to really think through all of its ramifications before they reach a decision. That said, I am also very keen to see this stabilised. :) |
Contributor
slanterns commented 28 days ago
I agree, better to leave it to the lang team's independent decision process than put it in urgency. |
Contributor
Author
jackh726 commented 28 days ago
No, there are not significant unknowns. These are the limitations and where the design is. I'm not sure what you expect to address. For the trait bounds, these are necessary for soundness. For the trait bound syntax, these are the status quo of stable Rust syntax. For the difference between associated type bounds and where clauses, I have yet to see an alternative proposal here. Any changes to 2 and 3 would be extensions to the current feature. And change for 5 could be done in a backwards-compatible manner or over an edition - but again, have yet to see alternatives proposed.
I hate to seem a bit rude here, but this question has been discussed. It was discussed on the RFC; it's been discussed over the years since the RFC during implementation; it's been discussed in this thread. The lang team checking their boxes on the proposed FCP affirm that this feature fits into the language. If you want to argue against it, then you should give concrete examples where this feature does not fit into Rust. I'm tired of general "this is too complicated for Rust" arguments that lead us in circles.
Whether the limitations I listed are "large" is a bit subjective. But they are not unknown - we know exactly what they are and what potential future extensions might look like. Importantly, those extensions are backwards-compatible with the current state.
Okay, then let me be crystal clear: I am overwhelmingly certain that stabilizing GATs is the correct next action. As I said, in my post above, the limitations I describe above can be removed in a backwards-compatible manner and do not undermine the feature itself. There are no proposed or theoretical alternatives to this feature, only extensions. |
Member
nrc commented 27 days ago
OK, thanks, that is clear and I misunderstood your earlier description (there were a lot of "might"s and "alternative"s there, but I think you were just trying to be complete).
I think we must agree to disagree here. In any case, this is something I expect from the lang team rather than you; you've done an excellent job of communicating the status of the project and the strengths of the proposal.
Thanks for clarifying! |
PureWhiteWu commented 16 days ago •
|
Hi, all. As a real production use case for GAT(and TAIT), we have open-sourced our internal RPC framework Volo: https://github.com/cloudwego/volo and its middleware abstraction layer Motore(which is greatly inspired by Tower): https://github.com/cloudwego/motore. Here's a feedback for that. We have also met some problems when GAT is used together with some other features, and we have submitted an issue for that #100267 (which caused our docs fail to build). And besides that, we have also encountered #92096 and #64552. Though there may still be some bugs with GAT, GAT (together with TAIT) are super useful features and we are quite looking forward for their stabilizations. Thank you for all your great works! |
Contributor
ohsayan commented 13 days ago
|
Hey everyone, I am going to chime in here regarding the stabilization of GATs. Our use-case at Skytable is very much similar to the example highlighted in RFC 1598, where we want to implement a single trait for a common group of data structures. GATs will enable us to do "hot swapping" of the underlying DS to test for performance impacts. Our goal is to have something like below (please note this is just an example): To test new structures, we'd simply implement |
Contributor
est31 commented 12 days ago
|
Your PR has been filed before the change, so will likely require modifications in order to comply with the new rules. I recommend you to:
That's it! The If you have any questions, feel free to drop by the zulip stream, or ping me directly in this PR's thread. Thanks! |
Hello, I'm writing this comment in this stabilization PR to notify you, the authors of this PR, that Collaborator
rustbot commented 9 days ago
|
Some changes occurred in src/tools/rustfmt |
Contributor
rbtcollins commented 8 days ago
|
Another data point, good or bad. I write some crates that do API client things, but users of the crate don't want to have to choose between async-std and tokio - so GATs would help in allowing a trait abstracting over those ecosystems, or any new one that arises. Today we've avoided anything particularly troublesome, though abstracting over async |
|
Based on my review of the comments in this thread, and also based on the T-lang design meeting discussion today, I am inclined to move forward with the plan to stabilize GATs (both lifetime and type) as they stand today. There are some subpar developer experiences in the implementation as it stands today. Some of those are bugs that we should able to address, and others will require longer term work on either the compiler's static analyses or on language features. Nonetheless, at this point I believe that the current feature delivers value for Rust developers and fits the character of the Rust language overall. @rfcbot reviewed |
added the final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. label
rfcbot commented 8 days ago
|
|
This is now entering its final comment period, as per the Contributor
zesterer commented 4 days ago
|
I've just encountered a minor bug with cyclical obligation resolution that I've not seen mentioned elsewhere in #101406 . It seems to be related to GATs, although I don't have the domain knowledge to confirm this. I'm personally of the view that it shouldn't block stabilisation (not that I have the authority to make such a call) given that it only seems to happen for code that would otherwise fail to compile anyway, but I thought it worth mentioning. |
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK