rust-analyzer in 2021

Probably one of the biggest improvements that landed this year was attribute proc-macro support, implemented in #9128. Attributes are used pervasively in some Rust codebases, usually expanding to enough new items (functions, impl blocks etc.), and are essential in making the IDE work without significantly degrading the user experience. It took us roughly four months until we felt comfortable enough to enable their expansion by default, the main reason being the amount of code that had to be adapted to properly support them. While this feature mostly works nowadays, there is still one big problem we are facing, that of how attributes can interact with incomplete code. This is described in issue #11014 in more detail, but attribute proc macros fail to expand when they encounter syntax errors, as is often the case when the user is typing. Of course, function-like proc macros are affected in a similar way. We are still unsure about how to address this problem, so if you have any thoughts on the matter please comment on the linked issue.

Attributes aside, a lot of improvements landed for proc-macros in general. We enabled proc-macros by default, started loading proc-macros asynchronously and added support for multiple proc-macro ABIs, a finicky feature. The Rust proc-macro ABI is deliberately outside of usual stability guarantees. It constantly evolves, but with every change it breaks our proc-macro server. To counteract this we now try to at least support ~3 ABI versions which nominally correspond to the latest stable, beta and nightly toolchains. In practice, we support much older versions: 1.47 and later, at this moment.

It’s worth pointing out that parts of our proc macro infrastructure are used by IntelliJ Rust. It’s a great example of cross-pollination of ideas between the two projects — our proc_macro_srv is based on a design originally implemented in IntelliJ Rust!

Finally, declarative macros also saw some love. We switched to an NFA parser which brings us closer to how rustc handles them, added basic support for macro 2.0 and type position macros.

Rust offers the ability to define new items inside of functions, consts and statics, making them effectively unnameable outside of these items. This idiom is seldom seen but is still useful at times. This is another piece of the language that is tricky to support, as an IDE wants to be lazy and only look at the big picture to understand a file’s structure. Unfortunately, local items require us to look inside function bodies and the like.

As such, finding a compromise took some work: #7336, #7359, #7366, #7375, #7426, #7431, #7466, #7480, #7481, #7482, #7485, #7518, #7525, #7541, #7544, #7554, #7555, #7557, #7559, #7561, #7568, #7575, #7627, #9244. The final piece, enabling it all, landed in #7614.

We managed to switch our type representation to the one used by chalk, avoiding the need to convert between them every time we invoke the trait solver.

This work was done over a bunch of different pull requests: #7804, #7813, #7814, #7816, #7823, #7826, #7833, #7870, #8016, #7998, #8001, #8018, #8038, #8136, #8190, #8139, #8309, #8327, #8856, #8921, #8938, #8856, and we were finally able to fully move over in #8419.

We started working on basic const generics support, tracking issue #8655. This is one of the major pieces left for us to properly support, but also one of the more difficult ones. This problem becomes more and more apparent as libraries are switching over to using them. A notable example being nalgebra, which rust-analyzer is completely confused by.

For an ideal experience we would need to evaluate expressions in const generic position like rustc does. Unlike rustc though, we are unable to use miri. The reason for that is the fact that we do not share the internal data structures that the compiler and miri make use of, nor do we plan to do so in the near future. Because of this, we are required to build our own basic evaluator.

This isn’t the only problem we have. As it currently stands, the majority of rust-analyzer’s type checking codebase doesn’t even know about const generics (or lifetimes), and just assumes that only type parameters exist.

Rowan, our concrete syntax tree crate, has been adjusted to allow creating mutable copies of the immutable syntax trees. For more in-depth information behind this, check out this issue.

To put it short, the assists like to take existing syntax in a file, modify it slightly, then paste back the result, changing only a few parts of the code. What most assists did before was to reconstruct the output syntax trees out of the original tree nodes, with the changes inserted or left out. This is unfortunately a tedious process depending on the type of syntax node that has to be edited. These mutable immutable trees now allow us to instead mutably clone a tree, modify it in-place and paste this modified tree back as is.

This does not come without its own problems of course, with the main one being that iterating while mutating, which is now possible, will now panic or produce an incorrect tree.

The final releases of the year brought two important changes to the way the language server binary is acquired. Until now, the extension called the GitHub API to find a matching release, and downloaded the server from there. In addition, if you opted in to the nightly channel, every day the extension would search for an updated VSIX.

While this suited us well for a long time, it had some downsides:

So when the Marketplace added support for platform-specific and pre-release extensions, we starting using it with #11053, #11071 and #11106. On a related note, we also replaced the client-side unit testing framework with a bit of custom code, dropping our transitive dependencies by the dozens.

Unfortunately, the update changes exposed some issues with both the Code Marketplace and Open VSX (used by open-source builds of Code). This is discussed in detail here, the gist being that you currently need to take manual action when installing or updating the extension. The two issues have been reported upstream, so we hope they will be fixed soon.

We now support standalone Rust files (#8955), so just opening a single file should allow you to use most of rust-analyzer’s functionality that is not reliant on cargo.

We also improved and added a bunch of smaller features, some noteworthy ones are:

An impressive ~37 new assists have been implemented, some by the main contributors, but most by newcomers:

We made a bunch of progress this year, finishing up work on important functionality pieces as well as quality of life changes. But there is still a lot more to be done, as a quick look at the issue tracker may tell.

A big thanks to everyone supporting the project, be it through donations, contributions or cheers. We wish you a fortunate next year 🎉.