rust-analyzer in 2021
A lot has happened this year, so we want to take a brief look back at what we have achieved and what has yet to come.
Unfortunately, we did not manage to make rust-analyzer an official Rust project in 2021. But if all goes well this should change at the start of the coming year.
We started the year out by trying out a different meeting style from our usual weekly syncs on Zulip. We now have six-weeks sprints, with a steering meeting between them, where the working group gets together and discusses what topics and issues to focus on next.
Probably one of the biggest improvements that landed this year was attribute
proc-macro support, implemented in #9128. Attributes are used pervasively in some Rust codebases, usually expanding to enough new items (functions,
impl blocks etc.), and are essential in making the IDE work without significantly degrading the user experience. It took us roughly four months until we felt comfortable enough to enable their expansion by default, the main reason being the amount of code that had to be adapted to properly support them.
While this feature mostly works nowadays, there is still one big problem we are facing, that of how attributes can interact with incomplete code. This is described in issue #11014 in more detail, but attribute proc macros fail to expand when they encounter syntax errors, as is often the case when the user is typing. Of course, function-like proc macros are affected in a similar way. We are still unsure about how to address this problem, so if you have any thoughts on the matter please comment on the linked issue.
Attributes aside, a lot of improvements landed for proc-macros in general. We enabled proc-macros by default, started loading proc-macros asynchronously and added support for multiple proc-macro ABIs, a finicky feature. The Rust proc-macro ABI is deliberately outside of usual stability guarantees. It constantly evolves, but with every change it breaks our proc-macro server. To counteract this we now try to at least support ~3 ABI versions which nominally correspond to the latest stable, beta and nightly toolchains. In practice, we support much older versions: 1.47 and later, at this moment.
It’s worth pointing out that parts of our proc macro infrastructure are used by IntelliJ Rust. It’s a great example of cross-pollination of ideas between the two projects — our
proc_macro_srv is based on a design originally implemented in IntelliJ Rust!
Rust offers the ability to define new items inside of functions, consts and statics, making them effectively unnameable outside of these items. This idiom is seldom seen but is still useful at times. This is another piece of the language that is tricky to support, as an IDE wants to be lazy and only look at the big picture to understand a file’s structure. Unfortunately, local items require us to look inside function bodies and the like.
We managed to switch our type representation to the one used by chalk, avoiding the need to convert between them every time we invoke the trait solver.
We started working on basic const generics support, tracking issue #8655. This is one of the major pieces left for us to properly support, but also one of the more difficult ones. This problem becomes more and more apparent as libraries are switching over to using them. A notable example being
nalgebra, which rust-analyzer is completely confused by.
For an ideal experience we would need to evaluate expressions in const generic position like rustc does. Unlike rustc though, we are unable to use
miri. The reason for that is the fact that we do not share the internal data structures that the compiler and miri make use of, nor do we plan to do so in the near future. Because of this, we are required to build our own basic evaluator.
This isn’t the only problem we have. As it currently stands, the majority of rust-analyzer’s type checking codebase doesn’t even know about const generics (or lifetimes), and just assumes that only type parameters exist.
To put it short, the assists like to take existing syntax in a file, modify it slightly, then paste back the result, changing only a few parts of the code. What most assists did before was to reconstruct the output syntax trees out of the original tree nodes, with the changes inserted or left out. This is unfortunately a tedious process depending on the type of syntax node that has to be edited. These mutable immutable trees now allow us to instead mutably clone a tree, modify it in-place and paste this modified tree back as is.
This does not come without its own problems of course, with the main one being that iterating while mutating, which is now possible, will now panic or produce an incorrect tree.
The final releases of the year brought two important changes to the way the language server binary is acquired. Until now, the extension called the GitHub API to find a matching release, and downloaded the server from there. In addition, if you opted in to the nightly channel, every day the extension would search for an updated VSIX.
While this suited us well for a long time, it had some downsides:
edge cases like MITM HTTPS proxies with untrusted certificates never worked
not being able to replace running executable on Windows (in case you had two running instances) has been a major source of complaints
Code checks for extension updates automatically, but the server was downloaded on activation. Many users had an unpleasant surprise when opening a Rust project without a working Internet connection.
GitHub Actions has been increasingly unreliable lately causing the nightly builds to fail very often. This broke the extension for users running the nightlies.
the download and update code brought a lot of non-essential complexity and even exposed a Node.js bug
the required client-side dependencies increased the surface for supply chain attacks
So when the Marketplace added support for platform-specific and pre-release extensions, we starting using it with #11053, #11071 and #11106. On a related note, we also replaced the client-side unit testing framework with a bit of custom code, dropping our transitive dependencies by the dozens.
Unfortunately, the update changes exposed some issues with both the Code Marketplace and Open VSX (used by open-source builds of Code). This is discussed in detail here, the gist being that you currently need to take manual action when installing or updating the extension. The two issues have been reported upstream, so we hope they will be fixed soon.
We now support standalone Rust files (#8955), so just opening a single file should allow you to use most of rust-analyzer’s functionality that is not reliant on cargo.
We also improved and added a bunch of smaller features, some noteworthy ones are:
An impressive ~37 new assists have been implemented, some by the main contributors, but most by newcomers:
We made a bunch of progress this year, finishing up work on important functionality pieces as well as quality of life changes. But there is still a lot more to be done, as a quick look at the issue tracker may tell.
A big thanks to everyone supporting the project, be it through donations, contributions or cheers. We wish you a fortunate next year 🎉.