The Algorithmic #5: Rust in the Dataverse

Rust is Coming in Hard on Data

In recent years, Rust has come to be a force to be reckoned with in the world of data. With technologies like Qdrant, the adoption of Arrow into the Apache Foundation and user-facing technologies like Polars, Rust has made a big name for itself in a short amount of time.

Here’s a brief list of Data-related technologies that are built in Rust, or have been moving in a Rusty direction:

• Dozer, a platform for rapidly deploying data products by converting data sources to ready-made APIs
• Fossil’s entire data platform was recently rewritten in Rust
• The Apache Arrow DataFusion SQL engine is all Rust all the time
• Both Arroyo and Fluvio are Rust-based engines for handling streaming data at scale
• delta-rs provides a low-level Rust API for accessing Delta Lake (without any burdensome JVM dependencies)
• Cube is a Rust-based semantic layer for data systems, which makes life easier on developers who build applications that interface with data stores
• Blaze is a Rust-based accelerator for Apache Spark that takes execution plans from DataFusion and uses Spark to carry them out
• Databend is a cloud data warehouse (similar to Snowflake) built in Rust.

Most certainly, Rust has proven itself in the world of data. However, you might be asking yourself why it’s suddenly so popular?

Why Use Rust for Data

Let’s get the two easy ones out of the way first: speed and memory efficiency. Rust was designed for performance comparable to C. Also, one of its central (and perhaps most controversial) features, lifetimes, provides automatic and aggressive memory management without the overhead of a garbage collector.

But those features aren’t unique to Rust. C, C++, Go and even FORTRAN can perform comparably to Rust. So what makes Rust special? I would argue it’s Rust’s type system. Rust’s type system combines three features that help to make it especially useful for data:

• Strong static typing: If you make a type error in your code, it won’t compile. Period. Type errors are a major source of issues for data technologies and Rust makes them (almost) impossible by construction.
• Algebraic data types: the ability to define and deconstruct sum types (“enums” in Rust parlance), together with pattern matching, allow users to write code where invalid states are logically impossible (or throw an explicit error)
• Traits: Rust’s secret sauce for staying flexible, despite all the restrictive mechanisms it throws at you, is traits. They define patterns of common behavior and even incorporate generics to some extent.

These three features, together with Rust’s strict mechanisms for enforcing memory safety (which also happen to make concurrency much simpler), make it much easier to express data technologies in Rust, while still maintaining a high degree of performance and reliability.

The Rust ML/AI Ecosystem

The Rust ecosystem for machine learning currently has some strong offerings:

• Polars for blazingly fast operations on tabular data (similar to Pandas in Python)
• ndarray for operations on n-dimensional arrays of data (similar to NumPy in Python)
  nalgebra for fast linear algebra (similar to SciPy‘s matrix module in Python)
• Charming and Plotters for data visualization, as well as Plotly bindings, and VegaFusion (similar to Matplotlib and Seaborn in Python)
• argmin for a wide variety of fast optimization algorithms (similar to SciPy‘s optimization module in Python)

Huggingface in particular has written several libraries in Rust, including:

• text-embeddings-inference for inference on text embedding models
• Tokenizers for fast, state-of-the-art tokenization in NLP pipelines
• Candle for high-performance neural network operations on CPU or GPU

And if you want to work with neural networks, you have no shortage of options:

• tch-rs provides Rust bindings to the PyTorch C++ API
• TensorFlow provides Rust bindings to the TensorFlow C API
• burn is a flexible, full-featured neural network library in pure Rust, offering multiple computational backends (including WebGPU for in-browser neural network applications)
• dfdx is another full-featured neural network library in pure Rust with the interesting twist that it checks the shapes of tensor operations at compile-time
• If you’ve ever had to debug neural network code, you’ll know that having the compiler tell you when your tensors don’t match can be a huge time-saver

What’s Missing

As impressive as Rust’s ecosystem is, there are some glaring omissions:

• A framework for data-focused distributed computations like Spark, Ray, or Dask
  • Ballista looked very promising to replace Spark, but is currently unmaintained and unlikely to be revived
  • Daft allows Rust to perform distributed computing through Ray, but this means rebinding your code into Python via Pyo3
• Specialized libraries with all the ML goodies from recent research, such as xformers or bitsandbytes
• A general-purpose ML library that matches the breadth of scikit-learn
  • Linfa comes the closest in this regard, but doesn’t have the same range of available features
• Neural network acceleration frameworks like DeepSpeed or Megatron-LM
  
• Most of the MLOps side of the data science stack, like model serving
  

Remember though, the Rust ecosystem is relatively young and under rapid development. This means that there are missing technologies and features, but it also means that as the ecosystem develops, the technologies and features that get developed are top-notch.

Roadblocks for Rust

As exciting as a Rust-powered future for data is, that future is far from being a certainty. There are some challenges the Rust community will have to overcome before it can live up to its potential for ML.

Recently Fractured Community

Over the past year, the Rust community has seen a lot of drama, most of it centered around the Rust Foundation. The full effect of these events has yet to be seen, but there are plenty of Rustaceans who are ready to jump ship if a viable alternative becomes available. If that happens, the Rust ecosystem may be DOA.

Momentum for ML Projects

Thus far, there have been several attempts at building high-quality Rust tooling for machine learning and none of them have gained traction (Linfa may make it, but it’s still too early to tell). It’s difficult to keep a machine learning project going in Rust because it requires a very specific set of skills to contribute to such a project. A contributor needs to have a strong software engineering background to make it past Rust’s steep learning curve, while also having a deep understanding of the machine learning algorithms being implemented. When you factor in the fact that the contributors to those projects are mostly working in their free time, it’s not hard to see why it’s difficult for a machine learning project to gain momentum.

The Rust Ecosystem is Still Maturing

Rust has a fantastic ecosystem, don’t get me wrong. But even the language itself has rough edges that require clever workarounds. Many of the excellent Rust packages available work well as long as you stay on their “happy path.” However, there are still major missing features from several Rust libraries.

That being said, the Rust community manages to churn through their to-do lists with impressive speed. So if the feature you need is missing today, it’s worth checking back in a couple of months to see if it gets addressed.

Reasons to Be Optimistic

At the end of the day, Rust is a language that its users love. There’s a reason “rebuild it in Rust” has become a meme. The Rust community has a lot of enthusiasm for the language and are constantly looking for new, cool things that can be done (or redone) in Rust.

Add to that the strong foundation of existing packages for core Rust development and the incomplete-but-still-impressive offerings Rust makes for Data. It’s too soon to tell where Rust is heading with Data and ML, but it’s definitely not something I would bet against.

It’s also worth mentioning that during their recent announcement of Grok, xAI revealed high-level details about their machine learning tech stack. Not only are they using Rust-based Qdrant as their vector database, but Rust is one of their central backend languages. (Fingers crossed Elon Musk decides to open-source some of the tooling xAI has developed internally.)

Could Rust Replace Python for Machine Learning

Short answer, no. The reasons for using Python and the reasons for using Rust are drastically different. Very few data analysis or machine learning engineers are going to want to fire up a Rust notebook and fight the borrow checker just to explore their data or prototype a quick model.

I foresee Rust’s machine learning packages having Python hooks (like Polars). Then the engineering and underlying implementations are done using Rust, with the analyses and models being produced in Python.

Looking Forward

There’s no doubt that Rust has massive potential for machine learning. It has several strong data technologies to work alongside and many of the core tools needed for machine learning workflows. However, the future for Rust with machine learning is uncertain.

The potential for a future of bulletproof machine learning servers and backends, with powerful tools that data analysis and machine learning engineers can call from Python, is certainly alluring. And with many of the foundational MLOps technologies still in their infancy, that future is not only alluring, but plausible.

There are a lot of reasons for people working in AI and machine learning to be excited right now. Hopefully, Rust is now one of your reasons.