How to encrypt a file in Rust (Using streaming AEAD encryption)
source link: https://kerkour.com/rust-file-encryption/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
How to encrypt a file in Rust (Using streaming AEAD encryption)
Wed, Dec 8, 2021
Choosing the right encryption algorithm
When you want to encrypt data, you may face a problem: how to encrypt files or data streams that don’t fit in memory?
You could simply split them into small chunks and encrypt each fragment individually. Unfortunately, it’s not that simple.
An attacker could modify or reorder the chunks making decryption impossible.
This is where streaming AEAD encryption comes into play, to guarantee that the entire data stream has not been modified, corrupted, or reordered. You can read more the benefits of stream ciphers in libsodium’s documentation (but we won’t use it).
In the first part of this post, we are going to see how to encrypt small files that can fit in memory, and, in the second part, how to use our AEAD cipher in stream mode to encrypt larger files or data streams that can’t be encrypted in one operation.
We will use XChaCha20-Poly1305 because I particularly appreciate its simplicity and speed. That being said, the RustCrypto organization uses a trait for all their AEAD implementations. Thus, replacing XChaCha20-Poly1305 by AES-256-GCM (for example) is just a few keystrokes away.
As always, the code is on GitHub: github.com/skerkour/kerkour.com (please don’t forget to star the repo 🙏).
Setup
[package]
name = "rust_file_encryption"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
chacha20poly1305 = { version = "0.9.0", features = ["stream"] }
anyhow = "1.0"
rand = "0.8"
In both cases, the first step is to securely generate a key and a nonce. XChaCha20-Poly1305’s key is 32 bytes long, and its nonce is 24 bytes long.
let mut key = [0u8; 32];
let mut nonce = [0u8; 24];
OsRng.fill_bytes(&mut key);
OsRng.fill_bytes(&mut nonce);
Encrypting small files
Encrypting small files that can be read entirely in memory is pretty straightforward:
fn encrypt_small_file(
filepath: &str,
dist: &str,
key: &[u8; 32],
nonce: &[u8; 24],
) -> Result<(), anyhow::Error> {
let cipher = XChaCha20Poly1305::new(key.into());
let file_data = fs::read(filepath)?;
let encrypted_file = cipher
.encrypt(nonce.into(), file_data.as_ref())
.map_err(|err| anyhow!("Encrypting small file: {}", err))?;
fs::write(&dist, encrypted_file)?;
Ok(())
}
Decrypting small files
The decryption is the same. You simply need to call decrypt instead of encrypt.
fn decrypt_small_file(
encrypted_file_path: &str,
dist: &str,
key: &[u8; 32],
nonce: &[u8; 24],
) -> Result<(), anyhow::Error> {
let cipher = XChaCha20Poly1305::new(key.into());
let file_data = fs::read(encrypted_file_path)?;
let decrypted_file = cipher
.decrypt(nonce.into(), file_data.as_ref())
.map_err(|err| anyhow!("Decrypting small file: {}", err))?;
fs::write(&dist, decrypted_file)?;
Ok(())
}
Encrypting large files
Encrypting larger files that can’t fit in memory either because they weigh many GBs, or because you want to limit your resources usage, is where all the fun is.
For that, we need to use our AEAD cipher in stream mode.
fn encrypt_large_file(
source_file_path: &str,
dist_file_path: &str,
key: &[u8; 32],
nonce: &[u8; 19],
) -> Result<(), anyhow::Error> {
let aead = XChaCha20Poly1305::new(key.as_ref().into());
let mut stream_encryptor = stream::EncryptorBE32::from_aead(aead, nonce.as_ref().into());
Then we need a buffer to read the file chunk by chunk.
const BUFFER_LEN: usize = 500;
let mut buffer = [0u8; BUFFER_LEN];
let mut source_file = File::open(source_file_path)?;
let mut dist_file = File::create(dist_file_path)?;
And finally, the magic.
First, we read a chunk of data from the file we want to encrypt.
If the number of bytes read is equal to the size of the buffer, it means that we didn’t reach the end of the file. Thus, we call encrypt_next on our stream_encryptor.
Otherwise, a read_count < BUFFER_LEN means that we reached the end of the file, and we need to call the special method encrypt_last on our stream_encryptor.
loop {
let read_count = source_file.read(&mut buffer)?;
if read_count == BUFFER_LEN {
let ciphertext = stream_encryptor
.encrypt_next(buffer.as_slice())
.map_err(|err| anyhow!("Encrypting large file: {}", err))?;
dist_file.write(&ciphertext)?;
} else {
let ciphertext = stream_encryptor
.encrypt_last(&buffer[..read_count])
.map_err(|err| anyhow!("Encrypting large file: {}", err))?;
dist_file.write(&ciphertext)?;
break;
}
}
Ok(())
}
Note that encrypt_next and encrypt_last allocate. Thus, our function uses roughly BUFFER_LEN * 2 bytes of memory. We could have used encrypt_next_in_place and encrypt_last_in_place to avoid the allocation.
Decrypting large files
Decrypting the stream is not exactly the same as encrypting it. A small tweak is needed.
fn decrypt_large_file(
encrypted_file_path: &str,
dist: &str,
key: &[u8; 32],
nonce: &[u8; 19],
) -> Result<(), anyhow::Error> {
let aead = XChaCha20Poly1305::new(key.as_ref().into());
let mut stream_decryptor = stream::DecryptorBE32::from_aead(aead, nonce.as_ref().into());
Then comes the tweak. Notice that our buffer is no longer 500 bytes long, but 516.
const BUFFER_LEN: usize = 500 + 16;
let mut buffer = [0u8; BUFFER_LEN];
let mut encrypted_file = File::open(encrypted_file_path)?;
let mut dist_file = File::create(dist)?;
Why a buffer size of 500 + 16 bytes? Because XChaCha20-Poly1305 is an AEAD cipher and appends a 16 bytes authentication tag to each encrypted message. Thus, each encrypted chunk has a size of 516 bytes: 500 bytes for the encrypted piece of data, and 16 bytes for the authentication tag.
loop {
let read_count = encrypted_file.read(&mut buffer)?;
if read_count == BUFFER_LEN {
let plaintext = stream_decryptor
.decrypt_next(buffer.as_slice())
.map_err(|err| anyhow!("Decrypting large file: {}", err))?;
dist_file.write(&plaintext)?;
} else if read_count != BUFFER_LEN {
let plaintext = stream_decryptor
.decrypt_last(&buffer[..read_count])
.map_err(|err| anyhow!("Decrypting large file: {}", err))?;
dist_file.write(&plaintext)?;
break;
}
}
Ok(())
}
The code is on GitHub
As always, the code is on GitHub: github.com/skerkour/kerkour.com (please don’t forget to star the repo 🙏).
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK