I’ve been in the process of coding a web crawler in Rust and faced a problem with keeping an in-memory to-visit queue. The idea is to have a collection of nodes to visit, which discards duplicates, with minimal memory overhead.
The Problem
A problem like this could be solved …
I am an Italian Software Engineer with a strong foundation in computer science and a passion for solving complex problems. I am interested in a various topics, including algorithms, distributed systems, databases, and information retrieval. Feel free to read about myself or to look at my posts.
I am an …
I studied LSM trees at university and after encountering them twice in Designing Data-Intensive Applications and Database Internals I decided to implement something in Java.
The idea behind this project is not to provide the most efficient implementation ever, but to experiment with storing data on disk, any suggestions are …