-
A basic Rust streaming ETL pipeline
Python is the go-to language for ETL pipelines, widely used in data engineering due to its simplicity, vast ecosystem (Pandas, Apache Airflow, PySpark), and rich community support. However, Python’s performance limitations, especially with high-throughput or real-time data processing, make it less than ideal for streaming-heavy workloads. Rust has been popping its head in the world of data engineering and is becoming increasingly popular among data engineers. Rust’s efficiency in memory usage and speed is particularly beneficial for tasks like data ingestion, where handling large volumes of data quickly is crucial. And in terms of crates, Rust has great libraries like Polars, DataFusion, Arrow-rs, Ballista, among many others. …
-
Advent of Code Day 1: Rust versus Python
The end-of-year holiday season is always the time where Advent of Code comes out. It’s an annual event in December that provides a series of daily programming puzzles, designed to challenge and entertain programmers of all skill levels. It runs from December 1st to December 25th, with a new puzzle released each day. Solving the first part unlocks the second, which is often a twist or an extension of the first part. The puzzles typically start easier in the early days and become progressively more challenging as the event progresses. …
-
Traveling salesman problem
The traveling salesman problem is one of those annoying interview questions you sometimes get hurled at you, accompanied by “How would you solve this”? I remember a time when I was a fresh hatchling straight out of college, finding myself in that exact situation. I’ll spare you the details, but I can tell you I didn’t get particularly far 😉 …
-
A way of life
Everybody and their grandma probably tried their hand at an implementation of Conway’s Game of Life at some point, in one or the other language. It’s a straightforward and elegant little algorithm that can give you cool looking results, and a nice goal to work towards while learning a language. Drilling plain syntax or frameworks is incredibly boring. When learning any new language, having a concrete goal in mind besides “just learning it”, helps speed the process along and makes it more enjoyable by far. So, while learning Python, I took a swing at it. …
-
Prime skills?
I was always charmed by the idea and the practice of algorithm-heavy programming. The heavy lifting, making seemingly complex and daunting tasks run like **** of a shovel. …
-
How artificial intelligence is transforming software engineering
Automation of the automation process …
-
Through the trees...
Some wise man - Jim Rohn, for the record - once said: “Celebrate your achievements” …
-
Image classification: MLP vs CNN
In this article, I will make a short comparison between the use of a standard MLP (multi-layer perceptron, or feed forward network, or vanilla neural network, whatever term or nickname suits your fancy) and a CNN (convolutional neural network) for image recognition using supervised learning. …