I quit my job after the trial period because I didn’t find what I expected from a startup, nor did I get the benefits of an established company. It was a dealbreaker.
However, I will remain for an extra 3 months since there is a project I want to be involved in. I’ve been granted more flexibility, and I want to use this time to prepare myself for what’s next.
To sum up these six months:
- I worked as a data platform engineer on Snowflake—my first time using this technology. It feels a bit too “magical” for me; there are too many things I’d like to control but can’t (like warehouse overhead).
- The traditional established tools are just not for me. Airbyte doesn’t inspire confidence.
- Airflow is too complex for 90% of use cases, and most people don’t know how to properly build data workflows with it.
- Rewriting systems is fun; implementing them is not.
- AWS isn’t superior to its competitors, at least not across all of its services.
- Everybody is talking about Iceberg, but its interoperability is still in its infancy.
To be honest, it was a hard hit when Databricks bet on Python instead of Scala. Scala was a really productive language for me; I used it for a few years with a lot of success, mostly alongside Spark. I tried solving some minor problems with scala-cli recently, but it just didn’t click the same way.
Over the last few years, I’ve found myself deeply interested in systems and solutions. Sadly, the AI wave prevented me from exploring this path at my previous company, but I think it will be my main challenge for the coming years.
It’s a bit tough to navigate right now, especially seeing the massive rise of DuckDB in the data space, but at the end of the day, I really love programming at a deeper level.
I am writing this publicly because I fear that if I don’t, I will end up trying Python for the 3,203,010,312th time without success.