I switched jobs all over again. I’m working in a data company called Shalion, where we provide insights about retail products online. If you’re brave enough to handle Snowflake’s quirks, we’re hiring… I’m working on revamping our permission grant system from permifrost to a terraform managed approach. We also decided to embrace a 2 layer permission grant, using Access Roles (AR) and Functional Roles (FR). The former has access to the resources with the typical permissions (SELECT, USAGE, CREATE)…, while the latter is the one that we assign to people and just groups access roles. Read more

2025 ha sido un año bastante loco. Pasé la mayor parte del año sentando las bases de una startup en el mundo de la seguridad, haciendo frontend, backend, mucho devops y… poco data. Me pilló de lleno la era de la AI y como siempre, el campo ajeno es más verde. Viví la burbuja de v0 y mi trabajo respecto a este, fue bastante miserable. El proyecto a medida que crecía dejaba de ser funcional. Un cambio manual que arreglase el mismo era incompatible. Read more

For the past seven years, I worked in data, and I have mixed feelings about it. I still believe data is the most important part of any app, but it’s meaningless without the app itself. Now that I’m working at a startup, I’ve decided to focus on building things. To start, I revisited one of my older projects: a PDF parser about professor designations in the Canary Islands, where one of my best friends works as a teacher. Read more

As my company undergoes layoffs, I’m back on the job hunt. While I’m in the field of data, I often find myself missing the hands-on experience that comes from personal projects. I realized that I’m not practicing all the skills I need. During a recent interview, I was asked about my experience with sending reports via email—something I hadn’t done in a few years. That got me thinking: could I turn this into a pet project? Read more

Over the years, I’ve been using MacOS at work and Ubuntu at home for my development tasks. However, my Lenovo P1 Gen 3 laptop didn’t work well with Linux, leading to frequent issues with the camera and graphics (screen flickering, I’m looking at you, and it hurts). I’ve triend Windows Subsystem for Linux (WSL) but it was quite bad to be honest. But as I’ve heard of WSL2 and WSLg, I decided to give it another shot. Read more

Exploring databricks SQL usage At my company, we adopted databricks SQL for most of our users. Some users have developed applications that use the JDBC connector, some users have built their dashboards, and some users write plain ad-hoc queries. We wanted to know what they queried, so we tried to use Unity Catalog’s insights, but it wasn’t enough for our case. We work with IOT and we are interested in what filters they apply within our tables. Read more

Well, I’m facing a huge interesting case. I’m working at Wallbox where we need to deal with billions of rows every day. Now we need to use Spark for some Kafka filtering and publish the results into different topics according to some rules. I won’t dig deep into the logic except for performance-related stuff, let’s try to increase the processing speed. When reading from Kafka you usually get 1 task per partition, so if you have 6 partitions and 48 cores you are not using 87.5 percent of your cluster. That could be adjusted with the following property **minPartitions.** Read more

After spending several weeks working on a ksql solution to filter billions of events and determine their destination topic, I was disappointed to find that it did not live up to my expectations. I had hoped for a more robust product that would align with our needs. Previously, we utilized a similar filter in Spark, incurring traffic costs for both Confluent and AWS. With kSQL, the advantage was that we could avoid paying for AWS traffic. Read more