Published:
Hello there 👋 Attending large machine learning conferences is both exciting and challenging. They can be expensive and not accessible to everyone, and I’ve certainly felt the fear of missing out when unable to attend in the past.
mle-monitor: A Lightweight Experiment & Resource Monitoring Tool 📺
Published:
“Did I already run this experiment before? How many resources are currently available on my cluster?” If these are common questions you encounter during your daily life as a researcher, then mle-monitor is made for you. It provides a lightweight API for tracking your experiments using a pickle protocol database
mle-scheduler: A Lightweight Tool for Cluster/Cloud VM Job Management 🚀
Published:
“How does one specify the amount of required CPU cores and GPU type again?” I really dislike having to write cluster job submission files. It is tedious, I always forget something and copying old templates feels cumbersome. The classic boilerplate code problem. What if instead there was a tool that would completely get rid of this manual work?
Published:
In this blog we implement the Centered Kernel Alignment (CKA) metric used to compare the representations of different neural network layers for the same or two separate networks. CKA measures the similarity of representations at different network layers of the same or different networks.
mle-hyperopt: A Lightweight Tool for Hyperparameter Optimization 🚂
Published:
Validating a simulation across a large range of parameters or tuning the hyperparameters of a neural network is common practice for every computational scientist. There are a plethora of open source tools that implement individual algorithms, but many of them are either combersome to set up and log or follow diverse syntax, which makes it hard to easily wrap them.
mle-logging: A Lightweight Logger for ML Experiments 📖
Published:
There are few things that bring me more joy, than automating and refactoring code, which I use on a daily basis. It feels empowering (when done right) and can lead to some serious time savings. The motto: ‘Let’s get rid of boilerplate’. One key ingredient to my daily workflow is the logging of neural network training learning trajectories and their diagnostics (predictions, checkpoints, etc.).
Published:
“So why should I switch from <insert-autodiff-library> to JAX?”. The classic first passive-aggressive question when talking about the new ‘kid on the block’. Here is my answer: JAX is not simply a fast library for automatic differentiation. If your scientific computing project wants to benefit from XLA, JIT-compilation and the bulk-array programming paradigm – then JAX provides a wonderful API.
Published:
Most learning curves plateau. After an initial absorption of statistical regularities, the system saturates and we reach the limits of hand-crafted learning rules and inductive biases. In the worst case, we start to overfit. But what if the learning system could critique its own learning behaviour?
Published:
Metaphors are powerful tools to transfer ideas from one mind to another. Alan Kay introduced the alternative meaning of the term ‘desktop’ at Xerox PARC in 1970. Nowadays everyone - for a glimpse of a second - has to wonder what is actually meant when referring to a desktop. Recently, Deep Learning had the pleasure to welcome a new powerful metaphor: The Lottery Ticket Hypothesis (LTH).
Published:
The iPad is a revolutionary device. I take all my notes with it, read & annotate papers and do most of my conceptual brainstorming on it. But how about Machine Learning applications? In todays post we will review a set of useful tools & venture into the love story of the iPad Pro & the new Raspberry Pi (RPi).
Published:
JAX, Jax, JaX. Twitter seems to know nothing else nowadays (next to COVID-19). If you are like me and want to know what the newest hypetrain is about - welcome to todays blog post!
Published:
Last week Kaggle announced a new challenge. A challenge that is different - in many ways. It is based on the Abstraction and Reasoning Corpus & accompanied by a recent paper by Francois Chollet.
Published:
2019 - What a year for Deep Reinforcement Learning (DRL) research - but also my first year as a PhD student in the field. Like every PhD novice I got to spend a lot of time reading papers, implementing cute ideas & getting a feeling for the big questions. In this blog post I want to share some of my highlights from the 2019 literature.
Published:
TL;DR: This blog post provides an overview of trends & events from the Cognitive Computational Neuroscience (CCN) 2019 conference held in Berlin. It summarizes the keynote talks and provides my perspective and thoughts resulting from a set of stimulating days. More specifically, I cover recent trends in Model-Based RL, Meta-Learning and Developmental Psychology adventures. You can find all my notes here.
Published:
Automatic Differentiation (AD) is one of the driving forces behind the success story of Deep Learning. It allows us to efficiently calculate gradient evaluations for our favorite composed functions. TensorFlow, PyTorch and all predecessors make use of AD. Along stochastic approximation techniques such as SGD (and all its variants) these gradients refine the parameters of our favorite network architectures.
Published:
Before starting to write a blog post I always ask myself - “What is the added value?”. There is a lot of awesome ML material out there. And a lot of duplicates as well. Especially when it comes to all the flavors of Deep Reinforcement Learning. So you might wonder what is the added value of this two part blog post on Deep Q-Learning? It is threefold.
Published:
In January I was considering where to go with my scientific future. Struggling whether to stay in Berlin or to go back to London, I got frustrated with my technical progress. At NeuRIPS I encountered so much amazing work and I felt like there was too much to learn until reaching the cutting edge. I was stuck. And then my former Imperial supervisor forwarded me an email advertising this new Eastern European Machine Learning (EEML) summer school.
Published:
In today’s blog post we discuss Representational Similarity Analysis (RSA), how it might improve our understanding of the brain as well as recent efforts by Samy Bengio’s and Geoffrey Hinton’s group to systematically study representations in Deep Learning architectures. So let’s get started!
Published:
Hola guapos! After finally deciding to stay in Berlin, I felt the desire to structure myself and to establish routines which are going to help me tackle the next phase of my life. Due to a fortunate visit to the National Gallery book store in London, I got to pick up Austin Kleon’s amazing piece of work “Steal Like an Artist”. A beautifully collected and visualized set of tricks to foster creativity.
Published:
Hey there! As some of you might know I have been quite actively contributing to the Data Science Barcelona GSE blog. Writing about technical topics and addressing a broad audience is challenging and fulfilling at the same time. I hope that this blog is going to help me learn to tell great narratives and influence people. So stay tuned!