Robert Tjarko Lange

Robert Tjarko Lange

Evolutionary Meta-Learning @SprekelerLab @TU Berlin

Blog posts

2021

Introducing mle-monitor: A Lightweight Experiment & Resource Monitoring Tool 📺

11 minute read

Published:

“Did I already run this experiment before? How many resources are currently available on my cluster?” If these are common questions you encounter during your daily life as a researcher, then mle-monitor is made for you. It provides a lightweight API for tracking your experiments using a pickle protocol database

Introducing mle-scheduler: A Lightweight Tool for Cluster/Cloud VM Job Management 🚀

20 minute read

Published:

“How does one specify the amount of required CPU cores and GPU type again?” I really dislike having to write cluster job submission files. It is tedious, I always forget something and copying old templates feels cumbersome. The classic boilerplate code problem. What if instead there was a tool that would completely get rid of this manual work?

All-CNN-C & Centered Kernel Alignment in JAX

13 minute read

Published:

In this blog we implement the Centered Kernel Alignment (CKA) metric used to compare the representations of different neural network layers for the same or two separate networks. CKA measures the similarity of representations at different network layers of the same or different networks.

Introducing mle-hyperopt: A Lightweight Tool for Hyperparameter Optimization 🚂

17 minute read

Published:

Validating a simulation across a large range of parameters or tuning the hyperparameters of a neural network is common practice for every computational scientist. There are a plethora of open source tools that implement individual algorithms, but many of them are either combersome to set up and log or follow diverse syntax, which makes it hard to easily wrap them.

Introducing mle-logging: A Lightweight Logger for ML Experiments 📖

14 minute read

Published:

There are few things that bring me more joy, than automating and refactoring code, which I use on a daily basis. It feels empowering (when done right) and can lead to some serious time savings. The motto: ‘Let’s get rid of boilerplate’. One key ingredient to my daily workflow is the logging of neural network training learning trajectories and their diagnostics (predictions, checkpoints, etc.).

Evolving Neural Networks in JAX

33 minute read

Published:

“So why should I switch from <insert-autodiff-library> to JAX?”. The classic first passive-aggressive question when talking about the new ‘kid on the block’. Here is my answer: JAX is not simply a fast library for automatic differentiation. If your scientific computing project wants to benefit from XLA, JIT-compilation and the bulk-array programming paradigm – then JAX provides a wonderful API.

2020

Meta-Policy Gradients: A Survey

35 minute read

Published:

Most learning curves plateau. After an initial absorption of statistical regularities, the system saturates and we reach the limits of hand-crafted learning rules and inductive biases. In the worst case, we start to overfit. But what if the learning system could critique its own learning behaviour?

The Lottery Ticket Hypothesis: A Survey

38 minute read

Published:

Metaphors are powerful tools to transfer ideas from one mind to another. Alan Kay introduced the alternative meaning of the term ‘desktop’ at Xerox PARC in 1970. Nowadays everyone - for a glimpse of a second - has to wonder what is actually meant when referring to a desktop. Recently, Deep Learning had the pleasure to welcome a new powerful metaphor: The Lottery Ticket Hypothesis (LTH).

A Machine Learning Workflow for the iPad Pro

20 minute read

Published:

The iPad is a revolutionary device. I take all my notes with it, read & annotate papers and do most of my conceptual brainstorming on it. But how about Machine Learning applications? In todays post we will review a set of useful tools & venture into the love story of the iPad Pro & the new Raspberry Pi (RPi).

Getting started with JAX (MLPs, CNNs & RNNs)

30 minute read

Published:

JAX, Jax, JaX. Twitter seems to know nothing else nowadays (next to COVID-19). If you are like me and want to know what the newest hypetrain is about - welcome to todays blog post!

My Top 10 Deep RL Papers of 2019

22 minute read

Published:

2019 - What a year for Deep Reinforcement Learning (DRL) research - but also my first year as a PhD student in the field. Like every PhD novice I got to spend a lot of time reading papers, implementing cute ideas & getting a feeling for the big questions. In this blog post I want to share some of my highlights from the 2019 literature.

2019

Cognitive Computational Neuroscience 2019 - A Mini-Report

23 minute read

Published:

TL;DR: This blog post provides an overview of trends & events from the Cognitive Computational Neuroscience (CCN) 2019 conference held in Berlin. It summarizes the keynote talks and provides my perspective and thoughts resulting from a set of stimulating days. More specifically, I cover recent trends in Model-Based RL, Meta-Learning and Developmental Psychology adventures. You can find all my notes here.

Forward Mode Automatic Differentiation & Dual Numbers

21 minute read

Published:

Automatic Differentiation (AD) is one of the driving forces behind the success story of Deep Learning. It allows us to efficiently calculate gradient evaluations for our favorite composed functions. TensorFlow, PyTorch and all predecessors make use of AD. Along stochastic approximation techniques such as SGD (and all its variants) these gradients refine the parameters of our favorite network architectures.

A Primer on Deep Q-Learning

33 minute read

Published:

Before starting to write a blog post I always ask myself - “What is the added value?”. There is a lot of awesome ML material out there. And a lot of duplicates as well. Especially when it comes to all the flavors of Deep Reinforcement Learning. So you might wonder what is the added value of this two part blog post on Deep Q-Learning? It is threefold.

EEML 2019 - A (Deep) Week in Bucharest!

11 minute read

Published:

In January I was considering where to go with my scientific future. Struggling whether to stay in Berlin or to go back to London, I got frustrated with my technical progress. At NeuRIPS I encountered so much amazing work and I felt like there was too much to learn until reaching the cutting edge. I was stuck. And then my former Imperial supervisor forwarded me an email advertising this new Eastern European Machine Learning (EEML) summer school.

Representational Similarity - From Neuroscience to Deep Learning… and back again

11 minute read

Published:

In today’s blog post we discuss Representational Similarity Analysis (RSA), how it might improve our understanding of the brain as well as recent efforts by Samy Bengio’s and Geoffrey Hinton’s group to systematically study representations in Deep Learning architectures. So let’s get started!

Steal, Stole, Stolen - A ML Perspective!

7 minute read

Published:

Hola guapos! After finally deciding to stay in Berlin, I felt the desire to structure myself and to establish routines which are going to help me tackle the next phase of my life. Due to a fortunate visit to the National Gallery book store in London, I got to pick up Austin Kleon’s amazing piece of work “Steal Like an Artist”. A beautifully collected and visualized set of tricks to foster creativity.

Barcelona GSE Articles and Interviews

less than 1 minute read

Published:

Hey there! As some of you might know I have been quite actively contributing to the Data Science Barcelona GSE blog. Writing about technical topics and addressing a broad audience is challenging and fulfilling at the same time. I hope that this blog is going to help me learn to tell great narratives and influence people. So stay tuned!