Have you ever been to a great meeting? The right people with a diverse set of skills, knowledge, and perspectives, where all the inputs are synthesized into a single, coherent output that addresses the issue being discussed. I have (twice!) and it’s a marvelous thing. The only problem is that…

Some background A few weeks ago, I published The mean misleads: why the minimum is the true measure of a function’s run time, where I explained that taking the means of the run times of two functions (with the aim of selecting the faster one) will produce incorrect results more often than…

When it comes to defining learning rate schedules in PyTorch, you have plenty of options. 15 different scheduler classes, to be exact. This offers good flexibility, but it sometimes means that making a small change to your learning rate schedule requires switching to a different scheduler or combining multiple schedulers. — It struck me one day that defining how a learning rate should change over the course of a training run is similar to defining an animation, with a combination of:

Imagine: you’ve got a function and you’d like to know how long it takes to run, on average. So you run the function a few times, measuring how long it takes each time, then take the average of those times. That couldn’t possibly be wrong, could it? In this post…

This article is about how we process text before passing it to large language models, the problems with this approach, and an alternative solution. — The Problem With (1D) Tokenization When passing text to a Large Language Model (LLM), text is broken down into a sequence of words and sub-words. This sequence of tokens is then replaced with a sequence of integers and passed to the model. LLMs contain an embedding matrix to store a representation for each of these…

This post is for anyone considering a career move into the field of machine learning. Of course, the answer to the titular question is “it depends”. But what does it depend on, exactly? Rather than try and give some generic answer to the question that suits no one in particular…

There are 7.674 billion people on this planet. Of those, how many are working on innovations that will make the world a better place? A few thousand people? Maybe a hundred thousand, a million? And what about the other 7.673 billion people? Sprinkled throughout are some truly brilliant individuals, people…

The world is currently in the process of rolling out a vaccine that is killing people. Every day, millions of people are getting a jab in the arm that could be their last. This is not disputed. There is, of course, another option: not getting the vaccine. In fact, there…

Putting an end to email scams is so simple that I’ll explain the entire process in one word in the second paragraph. The rest of the article is just padding. Ready to learn what you need to do? Ready?!…. Respond. Now for the padding… The scamming industry is profitable because…