Talk: Colin Carroll - Getting started with automatic differentiation

Presented by: Colin Carroll The derivative is a concept from calculus which gives you the rate of change of a function: for a small change in an input, how much does the output change? This idea turns out to be very important in natural sciences, and is used in many optimization algorithms, which find the maximum or minimum of functions. Automatic differentiation is a technique for computing the derivative of a function. Python has a number of libraries implementing automatic differentiation, many of which are put to use for deep learning, but can be used on their own. In this talk I will give intuition for the derivative and its high dimensional sibling, the gradient. We will take a tour of applications, including optimization and computational art, with examples using jax, TensorFlow, and PyTorch. We conclude with a brief description of alternative ways of computing derivatives in Python, and their relative strengths.