The Simple Essence of Automatic Differentiation - Conal Elliott

The most interesting thing I've seen on AD is "The simple essence of automatic differentiation" (2018) [1]. See past discussion [2], and talk [3]. I think the main idea is that by compiling to categories and pairing up a function with its derivative, the pair becomes trivially composable in forward mode, and the whole structure is easily converted to reverse mode afterwards. [1]: [LINK] [2]: [LINK] [3]: Talk at Microsoft Research: [VIDEO] Other presentations listed here: [LINK]

Another paper related to AD was posted last month: The Simple Essence of Automatic Differentiation [0]. Video presentation [1]. I found it pretty awesome, just in case you hadn't seen it. [0]: https//news.ycombinator.com/item?id=18306860 [1]: [VIDEO]

He also gave nice presentation on AD: [VIDEO] (surprised that you didn't already mention it, considering the venue ;-))

I'm yet to give reverse mode a second try, but my attempts at forward mode (linked in parent post) were on my own without the paper's help. There's a talk that Conal gives that makes the paper a bit more accessible - [VIDEO] - but only a bit more. I know it'll take effort to wrap my head around the implementation details ... but the paper certainly helped clarify things I was stumbling around about.

This paper won a best paper award at ICFP 2018. Conal Elliott recently gave a longer talk about it at Microsoft Research: [VIDEO]

AD is just a Cartesian lens of Jacobians and total derivatives in the category of smooth functions, what's the problem? [VIDEO]