I was glancing through some basic activation functions, etc., and I realized I was taking for granted exactly what the SoftMax function was doing mathematically, so I took a little closer look at the equations.

I like to see visuals and interact with these things when possible, (and because it’s fun) I used Desmos to build an interactive applet simulating the output of a 3-class SoftMax function. Desmos was a common tool I used for visuallzing functions for my physics students. Simple as my little interactive it is, it’s clearer than most of what I encountered on-line; so I thought I would share it here.

Some notes on the interactive, as Desmos doesn’t support comments very well:

• The horizontal axis (z₁) is the net input for 1 class

• the z₂ and z₃ sliders are the net input for the other two classes
• Note how the sum of the outputs for all 3 SoftMax outputs is always 1.