I also played around with caching the objective evaluations by having Fit.execute wrap self.objective.__call__ in a lru_cache, but that just plain doesn't work (discussion in #280).
I'm completely happy to accept the performance hit one additional objective evaluation incurs per iteration, if someone explicitly asked for verbose feedback.

See also:
https://stackoverflow.com/questions/53423121/why-does-functools-lru-cache-not-cache-call-while-working-on-normal-methods

That output looks great! Perfect information for slow fits. The "last" vs "avg" iter time will be particularly useful for diagnosing peculiarities. For example, I've recently been trying to fit large systems of ODEs that become "stiff" only for certain parameter values, resulting in massive slowdown. This would help catch that sort of thing.

I like this a lot too, just needs documentation and testing as far as I'm concerned. Other than that it's a good step in this direction, and it also sets a good example for how people could define their own callback for their specific needs.

This should address your code issues, and adds tests.
Where would you like the docs? I don't really see a "natural" place.

I'm getting unrelated test failures: HadamardProduct is not defined. Do you have any idea what could be causing them?