Add adaptive tolerance scaling for boundary and gradient algorithms in variational optimization

[leburgel]

Adds adaptive tolerance scaling, where the tolerance of the contraction algorithm is set based on the current norm of the gradient (default scaling factor of 1.0e-3). The tolerance of the gradient algorithm in turn is set based on the contraction tolerance (default scaling factor 1.0e1).

On the README example, these default settings give a 1.5x speedup in walltime needed to reach a given energy value starting from the same initial guess. I'll see what the tests say for now, and do some more trials in the meantime.

At the same time, this reworks the way the per-iteration info metrics are stored and added. Previously, the contraction_metrics and gradnorms_unitcell got a new entry every time cost function was called. These function evaluations don't necessarily match the iterations of the optimization loop. In particular, entries were added for every rejected step during the line search, which just seemed needlessly confusing. I changed to workflow to record the current values in the cost function, but only add entries in the finalize! routine, which is actually called after every iteration (i.e. every accepted step).

This is also in anticipation of adding preconditioning, which naturally requires access to the current gradient norm.

[codecov]

Codecov Report

❌ Patch coverage is 97.72727% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/algorithms/select_algorithm.jl	94.11%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/Defaults.jl	85.71% <ø> (ø)
...rithms/optimization/fixed_point_differentiation.jl	85.47% <100.00%> (-3.33%)	⬇️
src/algorithms/optimization/peps_optimization.jl	96.70% <100.00%> (+1.96%)	⬆️
src/algorithms/select_algorithm.jl	82.05% <94.11%> (+5.12%)	⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pbrehmer]

Up to one very minor thing, this looks good to me! I really like that this also clears up how the per-iteration information is logged.

[lkdvos]

Is there a reason to ask for time_ns to then immediately convert to * 1e-9?

[leburgel]

No idea, I didn't really touch this. Nanosecond resolution for a PEPS optimization iteration doesn't sound like something that will be necessary anytime soon, so I can change it to time.

[lkdvos]

This is more of a suggestion about code organization, which would achieve the exact same thing, but I'm not that used to having many of these Ref values around as a way to keep things mutable, and combining this with an anonymous function factory might not be the most readable way of getting to this.

It might make sense to create a simple mutable struct for this, and then overload function calling? Something like

mutable struct TrackedFinalizer{A, B, C, D}
    tol_state::A
    latest_metrics::B
    latest_gradnorms::C
    latest_time::Float64 # I'm guessing?
    const contraction_metrics::Vector{B}
    const gradnorms_unitcell::Vector{C}
    const times::Vector{Float64}
    const finalize!
end

I'm not saying this is necessarily better, but I might have structured it more like this so just wanted to leave this suggestion to see what you think.

Additionally, it might even be possible to get rid of the mutable ref values and simply have a finalizer that resize!s the vector's by length + 1, and then just update the last element in the vector, which is of course mutable. (although that might lead to subtleties for the last iteration, maybe?)

[leburgel]

I actually like resize! a lot better, didn't think of that.

I'm fine with using a callable mutable struct instead, let me give this a go.