LBFGS can restart, but the data used for restart is not the right one. That means it affects overall convergence, sometimes leads to failure. Solution: dump all the data needed to restart smoothly and read it properly when restarting.