There may be cases like the access token refresh from the Icon Chooser where, when the request fails due to 401, it may be because the WordPress user's session has expired. But because the Icon Chooser has a black box error message for all kinds of errors, it's not clear to users how to resolve the problem.
Maybe handle the 401 response more specifically.