VIM normal mode moving around resets the timer

I'm not sure if this is intended behavior or not, but when using VIM mode, pressing non editing keys in normal mode (ie, hjkl to move the cursor around but doesn't change any text) resets the timer.

Not really a big issue and might recall significant code change to change, so issue should probably just be closed.