To implement a 5-minute rolling window, you need to know the time of each attempt, and not just the first or last.
Store every login attempt in the database. As noted in the comments, just using the session cache is easy to get around.
username | time mr.mindor | 00:00:00 mr.mindor | 00:01:00
etc .. then for each login attempt, check that there are 10 or fewer attempts ...
Select count(*) from LOGIN_ATTEMPTS where username = @current_user AND time > DATEADD(m,-5,GetDate())
You can temporarily hold LOGIN_ATTEMPTS for security / audit purposes or periodically clear it or with a successful login.
Problems saving only the first or last time and quantity.
Without each time, you cannot determine the distribution of attempts.
Solution for the first attempt:
Only with the first time and quantity attempt can you prevent more than 10 attempts in the 5 minutes following first, but not prevent more than 10 attempts in any given 5-minute interval.
Example: you can have 1 at 0:00 and 9 at 4:59, and another 10 seconds at 5:01 for 19 seconds.
attempt_time | record 0:00 | {0:00, 1} 4:59 | {0:00, 2} 4:59 | {0:00, 3} 4:59 | {0:00, 4} 4:59 | {0:00, 5} 4:59 | {0:00, 6} 4:59 | {0:00, 7} 4:59 | {0:00, 8} 4:59 | {0:00, 9} 4:59 | {0:00, 10} 5:01 | {5:00, 1} // 10 since 4:59 5:02 | {5:00, 2} // 11 since 4:59
Solution based on last attempt:
By saving only the time of the last attempt and the counter, you can prevent more than 10 attempts in any 5-minute interval, but in fact it is much more restrictive. This will prevent more than 10 attempts that appear within 5 minutes of the previous attempt.
Example. If there are attempts every 4 minutes, your counter will never reset.
attempt_time | record 0:00 | {0:00, 1} 4:00 | {4:00, 2} // 2 in last 5 minutes 8:00 | {8:00, 3} // 2 in last 5 minutes ... 40:00 | {40:00,11} // blocked but only 2 in last 5 minutes.