You could always add some sort of incremental counter which increments on each subspicious situation and the ban/kick/spec action is only performed, oncethe counter reaches a specific threshold.
Like an email spam filter which scores the email based on different aspects and only filters when the score is above a specified value.