SP claims loss of votes for BJP