Step 1: Go to Settings, select from the "Number of Attempts Allowed" dropdown under "General Settings".
Step 2: For this feature to work, Email (or Name) capture must be enabled from "Security & Tracking" settings. If an attempt with same name or email id comes in, it is regarded as a repeat attempt by the same user.
Step 3: Save your course settings.