# How to Find and Fix Existing Duplicates
## Quick Start Guide
If you have duplicate entries in your database (like the example below), here's how to find and clean them up:
```
31 Dec 2025 N/A Book Voucher ₹659.94 - ₹170.54
31 Dec 2025 N/A Book Voucher ₹659.94 - ₹170.54
```
## Step-by-Step Instructions
### Step 1: Navigate to Conflicts Page
Go to `/conflicts` in your browser:
```
http://localhost:3000/conflicts
```
Or click "Conflicts" in the navigation menu.
---
### Step 2: Use the Duplicate Scanner
At the top of the page, you'll see a section called **"Scan Database for Duplicates"**:
```
┌──────────────────────────────────────────────────────────┐
│ Scan Database for Duplicates │
│ │
│ Scan your database to find existing duplicate entries │
│ that may have been imported before conflict detection │
│ was enabled. │
│ │
│ [Account: HDFC Bank ▼] [Type: Ledger ▼] [Scan for...] │
└──────────────────────────────────────────────────────────┘
```
**Actions:**
1. **Select Account** - Choose the account you want to scan (e.g., "HDFC Bank", "Zerodha")
2. **Select Type** - Choose what to scan:
- `Tradebook` - Scans for duplicate trades
- `Ledger` - Scans for duplicate ledger entries
3. **Click "Scan for Duplicates"** button
---
### Step 3: Wait for Scan Results
You'll see a success message:
```
✅ Scan complete. Found 5 duplicate(s)
```
Or if no duplicates:
```
✅ Scan complete. Found 0 duplicate(s)
```
---
### Step 4: Review Detected Duplicates
Below the scanner, you'll see all conflicts including the newly found duplicates.
**Example Duplicate Entry:**
```
┌────────────────────────────────────────────────────────────────┐
│ 🔵 Ledger Conflict │
│ Account: HDFC Bank • Type: Exact Duplicate (Already in DB) │
│ Detected: Jan 13, 2026, 3:45 PM │
├────────────────────────────────────────────────────────────────┤
│ │
│ ⚠️ Exact Duplicate Entry │
│ This ledger entry exists multiple times in your database. │
│ This duplicate was found during a database scan. │
│ │
├──────────────────────────┬─────────────────────────────────────┤
│ First Entry (Keep) │ Duplicate Entry (Delete) │
│ [Yellow Background] │ [Red Background] │
├──────────────────────────┼─────────────────────────────────────┤
│ Date: 31 Dec 2025 │ Date: 31 Dec 2025 │
│ Particular: N/A │ Particular: N/A │
│ Debit: ₹659.94 │ Debit: ₹659.94 │
│ Credit: ₹0 │ Credit: ₹0 │
│ Balance: ₹170.54 │ Balance: ₹170.54 │
│ Database ID: 1234 │ Database ID: 5678 │
└──────────────────────────┴─────────────────────────────────────┘
│ │
│ 💡 Recommendation: Click "Delete Duplicate" to clean up. │
│ │
├────────────────────────────────────────────────────────────────┤
│ [Keep First Entry] [Delete Duplicate] [Ignore] [Delete...] │
└────────────────────────────────────────────────────────────────┘
```
**Key Points:**
- **Yellow Box** = First entry found (will be kept)
- **Red Box** = Duplicate entry (will be deleted)
- **Database IDs** shown for transparency
- Blue icon (🔵) indicates exact duplicate
---
### Step 5: Clean Up Duplicates
For each duplicate, you have 4 options:
#### Option 1: Delete Duplicate (Recommended) ✅
**Button:** `[Delete Duplicate]` (Red button)
**Action:**
- Removes the duplicate entry (red box) from database
- Keeps the first entry (yellow box)
- Marks conflict as resolved
**When to use:** Almost always - this cleans up your database
---
#### Option 2: Keep First Entry
**Button:** `[Keep First Entry]` (Yellow button)
**Action:**
- Does NOT change the database
- Just marks the conflict as resolved
- Both entries remain in database
**When to use:** If you want to review the entry later or are unsure
---
#### Option 3: Ignore
**Button:** `[Ignore]` (Gray button)
**Action:**
- Marks conflict as "ignored"
- Both entries remain in database
- Conflict moves to "ignored" status
**When to use:** If you intentionally want to keep both entries
---
#### Option 4: Delete Conflict
**Button:** `[Delete Conflict]` (Red button, on the right)
**Action:**
- Removes the conflict record only
- Both database entries remain untouched
- Just cleans up the conflicts list
**When to use:** If this was incorrectly flagged as duplicate
---
## Example Workflow
### Scenario: You have duplicate ledger entries for account "HDFC Bank"
**Step-by-step:**
1. Go to `/conflicts`
2. In the scanner section:
- Select "HDFC Bank" from Account dropdown
- Select "Ledger" from Type dropdown
- Click "Scan for Duplicates"
3. See message: "Scan complete. Found 3 duplicate(s)"
4. Review each conflict:
- Conflict 1: Book Voucher duplicate
- Conflict 2: Another duplicate entry
- Conflict 3: Third duplicate
5. For each conflict, click `[Delete Duplicate]`
6. Done! Your database is now clean
---
## Understanding the Results
### For Ledger Duplicates
**Duplicate Detection Criteria:**
- Same date
- Same particular (description)
- Same debit amount
- Same credit amount
**Example:** These are duplicates:
```
Entry 1: 31 Dec 2025, N/A, ₹659.94 debit, ₹0 credit
Entry 2: 31 Dec 2025, N/A, ₹659.94 debit, ₹0 credit
```
**Example:** These are NOT duplicates:
```
Entry 1: 31 Dec 2025, N/A, ₹659.94 debit, ₹0 credit
Entry 2: 31 Dec 2025, N/A, ₹700.00 debit, ₹0 credit // Different amount
```
### For Tradebook Duplicates
**Duplicate Detection Criteria:**
- Same trade_id
- Same symbol
**Example:** These are duplicates:
```
Trade 1: Trade ID T123, RELIANCE, 10 qty, ₹2450.50
Trade 2: Trade ID T123, RELIANCE, 10 qty, ₹2450.50
```
---
## Safety Features
### 1. No Automatic Deletion
- Nothing is deleted automatically
- You review and approve each deletion
- You can choose to keep entries if needed
### 2. First Entry Always Safe
- The first entry found is always marked for keeping (yellow)
- Only subsequent duplicates are marked for deletion (red)
### 3. Database IDs Shown
- You can see the exact database ID being deleted
- Full transparency about what's being removed
### 4. Reversible Actions
If you accidentally delete something:
- You can manually re-enter the data
- Or restore from database backup
- (Consider implementing soft-delete in future)
---
## Troubleshooting
### Problem: "No duplicates found" but I know there are duplicates
**Possible causes:**
1. **Wrong account selected** - Check you selected the correct account
2. **Wrong type selected** - Ledger vs Tradebook
3. **Minor differences** - Entries might have slight differences in:
- Amounts (even 1 paisa difference = not duplicate)
- Dates (even 1 day difference = not duplicate)
- Particular text (case-sensitive)
**Solution:** Manually check the database or use a different scan
---
### Problem: Scan is taking too long
**Causes:**
- Large database with many entries
- Slow database connection
**Solution:**
- Wait for scan to complete (usually < 10 seconds)
- Check database performance
- Contact support if > 30 seconds
---
### Problem: Getting an error during scan
**Common errors:**
- "Account not found" - Account ID doesn't exist or doesn't belong to you
- "Unauthorized" - Not logged in
- "Database error" - Database connection issue
**Solution:**
- Refresh page and try again
- Check you're logged in
- Contact support if persists
---
## Best Practices
### 1. Scan Regularly
- Run scan after large imports
- Monthly maintenance scan
- Before generating reports
### 2. Scan Each Account Separately
- Don't skip accounts
- Different accounts may have different duplicate patterns
### 3. Scan Both Types
- Run for Tradebook
- Run for Ledger
- Don't assume one is clean
### 4. Review Before Deleting
- Always check the comparison
- Verify database IDs if needed
- When in doubt, click "Ignore" and investigate later
### 5. Keep Records
- Take note of how many duplicates were found
- Document why duplicates existed
- Improve import process to prevent future duplicates
---
## FAQ
**Q: Will this find duplicates from different accounts?**
A: No, it only scans within the selected account. Run separately for each account.
**Q: What if I have 3+ copies of the same entry?**
A: The scanner will create multiple conflicts - one for each duplicate. Keep the first, delete the others.
**Q: Can I undo a deletion?**
A: Not automatically. You'll need to manually re-enter or restore from backup.
**Q: Does "Ignore" delete the duplicate?**
A: No, "Ignore" keeps both entries and just dismisses the conflict.
**Q: What's the difference between "Keep First Entry" and "Ignore"?**
A: Both keep the entries, but "Keep First Entry" marks it as resolved, while "Ignore" marks it as intentionally ignored.
**Q: How do I know which entry is "first"?**
A: The one with the lower database ID (shown in the comparison). Usually the older entry.
**Q: Can I delete the first entry instead of the duplicate?**
A: Not through the UI, but you can use "Delete Conflict" to dismiss, then manually delete the first entry from database if needed.
---
## Need Help?
If you're unsure about any duplicate:
1. Click `[Ignore]` to skip for now
2. Review manually in the database
3. Ask support for guidance
4. Come back and resolve later
Remember: It's always safer to keep entries than to delete them if you're uncertain!
