Introduction
Build a code plagiarism detector that analyzes AST structure, token sequences, and semantic similarity to detect copied code. This comprehensive guide covers everything from design through implementation, testing, and deployment.
Build a code plagiarism detector that analyzes AST structure, token sequences, and semantic similarity to detect copied code.
Build a code plagiarism detector that analyzes AST structure, token sequences, and semantic similarity to detect copied code. This comprehensive guide covers everything from design through implementation, testing, and deployment.
Tokenize source code (remove whitespace/comments, convert identifiers to generic tokens: INT, ID, OP, etc.). Calculate Longest Common Subsequence (LCS) of token sequences. Similarity = 2 × LCS_length / (tokens_A + tokens_B). This is the core of JPlag and MOSS algorithms. Advantage: token normalization makes superficial changes (variable renaming, comment adding, whitespace changes) transparent — only structural changes matter.
10 components required for this project.
| # | Component | Purpose | Qty |
|---|---|---|---|
| 1 | Python 3.10+ | Implementation language | x1 |
| 2 | ast module (stdlib) | Python AST parsing | x1 |
| 3 | javalang (Java parser) | Java code parsing | x1 |
| 4 | sentence-transformers | Semantic code embeddings | x1 |
| 5 | difflib (stdlib) | Sequence similarity (SequenceMatcher) | x1 |
| 6 | Flask | Web interface for submissions | x1 |
| 7 | CodeBERT (HuggingFace) | Code-aware embeddings | x1 |
| 8 | Matplotlib + NetworkX | Similarity visualization graph | x1 |
| 9 | Celery + Redis | Async batch comparison | x1 |
| 10 | MongoDB | Submission and result storage | x1 |
Follow these 4 steps carefully.
Tokenize source code (remove whitespace/comments, convert identifiers to generic tokens: INT, ID, OP, etc.). Calculate Longest Common Subsequence (LCS) of token sequences. Similarity = 2 × LCS_length / (tokens_A + tokens_B). This is the core of JPlag and MOSS algorithms. Advantage: token normalization makes superficial changes (variable renaming, comment adding, whitespace changes) transparent — only structural changes matter.
Parse code to AST. Compare AST structures using tree edit distance: minimum operations (insert, delete, relabel node) to transform tree A into tree B. Lower edit distance = higher similarity. Hash AST subtrees using Merkle hashing — identical subtrees produce identical hashes. Count matching subtrees. Advantage over token comparison: captures code structure independent of textual representation.
CodeBERT is pre-trained on code-text pairs. Generate code embeddings: tokenize code, pass through CodeBERT, use [CLS] token embedding as code vector. Cosine similarity between code vectors measures semantic similarity. This captures algorithmic intent even when implementation differs significantly. Example: recursive and iterative factorial implementations are functionally equivalent — cosine similarity > 0.85 despite syntactic differences.
Combine all three techniques for robust detection: token similarity (40% weight), AST structural similarity (35% weight), semantic embedding similarity (25% weight). Final score = 0.4×token + 0.35×AST + 0.25×semantic. Flag pairs above threshold (0.7) for review. Present evidence: highlighted matching token sequences, highlighted matching AST subtrees, and a diff view showing structural changes between suspicious submissions.
Core code for plagiarism_detector.py:
Test Code Plagiarism Detection System by verifying each subsystem individually before full integration.
Verify power voltages, check ground connections, use serial monitor for debug.
An interactive simulator will be available here — simulate circuits and run code in-browser without hardware.