LSA with Randomized SVD
Clustering and searching Usenet posts with my own randomized SVD implementation.
What I explored
For this course project, I implemented randomized Singular Value Decomposition and used it for latent semantic analysis. I compared it with classical SVD, then used the reduced representation to cluster and search about 18,000 Usenet posts.
Dataset
20 Newsgroups — ~18,000 Usenet posts across 20 topic categories. The dataset was preprocessed with Snowball Stemming and custom stop-word filtering.
What I added around the algorithm
- Power iterations to tighten the randomized low-rank approximation.
- A sweep over cluster counts, compared with the Silhouette Score.
- A desktop view with Plotly charts and semantic search.
- A CLI path with Matplotlib output for running without the interface.
- Text cleanup with Snowball stemming and a custom stop-word list.
Under the hood
The rSVD Algorithm
Classical SVD scales as O(mn·min(m,n)) — prohibitive for large matrices. The randomized approach projects the matrix onto a lower-dimensional subspace first, then applies SVD to the smaller matrix, achieving near O(mnk) complexity for rank-k approximation. Power Iterations further refine the approximation quality.
Clustering Workflow
After dimensionality reduction, documents are clustered in the latent concept space. The Silhouette Score is computed across a range of k values to automatically identify the most cohesive grouping.
Built with
Language: Python 3
Libraries: NumPy, SciPy, Scikit-Learn, Matplotlib, Plotly, NLTK
Techniques: Randomized SVD, LSA, K-Means, Silhouette Score, TF-IDF, Snowball Stemming
Interfaces: GUI (Plotly + Tkinter), CLI (Matplotlib)
Source: GitHub