Question 1

What is the dataset scale of the AI Recommendation System?

Accepted Answer

The pipeline processes over 21 Million records in PySpark, including 1,000,000+ TMDB movies and 20,000,000+ MovieLens user interaction ratings joined and aggregated across a Delta Lake Medallion architecture.

Question 2

How does the vector database and online serving tier operate?

Accepted Answer

Gold vector embeddings (768-D) are hash-partitioned across a 10-shard Neon Serverless PostgreSQL cluster using pgvector HNSW graph indexes, delivering sub-5ms cosine similarity vector search.

Question 3

Which deep learning models power the recommendation ensemble?

Accepted Answer

The system ensembles 6 PyTorch neural architectures: SASRec (Self-Attention Transformer), KAN (Kolmogorov-Arnold Network B-Splines), LightGCN (Graph Convolution), Neural ODE (Continuous Fluid Dynamics), Poincaré Hyperbolic embeddings, and Latent Continuous Diffusion.