USC Internship

Summary

Note: Some details and results are omitted due to sensitive data

Internship: June 2019 - August 2019

Project for USC ISI REU Site: Human Communication in a Connected World Summer 2019.

Independently created and executed a project for data collection (web scraping & cleaning) and data analysis (natural language processing) for the NSF Project “Coordinating Curricula and User Preferences to Increase the Participation of Women and Students of Color in Engineering” Award #1826632

View the Poster

Overview

Mindmap of Workflow (Click & Drag):

Data Collection: Web Scraping & Cleaning:

• Library/Frameworks/Tools used:
  • Scrapy, Scrapy Selectors, re
  • CSS & XPath Selector Reference, XPath tester

Natural Language Processing

• Library/Frameworks/Tools used:
  • NLTK, matplotlib, wordcloud, NumPy, pandas
  • Scikit-learn, Gensim

Data Collection Results

• 4222 reviews on USC & UHM Computer Science & Engineering professors
• 4244 course sections from USC Schedule of Classes
  • 1212 syllabi (.doc & .pdf)