OurCampus
The OurCampus menu function puts all of the Wesleyan eatery menus in one place. I wrote a series of Javascript webscrapers for this that use Cheerio to scrape the Swings and Red & Black Cafe menus and then use regular expressions to find the actual menu items. I used Firebse functions to automate the scraping, as well as to cleanup the firestore every day and to send push notifications to users when a dish they want is available that day.
Wesleyan has several different eateries on campus, the main ones being Usdan, Red & Black Cafe, and WesWings (Swings). Unfortunately, Swings/RBC and Usdan each have different websites, both of which are horribly designed and maddeningly confusing, making checking whats for dinner a whole ordeal. Therefore, I joined with the OurCampus team to work on a new feature that would have all the daily menus in one place, as well as some other neato features.
I worked on the scripts that scraped the menus from the R&B + Swings website. To do so, I wrote a JavaScript Firebase function that used request-promise and cheerio. I had never used JavaScript before, so I had a lot of fun learning the language, and then not a lot of fun learning about JavaScript asynchronous calls. The menus for the two eateries have a lot of quirks that made scraping them a bit of a pain. For starters, the R&B menus are only posted as a picture of a printed menu, so scraping that menu requires the use of image recognition. I used the Google Vision API to recognize the text from the image, and then a crazy regular expression (it was /^(([^.\s]+ ?){1,4}(?![^.]{1,20}\n)(?=[^\n]*\n.+)|([^.\s]+ ?){1,4}(of the day))(?=\n{1,2})/gim in case anyone was wondering) to differentiate the dish titles from the dish descriptions. Then, the other main quirk of the menus was that a different person uploads the menus each day, leading to wild inconsistencies in formatting, requiring the use of several different regular expressions and clever ways of squishing the menus into a usable form for us.
In addition to just scraping the menus, I had the idea that users should be able to get a notification when their favorite dishes are available. To make this happen, every dish we scrape is added to a "dish" collection in Firestore, and then when that dish is available all users who are subscribed to it will receive a push notification. Furthermore, I created an error handling function that notifies the developers if certain errors occur, such as the swings website being changed, the image recognition API failing, or any manner of errors that could harm production. Lastly, I created a cleanup function to clean up old menus each day to save Firestore space and to ensure that old menus aren't displayed.
At the time of writing this, the OurCampus menu feature has over 700 daily users, which is about a quarter of the ENTIRE Wesleyan student body