Close

Sam Showalter

Industry CV      Academic CV

About Me





I am a Computer Science PhD student in UC Irvine's DataLab Group. My research explores machine learning robustness in computer vision, natural language, and reinforcement learning applications. In particular, I explore methods of conducting novelty or out-of-distribution (OOD) detection with deep learning models.

Recent News:

  • Oct 2022 - Our paper on probabilistic querying for neural sequence models was awarded an Oral at NeurIPs 2022!
  • Apr 2022 - I was awarded the NSF Graduate Research Fellowship!
  • Nov 2021 - I was awarded the Hasso-Plattner Institute (HPI) PhD Fellowship (3 years)
  • Sep 2020 - I was awarded the UCI ICS Deans Award of $10,000

Experience

Amazon Science (Personalization)

Applied Scientist: Personalization

Conducted research to improve Amazon's ability to recommend products that facilitate holistic online engagement (clicks, hearts, time on page, etc.). Formulated task as a multi-task learning problem and implemented+evaluated several SOTA algorithms. Achieved an average relative improvement of over 10% over existing production baselines. Augmented my research code into MultiRec: a configuration-driven machine learning research software package to accelerate research. MultiRec supports PyTorch and sklearn-style models, allowing me to train 1443 models over 74 experiment settings in a single summer.

Stanford Research Institute (SRI), International

Machine Learning Researcher

Visiting research scientist at the Stanford Research Institute (SRI International) on the competency-aware machine learning (CAML) project team. Developed CAMeLeon, a reinforcement learning competency assessment research package. The first of it's kind, CAMeLeon integrates many of the most popular reinforcement learning software packages and allows developers to easily train an agent, collect data on its internal state, and then analyze said data with a suite of competency-awareness metrics developed by SRI. CAMeLeon was presented to the entire company and open-sourced as official SRI software release. See the Projects section below for more information.

West Monroe Partners

Technology and Data Science Consultant

Worked primarily as a Data Scientist with large financial institutions. My largest project was a knowledge extraction and classification initiative for data privacy at one of the world's largest banks. I also integrated demand forecasting at VisionWorks of America (POC), and completed customer segmentation projects to assist in marketing various middle-market financial institutions.

National Institute of Standards and Technology

Quantum Computing Research Fellow

Selected as one of 21 students and new graduates in the country to join the NIST laboratories of Boulder, Colorado. Operating within the Information Technology Laboratory, I developed quantum state tomography software to simulate qubit measurement by a computer. Our interdisciplinary group was predominantly theoretical and interested in maximizing the fidelity of qubit measurements without facilitating decoherence. My simulation experiments identified peculiar qubit behavior when measured under different purity and optimization methods. Collaboration with experimental groups to test our theories was common.

84.51°

Data Scientist: Merchandising Insights

Data Science arm of Kroger stores. Developed and outlier detection system in SQL to programmatically parse incoming Controlled Store Test (CST) data for executional errors. CSTs allow Kroger and its consumer packaged goods (CPG) providers (P&G, Burt's Bees, etc.) to experiment with different store layouts and product packaging. These tests are difficult to conduct because holding extraneous variables constant is extremely difficult. My task was to validate testing data was suitable for further analysis and not greatly impacted by confounding factors. Ultimately, my solution ran in less than 30 seconds and detected over 85% of known executional issues historically, saving analysts 4 - 8 hours per test.

Education

University of California, Irvine

August 2020 - Present

PhD in Computer Science

My PhD focus is machine learning robustness, namely exploring novelty detection with deep learning models. Novelty is an incredibly important skill for intelligent agents to have, particularly in safety-critical fields like medicine, law, and self-driving cars.

DePauw University

August 2014 - May 2018

Bachelors' of Arts in Mathematics, Economics
Minor in Computer Science

With the help of a few friends, I launched DePauw's Data Science Group (DPUDS) in 2017. Over the two years I served as President, DPUDS won a Data Science hackathon, completed over a dozen projects, and grew to be one of the largest groups on campus. I also served as President of DePauw's Investment Group. I graduated in 2018 with degrees in Mathematics and Economics, Summa Cum Laude.

Research

Predictive Querying for Neural Sequence Models

NeurIPs 2022 (Oral):      Alex Boyd*, Sam Showalter*, Stephan Mandt, Padhraic Smyth
(* Denotes equal contribution)

In reasoning about sequential events it is natural to pose probabilistic queries such as “when will event A occur next” or “what is the probability of A occurring before B,” with applications in areas such as user modeling, medicine, and finance. However, with machine learning shifting towards neural autoregressive models, probabilistic querying has been largely restricted to simple cases such as next-event prediction. In this paper we introduce a general typology for predictive queries in autoregressive sequence models and show that such queries can be systematically represented by sets of elementary building blocks.

PDF Poster Code

Augmenting Unilabel Crisis Tweets to Develop a Lexically Invariant Sequence Tagger

When a large-scale crisis occurs, the most common destination for up-to-date information is social media, namely Twitter. As crises affect people, it is essential to extract and categorize all relevant information to ensure a thorough and swift response. However, there currently are no sequence tagging datasets for crisis reponse. We programmatically generate a proxy for this dataset with unilabeled tweets and data augmentation from HumAID and successfully build a lexically invariant sequence tagger.

PDF Code

Minimizing the Societal Cost of Credit Card Fraud with Limited and Imbalanced Data

Machine learning has automated much of financial fraud detection, notifying firms of – or even blocking – questionable transactions instantly. However, data imbalance starves traditionally trained models of the content necessary to detect fraud. This study examines three separate factors of credit card fraud detection via machine learning. First, it assesses the potential for different sampling methods – undersampling and Synthetic Minority Oversampling Technique (SMOTE) – to improve algorithm performance...

PDF Ppt. Talk Code

Validating Weak Form Market Efficiency in U.S. Stock Markets with Trend Deterministic Price Data and Machine Learning

The Efficient Market Hypothesis has been a staple of economics research for decades. In particular, weak form market efficiency – the notion that past prices cannot predict future performance – is strongly supported by econometric evidence. In contrast, machine learning algorithms implemented to predict stock price have been touted, to varying degrees, as successful. Moreover, somedata scientists boast the ability to...

PDF Code

Simulating the Characterization of Qubits with Quantum State Tomography Software

Characterizing quantum bits (qubits) is an essential aspect of quantum computing and quantum information science because successful quantum computation requires high fidelity preparation, manipulation, and measurement of qubits. However, fully characterizing qubits with a single measurement is impossible. Therefore, quantum state tomography (QST) – estimating a quantum state using a collection of measurements of copies of that state – is necessary to characterize experimental quantum devices...

Abstract Presentation Code Lab
View More Research

Cuteness in Japanese Design: Investigating Perceptions of Kawaii Among American College Students

Japanese products and pop culture, such as Hello Kitty, Pokemon, J-pop, and Anime, have gained global popularity, including in the United States. As a result, Japanese Kawaii (cute) design has also spread. In the current paper...

Ultimately, the intention of this work is to assess which styles of design are more readily accepted by humans to facilitate human-robot interaction. Paper published in International Conference on Applied Human Factors and Ergonomics.

PDF

Skills

Selected Projects

MultiRec: A Machine Learning Research Package for Recommender Systems

MultiRec accelerates machine learning research for recommender systems by procedurally generating experiments with a single configuration file. MultiRec offers CPU and GPU support, data parallelization, automatic preprocessing, and standard+stratified evaluation. It also works natively with local and S3 directories and can deploy its trained models to SageMaker using the bring-your-own-model design pattern.

CAMeLeon: A Reinforcement Learning Competency Awareness Research Package

Machine learning robustness is central to ensuring we can trust intelligent, automated systems. Despite the huge advances in Machine and Reinforcement Learning, many of these systems still remain black box classifiers. Research in explainable AI (XAI) and ML robustness is ongoing, but largely has not been explored with Reinforcement Learning. CAMeLeon addresses this gap, allowing a user to examine RL agent competency in different environments.

Code Ppt. Talk

Airbender: Automate ML with Apache Airflow DAGs

Airbender is a meta-progamming tool that allows Data Scientists to quickly and effectively build high quality machine learning solutions. After receiving a configuration, Airflow writes a python code file that encapsulates all of the functionality in nodes and maps all relevant dependencies as a directed acyclic graph (DAG). With intelligent package detection, Airbender creates a file that is ready to compile in Airflow immediately, allowing users full transparency into and control of their experiments.

Code

Hunger Snakes: Neuroevolution of Fixed Topologies

There are many examples on the internet of developers and Data Scientists utilizing Neuroevolution of Augmenting Topologies to teach the agents of simple computer games to play (Snake, Mario Bros., etc.). However, few of these implementations conduct such an experiment under adversarial circumstances, when multiple agents compete for limited resources. In this project I build, from scratch, an adversarial implementation of Neuroevolution for the game snake. For simplicity, topologies were fixed for all agents based on a common fitness function.

Code

Web Application for CalTech256 Image Classification

The CalTech 256 image dataset was created to assess the performance of different image classification models, particular under different input data size (number of photos) constraints. Using a transfer learning model build on the Xception classifier, myself and small team I led were able to achieve a holistic macro and micro F1 Score of 86% across all classes. We then integrated this model into a frontend Flask web application where a user can upload any photo and see the top 10 class prediction probabilities as well as the overall prediction.

Code
View More Projects

MNIST Handwriting Detection GUI

The MNIST dataset from the U.S. Postal Service is one of the most commonly used datasets when teaching optical character recognition with basic convolutional neural networks. After seeing a Javascript and Lua implementation here (ADD HREF), I decided to see if I could build something similar, but with more transparency into the process and completely in Python. The solution I completed was 98 percent accurate for classifying numbers.

Code

Shakespeare Chains: Markov Chain Story Writing

Markov Chains are simple networks that rely on probablistic state transfer. These transfer probabilities are based on a specific corpus, something that can be unfluenced by the user. This project allows the user to upload a text file (PDF support coming shortly!) and then generate a unique story with the style of the input authors. This process will include all punctuation and spacing is provided by the system, allowing for stylistic spacing like in poetry to be accommodated.

Code

DePauw Adventure Game: Java-based Campus Adventure

The DePauw Adventure Game is a school project that evolved into something greater, a complete, dynamic gameboy-style game where the user is a student trying to fight his or her way through college. The player must grow in strength by roaming the map before he or she tackles each of the 8 academic buildings, unlocking the 9th and final building, the office of administration. Here, players come face to face with their strongest opponent yet, the University President. Download the .jar file and play today (no, this is not a virus)! See the presentation for more details on the functionality and dynamism of this game.

Code Presentation

Sudoku Solver GUI: Intelligent Heuristics and Search Algorithms

Instead of relying on simple backtracking algorithms and the blazing speeds of a CPU, this Sudoku solver and GUI walks through the puzzle step-by-step using intelligent heuristics to quickly finish the problem. In cases of uncertainty, the solver will posit a guess and continue its execution. If it runs across an error, this guess is corrected or adjusted to accommodate the new information created. NOTE: This system is not yet perfect, and on rare occaisons (specifically when no input is provided), this solver will fail. Future development is forthcoming to patch this, but in the mean-time check out the list of sample puzzles provided. Before a cell is solved, it is color-coded and potential matches appear in the corners, just like how my Grandma solves them!

Code

Maze Solver: BFS and DFS Visualized

Bread first and depth first search are two essential concepts for software development. In order to visualize the differences in execution for each, I built a maze solver system that takes a text file maze as input and then utilizes either BFS or DFS to find the exit, if there is one. The maze solver accomodates no entrances and ill-formatted mazes without crashing.

Code

Get in Touch