Sam Showalter

Bayesian Online Learning for Consensus Prediction

AI Stats 2024:      Sam Showalter*, Alex Boyd*, Padhraic Smyth, Mark Steyvers
(* Denotes equal contribution)

Given a pre-trained classifier and multiple human experts, we investigate the task of online classification where model predictions are provided for free but querying humans incurs a cost. In this setting, oracle ground truth is not available. Instead, labels are defined as the consensus vote of all experts. Human time is costly, so we propose a general framework for online Bayesian consensus estimation.

PDF	Poster	Code

Predictive Querying for Neural Sequence Models

NeurIPs 2022 (Oral):      Alex Boyd*, Sam Showalter*, Stephan Mandt, Padhraic Smyth
(* Denotes equal contribution)

In reasoning about sequential events it is natural to pose probabilistic queries such as “when will event A occur next” or “what is the probability of A occurring before B,” with applications in areas such as user modeling, medicine, and finance. However, with machine learning shifting towards neural autoregressive models, probabilistic querying has been largely restricted to simple cases such as next-event prediction. In this paper we introduce a general typology for predictive queries in autoregressive sequence models and show that such queries can be systematically represented by sets of elementary building blocks.

PDF	Poster	Code

Augmenting Unilabel Crisis Tweets to Develop a Lexically Invariant Sequence Tagger

When a large-scale crisis occurs, the most common destination for up-to-date information is social media, namely Twitter. As crises affect people, it is essential to extract and categorize all relevant information to ensure a thorough and swift response. However, there currently are no sequence tagging datasets for crisis reponse. We programmatically generate a proxy for this dataset with unilabeled tweets and data augmentation from HumAID and successfully build a lexically invariant sequence tagger.

PDF	Code

Minimizing the Societal Cost of Credit Card Fraud with Limited and Imbalanced Data

Machine learning has automated much of financial fraud detection, notifying firms of – or even blocking – questionable transactions instantly. However, data imbalance starves traditionally trained models of the content necessary to detect fraud. This study examines three separate factors of credit card fraud detection via machine learning. First, it assesses the potential for different sampling methods – undersampling and Synthetic Minority Oversampling Technique (SMOTE) – to improve algorithm performance...

PDF	Ppt.	Talk	Code

Validating Weak Form Market Efficiency in U.S. Stock Markets with Trend Deterministic Price Data and Machine Learning

The Efficient Market Hypothesis has been a staple of economics research for decades. In particular, weak form market efficiency – the notion that past prices cannot predict future performance – is strongly supported by econometric evidence. In contrast, machine learning algorithms implemented to predict stock price have been touted, to varying degrees, as successful. Moreover, somedata scientists boast the ability to...

PDF	Code

Simulating the Characterization of Qubits with Quantum State Tomography Software

Characterizing quantum bits (qubits) is an essential aspect of quantum computing and quantum information science because successful quantum computation requires high fidelity preparation, manipulation, and measurement of qubits. However, fully characterizing qubits with a single measurement is impossible. Therefore, quantum state tomography (QST) – estimating a quantum state using a collection of measurements of copies of that state – is necessary to characterize experimental quantum devices...

Abstract	Presentation	Code	Lab

MultiRec: A Machine Learning Research Package for Recommender Systems

MultiRec accelerates machine learning research for recommender systems by procedurally generating experiments with a single configuration file. MultiRec offers CPU and GPU support, data parallelization, automatic preprocessing, and standard+stratified evaluation. It also works natively with local and S3 directories and can deploy its trained models to SageMaker using the bring-your-own-model design pattern.

CAMeLeon: A Reinforcement Learning Competency Awareness Research Package

Machine learning robustness is central to ensuring we can trust intelligent, automated systems. Despite the huge advances in Machine and Reinforcement Learning, many of these systems still remain black box classifiers. Research in explainable AI (XAI) and ML robustness is ongoing, but largely has not been explored with Reinforcement Learning. CAMeLeon addresses this gap, allowing a user to examine RL agent competency in different environments.

Code	Ppt.	Talk

Airbender: Automate ML with Apache Airflow DAGs

Airbender is a meta-progamming tool that allows Data Scientists to quickly and effectively build high quality machine learning solutions. After receiving a configuration, Airflow writes a python code file that encapsulates all of the functionality in nodes and maps all relevant dependencies as a directed acyclic graph (DAG). With intelligent package detection, Airbender creates a file that is ready to compile in Airflow immediately, allowing users full transparency into and control of their experiments.

Code

Hunger Snakes: Neuroevolution of Fixed Topologies

There are many examples on the internet of developers and Data Scientists utilizing Neuroevolution of Augmenting Topologies to teach the agents of simple computer games to play (Snake, Mario Bros., etc.). However, few of these implementations conduct such an experiment under adversarial circumstances, when multiple agents compete for limited resources. In this project I build, from scratch, an adversarial implementation of Neuroevolution for the game snake. For simplicity, topologies were fixed for all agents based on a common fitness function.

Code

Web Application for CalTech256 Image Classification

The CalTech 256 image dataset was created to assess the performance of different image classification models, particular under different input data size (number of photos) constraints. Using a transfer learning model build on the Xception classifier, myself and small team I led were able to achieve a holistic macro and micro F1 Score of 86% across all classes. We then integrated this model into a frontend Flask web application where a user can upload any photo and see the top 10 class prediction probabilities as well as the overall prediction.

Code