Ken Zheng Software Engineer

The Archive

Here's a pretty exhaustive list of every project I've worked on during my college and graduate years. This page is very much a work in progress, and more details will be added to each project eventually!

  • Farm of the People

    A 2D farming game with a dark humor twist - made for Global Game Jam 2023
    CMU | 2023
    Unity C# Game Programming Game Jam
    w/ Brandan, Josh, Chelsea, Shelly

    This was my first in-person game jam, I had a lot of fun working with a group from my game development class on this project! I took the lead on programming, developing most of the core game mechanics and integrating everyone’s work into the Unity project. Check it out on itch.io!

  • This Website

    Built with Jekyll
    Personal | 2023
    Jekyll HTML CSS

  • Photobook - On-Device Image Captioning

    A live image captioning system running on a Jetson Nano
    CMU | 2022
    Image Captioning Multimodal On-Device ML PyTorch
    w/ Rosa, Anudeep, Perez

  • MIIS Capstone Project - ESPnet ONNX

    Optimizing the ESPnet speech toolkit for production applications
    CMU | 2022
    Speech Recognition Inference Optimization PyTorch ONNX ESPnet
    w/ Karthik Ganesan, Samarth Navali, Masao Someki, Prof. Shinji Watanabe

    In this project, we optimized the open-source ESPnet speech toolkit to support ONNX exporting via the PyTorch ONNX API.

  • NeuroMV

    A Neural Music Visualizer
    CMU | 2022
    Image Generation Music Neural Network Design PyTorch
    w/ Kayo Yin, Maya Shen, Kevin Chen

    In this project, we visualize songs using images that pulsate and move along with music using a type of neural network called a Compositional Pattern Producing Network (CPPN).

  • Pipeline models for VQA on VizWiz

    Tackling challenges in visual question answering for the blind community
    CMU | 2022
    Visual Question Answering Multimodal PyTorch
    w/ I-Tsun Cheng, Qi Jiang, Lyuyang Hu, Yuanyuan Wang

  • Song of the Elements

    A 3D VR spellcasting game - made for Global Game Jam 2022
    Personal | 2022
    Unity Blender Music Production

    This was my first experience at a game jam, with a team I found remotely through Discord. For this game, I was primarily responsible for 3D environment art, level layout in Unity, and sound/music. Check it out here!

  • Planetary Defender

    A simple arcade-y space shooter game made with Unity
    Personal | 2022
    Unity C# Game Programming Blender
    w/ Dan, Esmolis, Jeremy, Rafael, Sanjit

    This was my first major project in Unity, made to learn the fundamentals of the game engine. All of the programming and art assets were made from scratch by me. Play it on itch.io!

  • Fig-QA - (Published @NAACL 2022)

    Testing the ability of language models to interpret figurative language
    CMU | 2021
    Natural Language Processing Dataset Creation Language Modeling Explainability
    w/ Emmy Liu, Chen Cui, Prof. Graham Neubig

    In this project, we designed and crowdsourced a novel benchmark dataset containing pairs of similes/metaphors along with their literal interpretations. We then analyzed the performance of modern language models to understand their strengths and limitations for this task. Published at NAACL 2022.

  • Vision Techniques for Speech Emotion Recognition

    Adapting computer vision models and techniques for SER
    CMU | 2021
    Speech Processing Computer Vision Emotion Recognition PyTorch
    w/ Dong Hyun Lee, James Li, Sanika Natu

    In this project, we adapted pre-trained image classification models like ResNet and ViT to the task of speech emotion recognition. We found that using the pretrained weights had some benefit over training from scratch even though the domains are significantly different.

  • Unsupervised audio labeling

    Investigating pseudo-labeling techniques for a large audio event dataset
    CMU | 2021
    Audio Classification Unsupervised Learning PyTorch
    w/ Ankit Shah, Prof. Bhiksha Raj

    I was given access to the YFCC-100M dataset, which contains over 8000 hours of audio data scraped from videos on Flickr. I created a preprocessing pipeline to extract and clean the audio data, and investigated some techniques to perform pseudo-labeling and self-training using the data.

  • MIDI Transcription System

    A signal processing system to automatically transcribe monophonic melodies to the MIDI format
    JHU | 2021
    Signal Processing MATLAB Music MIDI

  • Weakly Supervised Robot Tool Segmentation

    Testing weak labels for the EndoVis 2017 Robotic Instrument Separation challenge
    JHU | 2021
    Image Segmentation Computer Vision PyTorch
    w/ Nhat Le, Wally Niu, Jared Pangallozzi

  • Gecko

    An exam system designed for computer science courses at JHU
    JHU | 2020
    React Java Web Development
    w/ Brandon Lax, Owen Campbell, Sadie Garber, Dan Chong

    https://github.com/jhu-oose/2020-spring-group-Gecko/tree/main/project

  • Synthesia (EP)

    An EP of original music I produced
    Personal | 2017
    Music Production Logic Pro X

    Synthesia is a 5-track synthpop EP that I produced and released back in high school, but I’m still very proud of it! Take a listen below - my favorite tracks are Sensate and Right Side.