This may be helpful for those wondering what a machine learning internship is like at a medium-sized and established company.
I’m doing a ten-week internship at Synaptics, which specializes in chips for interfaces, sensors, and IoT devices. Their chip is what’s used in Chromecast, the Google dongle. They’re currently working to integrate their chip into Alexa and other IoT devices. The past few years, they’ve acquired two startups and are moving into edge computing for the smart home.
My internship is under the newly-established IoT group, and my task to train video-object-detection models on their newest NPU chips for realtime TV. Object detection is the task of taking a frame and drawing a bounding box around an object of interest.
The end customer product will be integrating this video object detection model into one of Synaptics’s new NPU chip, so it runs in a phone/television in real-time. The challenge is to build models that are extremely computational-efficient so it runs right on their chip (very different from scaling cloud servers). Unfortunately that means there are only a few models I can work with. Fortunately, that means I get to study these few state of the art object detection models in depth.
By the end of week 2, I’ve trained a SSD, a state of the art object detection model, for detecting logos in images for relevant sponsor ads, on a dataset of about 3,000 images. Only once the model works well on images can we then apply it frame by frame to fit on videos. Here‘s my rudimentary model making a prediction.
The green box is the true label, and the blue one is my prediction. As you can see, this easy image only had one Nike logo, which it got roughly right, but with only 23-percent confidence.
Most of my first two weeks was reading papers, researching existing work, and going through the code of a repo that reproduced the SSD paper. Now that I set a baseline, I can start to improve the model pipelines for better performance. Hopefully next week will feature better demo pics I can showcase.