Segregating waste accurately at an individual level is paramount for efficient waste management, especially considering the staggering 268 million tons of waste generated each year in the USA, a large portion of which is recyclable. To address this issue, we developed a Smart Waste Sorter (SWS), which is a portable device that can be placed on any waste bin. It uses deep learning models to identify whether a piece of waste is a battery, recyclable, compostable, or recyclable, or trash, and provides a real time alert if the user is about to dispose of the item incorrectly. To develop the SWS image classification model, we utilized a dataset of 4,122 images, obtained from a combination of publicly available and manually collected images from households over several months. We experimented with four models of varying sizes: VGG16, EfficientNetB1, MobileNetV2, and ResNet50, to investigate whether a smaller model could achieve comparable performance, given that our device is portable and requires a compact model that can operate on limited memory without internet connectivity. Our experiments showed that ResNet50 achieved the highest validation and test accuracy of 77.91% and 96.39% respectively over four categories, suggesting that smaller models can be effective. Our results demonstrate the potential of the SWS to improve real-time waste segregation at the individual level, while considering practical constraints for implementation. The proposed solution utilizes a Raspberry Pi to detect motion, capture images and classify them. Our solution provides an effective, practical, and low-cost method for accurately segregating waste and contributing to sustainable waste management.