Urban rivers provide a water environment that influences residential living. River surface monitoring has become crucial for making decisions about where to prioritize cleaning and when to automatically start the cleaning treatment. We focus on the organic mud, or "scum", that accumulates on the river's surface and contributes to the river's odor and has external economic effects on the landscape. Because of its feature of a sparsely distributed and unstable pattern of organic shape, automating the monitoring process has proved difficult. We propose a patch-wise classification pipeline to detect scum features on the river surface using mixture image augmentation to increase the diversity between the scum floating on the river and the entangled background on the river surface reflected by nearby structures like buildings, bridges, poles, and barriers. Furthermore, we propose a scum-index cover on rivers to help monitor worse grade online, collect floating scum, and decide on chemical treatment policies. Finally, we demonstrate the application of our method on a time series dataset with frames every ten minutes recording river scum events over several days. We discuss the significance of our pipeline and its experimental findings.