So it seems it's the faces that cause the biggest problems with this, once I added all the faces to the cube and had everything updating on 0s timers, it became quite laggy. It's still better than what it used to be though.
Points updating every 0.1s but moving smoothly, faces every 0.1s but not smooth: 0_1562636748362_9ED684B2-4964-4520-8FDA-170409D82A3F.MP4
Faces updating every 0s (big frame rate drop): 0_1562636772122_12D26D09-2856-4FE9-8D55-CA4F16D51953.MP4
Faces updating every 0.1s and transforming "smoothly" (this just looks ridiculous, I feel like I should put an epilepsy warning here): 0_1562636890094_4B8C4623-C1A5-47E3-84B1-3CBE8F72FAA0.MP4