I have another case similar, my 1UZ. On cold starts, and it’s been extremely bad lately from being seasonally cold, it sounds and shows (on the tach) like its breaking up like one cylinder is not getting enough fuel, spark, or compression. If I try to drive it before the "popping" subsides (from warming up) it’s almost unbearable to drive it. If I let it warm up, I can pull usually to 4500-5000 rpms before I get any sign of breaking up. If I do not let it warm up, and start driving after reaching operating temps I can only pull 3000 rpms before it breaks up.
At first I thought it was power to the fuel injectors/ ignition coils. I made sure they both had ample power and the "popping" persisted. I recently replaced the fuel injectors, thinking maybe one was bad and the “popping” still were there. So that leaves, compression, I have not done a proper compression test but I have turned the crank over by hand. Literally I can take my two hands and turn the crank, vs. my spare engine which requires a rather large wrench to get it moving, both being cold when turned.
Maybe you need to do a compression test on your engine to see if there are any issues.