I would listen to IJ and take his advise.
I am going through the swap as of right now. Been through 7ms, 1JZ and now a 2jz. If you don't know what you are doing, it can be costly and frustrating. Fortunately for me, I can do alsmost all of my work myself except for making pretty welds... lol.and welding can get expensive.
You are also talking about have a big turbo and a small turbo. Both at the same time or replace them as needed?
You have skills and knowledge of fabrication?
You may end up needing a manifold, IC, exhaust setup if you are going that route.
Wiring harness and sourcing swap parts like the tranny/bellsousing/clutch kit,mounts etc are some of the things you may want to think about as well.
Is it worth it? That is only something you can answer after you do tha swap. For me , it is.