Even a brand new battery is only good for a few starts if the alternator is bad. The clicking from the stock relay is telling you that it is good, not bad. If you had no click, it would be time to check it out. The click tells you that the signal is getting that far. After that, your problem is either in the wiring harness, the starter, or a weak battery.
If you don't have a multimeter, invest $20 on one, and test your battery first. Then test to see how much voltage you are getting out of the wire leading to the solenoid. If you get 12+ volts there, the problem is in the starter. If not 12 volts(assuming battery is tested good), it's in your wiring harness. The aftermarket relay is designed because our wiring harnesses are old and getting older. Pre 89 can't even be replaced with new anymore. The relay allows for a weaker signal to trigger a direct wire from the battery as a substitute.