It sounds like you have the E-TEC V4 engine with 115-HP. At 390-lbs it is a heavier engine than the three-cylinder 90-HP 335-lbs engine that is often used for re-powering a MONTAUK 17. The boat trim will be down-by-the-stern. You are over the rated power. But there is no point talking about that, because you already bought the engine.
For optimum performance, the
full-throttle engine speed of a 2008 E-TEC 115-HP should be in the range of 5,600 to 5,700-RPM.
The gear ratio should be 2.25:1 from what information I can find.
In a prior discussion, a 1987 MONTAUK 17 with an E-TEC 90 reported a top speed of 42.5-MPH using a 17-pitch propeller with minimal gear and one person aboard. We can estimate the hull weight was 1,800-lbs (using the
Crouch speed prediction formula).
Using that data we can predict the boat speed for 115-HP and 1,860-lbs will be 47-MPH.
On that basis, a propeller pitch for a 2.25:1 gear ratio and engine speed 5,700-RPM that will produce 47-MPH (allowing for a SLIP of 10) will be around 21-inches. For all-round use a good three-bladed propeller is an excellent choice. Be sure to get a propeller that fits the gear case diameter, the propeller aperture space, the number of splines on the propeller shaft, and has the correct thrust washers or spacers if not an Evinrude propeller.
The most important element in this case is getting the full-throttle engine speed into the recommended (and rather narrow) optimum range. If the 21-pitch is too much and the engine cannot accelerate into the optimum engine speed range, drop to a 19-pitch.
The speed is all about power-to-weight ratio, and there is no way to know what your particular 46-year-old hull is going to weigh when you get done with all the repairs. Old boats to not tend to become lighter as they age and acquire significant additions of new laminate and other repairs. Also, crew weight is influential on boats, and particularly on smaller and lighter boats. So if you are going to be out boating with four people that each weigh over 200-lbs, you won't get the same speed as if the boat had one person that weighs 175-lbs aboard.