posted 09-10-2009 08:22 PM ET (US)        

I wrote this article as a completely serious article. Apparently several readers thought I was writing this so they could add nonsensical comments or make light of all my points. What I present below is completely serious and I belive important information. Many boats and motors are compared by tests, and the results are often lauded as proof of superiority. However, the performance data collected in many of these tests is not completely unbiased. This article tries to point out ways in which bias and misrepresentation can enter into comparative test data. As for the heckers' comments, I have deleted them all.

Having been involved now for several years in the collection of performance data, I have found that obtaining accurate and meaningful data is difficult. It is quite easy to collect misleading and inaccurate data about performance. So many variables affect boat performance that it can be difficult to eliminate the influence of them and isolate the target parameter. Based on my experience, I can easily tell you how to collect misleading data.

Weather
Perform your testing under different weather conditions. To boost the performance, test on a cold day, in cold water, with low humidity. To decrease performance, test on a hot day, in very warm water, with high humidity.

Water
Perform your testing in different waters. Use favorable winds and currents to help boost your results if desired. To unfavorably skew results, use adverse current and wind. These influences can be substantial. If not controlled for in the testing, they render the results meaningless. To make things totally meaningless, use saltwater for some tests and freshwater for others.

Boat
Use different boats. By this I mean use the same model, but just use two different boats. Hull variations will tend to favor one hull over another, so pick the hull according to the outcome you wish to demonstrate. Hulls with more weight, less fairness, lack of stiffness, and irregularities in their surface will kill performance, which you can later attribute to another variable. Don't bother using boats of completely different types or models, as even the most naive boater understands that those comparisons are meaningless.

Propellers
Use different propellers. To boost performance, carefully select an optimized propeller. To kill performance, use an inappropriate or poorly chosen propeller.

Trim
Vary the trim on the boat. To boost performance, dial in the engine mounting height and trim. To kill performance, mount the engine too high or too low, and don't trim it for optimum.

Speed
Measure speed with a garden-variety GPS receiver. GPS receivers do not actually measure speed, but instead deduce speed from consecutive position fixes. Use a GPS with few channels, no precision augmentation, and slow updating. Pick the speed results to favor the desired outcome.

Fuel Consumption
Don't actually measure fuel consumption at all. Use your seat-of-the-pants estimate to calculate fuel consumption based on about how much fuel you bought during the week. If you do actually measure fuel consumption, use different techniques for each motor, and use different instrumentation. Adjust the numbers as you see fit, as no one can really question how you obtained them.

posted 09-11-2009 09:05 AM ET (US)            

The problem, then, becomes: where do you find an indoor, climate-controlled environment large enough to actually measure performance?

There are always certain problems or challenges with any scientific study and the data collection methods. The very fact that humans collect the data introduces things like bias, error and other problems.

I think that it would be a reasonable comparison to take two stock production motors and stock production boats (of the same model and options), weigh them and/or handicap one or the other to compensate for weight, fuel load, etc.

I agree that some calibrated instrument should be used to measure fuel flow data and speed.

To eliminate performance bias, have the vessel then set up by motor manufacturer's experts on scene, re-weigh the boats for accuracy and then run them with an unbiased third party expert operator.

posted 09-11-2009 09:43 PM ET (US)            

MPH = [ ( HP / LBS )^0.5 ] x C

namely, change C.

As a matter of simple mathematics, Crouch's formula shows that speed is more sensitive to changes in hull shape than it is to changes in applied HP or weight. A 4 percent change in C, for example, will yield a 4 percent change in speed. A 4 percent change in HP, weight or combination of HP and weight will only yield a 2 percent change in speed.

posted 09-14-2009 12:49 PM ET (US)         

Take the nominal propeller pitch as gospel truth. All 15" propellers are identical in pitch and function. Therefore, all calculations taken therefrom are accurate to four decimals.

posted 09-14-2009 03:46 PM ET (US)            

I don't believe that it does. The article JimH mentions doesn't run hull X with a Yamaha and find it goes slower than a similar hull with a Suzuki, and then improve performance of the Suzuki powered hull by making those mods.

It simply shows what everyone knows - that hull modifications can help or hinder a boat's overall performance.

That is why I disagree that pulling IDENTICAL production hulls that are outfitted the same with different motors and conducting a test is not legitimate - or somehow flawed.

Of course, testers could eliminate this potential data corruption by then swapping motors and conducting a second head to head test the following day.

I think the real place we see bias introduced in these tests is that the trim and outfitting of the boats supplied for testing is often different - even among the same model in many magazine shootouts - i.e. Running my 18' Boston Whaler with twin 90's against another 18' Boston Whaler with twin 90's would be flawed - mine has extensive mods to the deck (teak and rubber inlay), a forward platform built out of teak and I have a radar arch, flags and antenna dragging in the wind.

Two stock 18' Outrages with identical build lists, one with a Merc 150 and one with a Yamaha 150 of the same technology would be a more clean comparison.

posted 09-14-2009 04:26 PM ET (US)            

Firstly, Newt - there will not be a significant difference in the performance of two boats from the same mold - in the same water, wind, altitude, loading, et al. conditions. Certainly, the surface condition (fouled, painted, waxed/non-waxed) can make a difference in two "identical" hulls.

Frankly, I put minimal significance to "user" test data because many are, knowlingly or unknowlingly, biased, one way or the other, and may be done or reported in accordance with one's interpretation, bias or desire. And then some "tests" are conducted with in-accurate instrumentation.

Granted, some of my comments stem from my engineering experience regarding testing, instrumentation and accident analyses. Certainly, our instrumentation is not certified or traceable to the National Bureau of Standards - but it should be the most accurate instrumentation obtainable.

Crouch's formula is but an empirical relationship and the coefficient "C", for that given test configuration, is calculated - and conveys all other test variables such as boat surface condition, altitude, wind, water conditions, temperature, wind velocity and direction, salt/fresh water, altitude, fuel energy content, et al.

Therefore, to discuss a specific percentage change in Ctouch's "C" is meaningless.

Having said that - conducting really meaningful tests is difficult - and must be done objectively. Certainly, I, and I suspect the majority of others don't have the elaborate testing faciltiy to conduct a perfect test. As such, I guess that I intreprete CW "test reports" and other "tests" in the boating world as interesting - but don't put much weight on the report. ---- Jerry/Idaho

posted 09-22-2009 10:15 PM ET (US)            

Modern manufacturing processes with tighter controls may reduce variation, but this is not universal. Not every boat builder constructs boats which are identical.

Measurement instrumentation does not have to be absolutely accurate for test data to be comparable, as long as the same instrumentation was used for all tests. For example, a scale might be slightly out of calibration, but if all boats are weighed on the same scale, the difference in their weights is probably meaningful and can be used for comparisons.

posted 09-24-2009 01:34 AM ET (US)            

And doing a test with only one variable is very preferrable. If more than one variable, the test program and analyses are vastly more complicated. That is one of the "qualms" I have regarding tests by CW members - typically, there are many more than one variable - and only one or two data sets.

Using the same instrument for all tests alleviates the accuracy problem. About all one can do is to use the best quality instruments possible and calibrate, as possible.