In the first portion of Sport Rider's oil test ("Oils Well That Ends Well?" August 2003), we covered the overall makeup and functions of motor oil to give you a basic understanding of its role in the performance of your engine. In this portion-the second and final part of the article-we go into a detailed analysis and testing of 22 oils to see what makes them different from one another, including comparing motorcycle-specific oils to automotive products. We also run a dyno test to see if the claims of increased horsepower made by some oil producers are really true.
Spectrographic Analysis
Presented first is the spectrographic analysis of each of the tested oils. Using units of parts per million (ppm) to show the amount of additives in each product, this test utilizes an atomic emission spectrometer to measure the wavelength of light emitted from each oil sample as it is "ionized;" in simplistic terms, this is similar to sticking the oil into a microwave oven, then using a prism to split the light emitted as the oil burns. Since each element has its own light wavelength, a computer compares each light measurement to a standard emission, and then calculates the amount of that particular element.
We called on Analysts Inc. in Norcross, Georgia (www.analystsinc.com, 800/241-6315), to perform the spectrographic analysis testing. An ISO-9002-certified facility (meaning their lab meets strict worldwide quality-control specifications), Analysts Inc. has been in business since 1960, and is considered one of the top oil-testing labs in the country. They are able to identify extremely small amounts of metals and additives, and in some cases can detect as little as one ppm. If you send them used oil for analysis, they can generate a metal contents report that will help you discover internal engine problems before they occur. Most large diesel fleets use this to determine maintenance schedules.
This type of analysis also reports the absolute viscosity of the oil, and the total base number (TBN). The TBN is determined by measuring the milligrams of acid neutralizer (potassium hydroxide) required to nullify all the acids present in a one gram sample of oil. Viscosity retention and TBN are very important in deciding when to change your oil. A TBN of three or less typically denotes a failure of the oil to absorb acids. Oils with a higher initial TBN are therefore better suited for longer change intervals, assuming the base oil is of sufficient quality to maintain its specified viscosity over time. The subjects of base oil quality and viscosity retention are very complex, and are discussed later.
These elements are the most commonly discussed because they are one of motor oil's most important components. Several additives fall into this group, including phosphorous. The maximum level of phosphorous allowed in some automotive oils has been reduced by the new API standards, due to its effect on catalytic converters. Zinc is another additive in this group, as is molybdenum, usually referred to as moly. These antiwear additives serve as a back-up to the oil film in protecting engine components. They are activated by heat and pressure, forming a thin layer between metal parts that would otherwise come in direct contact, preventing permanent engine wear.
Looking at the graphs, it's interesting to note a wide variation in additive amounts. For instance, examining phosphorous levels in the antiwear additive graph (remembering the API limitations) shows that two automotive oils contain approximately 1000 ppm (Valvoline and Castrol Syntec), while the Mobil 1 product contains 1391 ppm. The average of the motorcycle-specific oils is 1322 ppm; the automotive oils average 1157 ppm. The Maxima Maxum products have the highest levels overall, with almost three times the amount found in the lowest product tested. The products with the lowest levels are Silkolene Comp 4, Yamalube and Honda HP4.
A similar correlation can be seen with zinc. The Maxima products again show the highest levels at almost 2000 ppm, while the Yamalube and Silkolene products again end up on the bottom of this list. The difference here between automotive oils and motorcycle-specific products is not as great, presumably because this additive is not regulated by the API. In fact, Valvoline is the only auto oil containing less than 1400 ppm. While the average motorcycle-specific product contains 1414 ppm, the automotive oils average 1328 ppm-not a huge difference.
Moly is often referred to as a friction modifier, but it is actually a solid metal dispersed in some oils. Because it has such a high melting temperature (4730 F versus 2795 F for iron), it works great as a high-temperature, high-pressure antiwear agent. Some claim that because moly is so slick, it can cause clutch slippage. In fact, some motorcycle manufacturers specify oil without moly due to this problem. The moly issue is such that Honda offers its HP4 both with and without it. Looking at the moly graph data, however, shows that even Honda's "moly-free" product contains 71 ppm. Many of the products contain less than five ppm of moly, which is the threshold measurement on this test (meaning any amount less than five ppm will not be detected). Both Torco oils contain a significant dose of moly, while the Maxum Ultra and Motul 300V Factory contain far less. The Mobil 1 automotive oil contains 92 ppm, while the MX4T motorcycle-specific version has an undetectable amount. Only six of the 19 motorcycle oils we tested use moly at all. Those that do, however, average 298 ppm. Considering that many oils contain five ppm or less, 298 ppm is a significant dose.
One common claim is that motorcycle oils have specific additives that are more suited for motorcycle engines. Based on an average of the three automotive oils we tested, the bike oils do in fact contain more of everything except calcium and boron. Note that the average moly content, which is often the friction modifier of choice, is higher in the motorcycle oils than the car oils mainly due to the three bike oils that use an extremely high moly content.
Acid Neutralizers
We charted the three most common additives (boron, calcium and magnesium) used to neutralize acids produced inside an engine during combustion. In this category, we can see that the car and bike oils are different in some cases. Every company seems to agree that some dosage of calcium is required. The highest amount is Amsoil at 4843 ppm, which explains its very high TBN of 14.42. Amsoil does not use significant dosages of either magnesium or boron, though; many other oils use both of these to bolster their acid-fighting ability. Maxum Ultra contains only 986 ppm of calcium, but supplements that with the highest dose of magnesium in the test at 1275 ppm. The Mobil MX4T product uses 699 ppm of magnesium and 221 ppm of boron. Another difference between the auto and bike products offered by Mobil is the use of magnesium. Mobil 1 automobile oil contains only 33 ppm of magnesium.
Another common claim is that the higher price of motorcycle-specific synthetic oils allows oil manufacturers to use not only better and more heat-resistant base stocks (as shown in the heat aging data), but also more additives. Our averaged data shows that in general, the synthetic oils contain as much or more of each additive. Note, however, that we only tested two motorcycle-specific petroleum oils, and results could vary with more oils tested.
Looking at overall averages, the bike oils have an average of 1986 ppm of calcium versus the car oils' 2702 ppm. While the bike oils average 296 ppm of magnesium, the car oils muster only 54 ppm. Since many of the bike oils do not use any boron, their average is only 96 ppm compared to the car oils' 116 ppm. However, looking only at bike oils that use boron as part of their additive package, the average is 253 ppm. The bike and car oils are clearly different in this category.
This photo of two oil samples...
This photo of two oil samples before and after the heat test shows how some of the Group III synthetics are now using better-quality base stocks. The two top tins show Castrol Syntec automobile (left) and Motul 300V (right) before the heat test; the two lower tins are both oils afterward (again, Castrol left, Motul right). Note the similarity in color between the oils in the post-heat test samples. It's pretty obvious which of these products should do the best job of keeping corrosive acids in check when looking at the TBN. Topping the list is Amsoil, both Motul products and the automotive oil Castrol Syntec. A lower TBN does not mean the oil is bad, it just means that the drain-interval potential is not as great. If you change your oil every 1000-2000 miles, then you shouldn't be concerned with this value. Others should take at least a cursory look at this category, however.
It's interesting to note a trend toward longer oil-change intervals in the automotive world. For example, BMWs now come with factory-proprietary synthetic oil, and the on-board computer usually suggests oil changes every 15,000 miles or so. However, BMW engines have oil sumps larger (their 2.5L six-cylinder holds seven quarts) than most similarly sized engines, as well as high-capacity oil filters. Mercedes-Benz follows a similar plan, using full synthetic oil with a change interval of 10,000-16,000 miles. Being the skeptical type, we tested oil from a BMW engine at 7500 miles, only to find the oil within viscosity and all other standard values-meaning it could have been left in longer.
The truth is that engine oils are better than ever with regard to base stocks, as well as viscosity improvers and acid neutralizers. If you don't have a 12-month riding season, you should add an extra oil change before you winterize your bike to prevent that used oil (with corrosive acid buildup) from sitting and possibly damaging your engine internals. As long as your engine isn't highly stressed, whether through competition or extreme mileage, our suggestion is to simply follow the change interval specified in your owner's manual, and spend more time riding and less time worrying. Of course, this assumes that your engine is in good mechanical condition; problems like fuel or coolant diluting the oil could mean disaster a lot sooner than 1500 miles.
Evaporative Heat Stability Test
The oil inside your engine is subjected to an extreme environment. Sure, the coolant-temperature gauge may only show 200 F, but there are many internal engine parts that become far hotter. In order to determine each oil's ability to survive in such a climate, we subjected samples to a test commonly known as the Noack method. This test takes an oil sample and cooks it at 250 C (the estimated temperature of the piston-ring area, which is the hottest an oil should get) for one hour. Before and after the exposure, the sample is carefully weighed on a precise laboratory scale. Because parts of some oils are unstable at these temperatures, they burn off during the test, and that loss can be accurately measured.
The higher the percentage of weight retained (meaning less oil has burned off), the better. As you can see in the charts, there is quite a difference between the best and worst oils. The top product on this test is the Mobil 1 car oil at 96.1 percent. What is not so clear is that Group III oils (synthetics processed from a mineral-base stock) like Castrol Syntec and Motorex Top Speed test about as well as Group IV (PAO synthetics) and V (ester synthetics) products such as Motul, Bel Ray, Maxum and Torco. This shows that Group III oils are getting better and more heat stable (i.e., using better base stocks) for these applications than they were a few years ago.
We ran both bikes with standard...
We ran both bikes with standard petroleum automobile oil (Valvoline 10W-40) to do our baseline dyno runs. We then drained the oil, changed oil filters and ran the synthetic oil for at least 15 minutes to circulate it through the engine. As expected, the petroleum-based oils such as BelRay EXL, both Valvoline oils and the Yamalube and Torco synthetic blends are on the low end of the scale. Proving how good some synthetic blends are, top blend performer Castrol GPS actually out-performs one of the full synthetic oils (BelRay EXS). In general, however, the full synthetic oils are the winners here, with an average value of 93 percent, compared to the synthetic blends at 89 percent and the dinosaur oils at 86 percent.
We suggest you look at this data carefully and determine your needs before picking an oil for your bike. While not the only important factor, heat stability is one of the top issues because most sportbikes are tuned to the highest levels of performance possible, usually generating intense heat in the process. Engine oil must be able to survive these temperatures and not evaporate when you need it most.
We were as surprised as anyone that just changing oil can produce a horsepower boost. Both the R1 and GSX-R1000 posted some significant gains in midrange and top-end, and were gaining power with every run until coolant temps got a little too hot. Before you go rushing to buy this stuff, however, check out the viscosity retention test.
Dyno Test
Some oil manufacturers and their representatives claim that using their product will result in more horsepower. These are special ultra-lightweight-viscosity racing synthetic oils that are said to reduce the parasitic drag that oil has on an engine's internal reciprocating components. We decided to put these claims to the test-an actual dynamometer test. Two of the full synthetic oils in this test make these horsepower claims on their labels: Maxima Maxum Ultra (in 0W-30 and 5W-30) and Motul Factory Line 300V (in 5W-30). We took two open-class sportbikes-a Suzuki GSX-R1000 and a Yamaha YZF-R1-and ran them with common off-the-shelf Valvoline 10W-40 automobile mineral oil to set a baseline dyno run. That oil was drained and replaced with the 0W-30 Maxum Ultra in the Suzuki, and the 5W-30 Motul 300V in the Yamaha. After about 15 miles of running to get the oil fully circulated through the engine, the bikes were then dynoed again.
Lo and behold, both the Suzuki and Yamaha posted horsepower gains. While not an earth-shattering boost in power, the gains were far beyond common run variations, and weren't restricted to the very top end. The GSX-R1000 posted an increase of 3.3 horsepower on top, with some noticeable midrange gains as well; even more interesting was that the power steadily increased for several dyno runs (as the coolant temp increased). The Yamaha responded nearly as well, with a 2.7 horsepower boost on top. It should also be noted that while riding both bikes, there was a noticeable ease in shifting with the synthetic oils compared to the automobile mineral oil. Pretty impressive for just changing oil, in our opinion.
But before you go rushing to buy these products, it should be noted that these are racing oils, and, despite manufacturer claims of viscosity retention performance identical to standard viscosity oils, are made to be changed on a much more frequent basis. You should take a close look at the Tapered Roller Shear Test that demonstrates an oil's ability to maintain viscosity over time.
Four-Ball Wear Test
With an eye toward evaluating oil's ability to lubricate under extreme pressure conditions, we picked a few candidates and ran them through the "Four-Ball Wear Test" (officially designated ASTM D-4172). To conduct this test, we enlisted the help of the Southwest Research Institute in San Antonio, Texas (www.swri.org; 210/684-5111). SwRI is a huge nonprofit independent testing and engineering firm with an entire group of people dedicated to motorcycle-related products.
This test is used to determine the wear properties of engine oil in sliding contact (such as a piston sliding against a cylinder wall). Three half-inch-diameter ball bearings are placed in a triangular fixture, with a fourth half-inch ball in the center (in contact with the other three) held in place with a clamp. The balls are then immersed in the test lubricant while the top ball is spun at 1800 rpm for a period of one hour with a prescribed load of 40 kg (88 lbs.) and a constant temperature of 75 C (161 F). The "wear scar" on the three lower ball bearings is then carefully measured (in millimeters) using a microscope and averaged. The smaller the wear scar, the better the protection.
Because this test is expensive, we could not test every product listed in the spectrographic analysis, so we picked a few we thought would reveal the most information. We chose the Castrol GTX 10W-40 automotive oil because it is a simple Group II mineral-oil product that is widely used and inexpensive. As an example of motorcycle-specific oils, we picked the popular Mobil 1 MX4T motorcycle oil in 10W-40. It is a moderately priced full synthetic oil (approximately $8.99 per quart), and should represent all the technology and economy of scale that a large oil producer like Exxon/Mobil can offer. We also chose the Amsoil Group IV motorcycle oil. Amsoil makes product claims related to the performance of its oil on this test, so we decided to see if they could live up to their claims.
The four-ball wear testing did not show the huge variation expected. All of these oils basically perform the same. With any test there is some variation from sample to sample, and this data is so close we have to call it a tie, which means all these oils in their new, virgin state do a good job of protecting against sliding friction wear. Incidentally, Amsoil did perform up to the test claims stated on its label.
Tapered Roller Shear Test
We decided to conduct some additional testing aimed at evaluating an oil's ability to withstand the shearing loads present in a motorcycle gearbox (but not in the typical automotive engine). One of the claims made by most motorcycle-specific oil producers is that motorcycles present a different set of conditions than typical cars do, and that therefore you should spend more money to get oil formulated specifically for this environment. The meshing of transmission gears is said to shear or tear oil polymers over time, resulting in the degradation of oil viscosity and severely reducing its performance. As we stated earlier, this may not be so critical if you frequently change your oil. However, if you run longer than standard intervals, this oil property is something to strongly consider.
The test we selected to measure this effect is officially called the "Tapered Roller Bearing Test" (CEC L-45-99), commonly called TB-20. Recent trials have shown that this test provides the best correlation to actual performance compared to other industry shear tests. For the TB-20 test, a tapered bearing fitted into a four-ball test machine spins submerged in 40 mL (1.3 fluid ounces) of lubricant at 60 C (140 F) at a constant speed for 20 hours. The viscosity of the used fluid is measured and compared to the new/original viscosity, and the percentage of change compared to the original viscosity is reported. The higher the number, the more viscosity loss the oil experienced during the test.
We picked Valvoline 10W-40 automotive, Motul 300V 5W-40 Factory line, Mobil MX4T 10W-40 and Motul 300V 10W-40 oils for this test. Part of the analysis also involves the testing of a reference oil with a known viscosity performance in order to measure the variation between tests. In our case the reference oil had a total variation of 2.5 percent. This means that differences of 2.5 percent or less should be judged as the same, and that these small differences are related to the test method rather than product differences.
We also found the viscosity...
We also found the viscosity index, or absolute viscosity, of each sample. This is a measure of how long it takes for a set quantity of the oil to flow through a hole at a certain temperature, and is expressed in centiStokes (cSt). Unless noted, each sample is a 10W-40 grade. The actual viscosity raw data test results are expressed in centistokes (cSt), the scientific unit of viscosity measurement. However, after the percentage of viscosity loss column, we have converted the centistokes to an approximation of SAE grade to give you an idea of how much viscosity breakdown has occurred.
The various oils show large differences in their ability to endure this difficult test. The one commonly available automotive mineral oil tested suffered a 41 percent loss. While this limited data does not conclude that all mineral-based automotive oils are bad, it is definitely not a good sign. Looking at the motorcycle-specific oils, it's notable that the Motul 5W-40 version does not hold up nearly as well as the 10W-40 version (in fact, slightly worse than the auto oil). Motul and Maxima both claim that their ultra-lightweight-viscosity oils would last as long as normal 10W oils. Because we only tested the Motul version, we cannot say for sure that the Maxima Maxum Ultra would suffer the same loss. Yet our dyno test shows that both these oils post a horsepower gain. We consider ultra-lightweight racing oils such as 0W and 5W a special category of race products that should be changed on a strict regimen. Before you decide to run them, you need to weigh the risk of viscosity loss versus horsepower gains and make your own decision. Until more data convinces us otherwise, we would stick to something more practical for the street.
Conclusions
With all this testing data (and expense), you'd think making a clear-cut decision as to which oil is best would be easy. In the case of engine oils, however, there are too many products and variables that go into this equation. Due to the financial reasons stated earlier, not every test was run on every product, so crystal-clear conclusions aren't in the picture. You must weigh all the data we have made available; for instance, the fact that some oils may absorb acids better, but may not handle high heat as well. Or that while the four-ball wear test shows that particular automobile and motorcycle-specific oils perform identically, the heat and viscosity shear tests show otherwise.
We did, however, unequivocally answer a few questions. For one, most name-brand motorcycle-specific oils are indeed different than common automotive oils, even within the same brand, debunking a common myth. Mobil One automotive oil is definitely different than its motorcycle-specific version. The same is true for the three oils provided by Castrol, showing that both companies have different goals when formulating their automotive and motorcycle products. Whether they perform better-despite the data we've gathered-is still a matter of opinion. Another manufacturer, on the other hand, appears to have selected the same additives in both of its offerings, which begs the question: Are they actually identical and simply relabeled?
Once again, the final decision is up to you. It's your bike and your hard-earned money-so only you can make the decision whether to spend the extra bucks for full synthetic motorcycle oil or simple mineral-based car oil. Review the data we have presented, and select the product that is most suited to your bike and riding style.
This article originally appeared in the October, 2003 issue of Sport Rider.
Part 1: What is motor oil really made of?
More tech stories