The most commonly requested addition to Gun Tests has been for us to renew testing on ammunition. We’ve been examining the issue of ammunition testing for nearly two years, and eventually we came to the conclusion that much of the published work on ammunition testing focused too much on terminal performance—that is, what the bullet did after it hit the target.
Certainly this is important, but getting the bullet on target is much more important, in our view, so most our focus is on performance before the bullet hits the target. Because we realize that results from our test gun can’t be extrapolated to every possible gun readers might own, we wanted to test ammunition in a fashion most consumers would be able to duplicate, and in fact, should conduct on their own. But our framework can potentially save you money and time by narrowing the list of candidates you might try in your own gun, and you can also follow our system to see if a given round performs for you like it did for us.
We don’t ask you take anything on faith. Most shooters are not able to do deflective glass testing, nine sets of different media penetration, etc., and there is no single protocol that can test for every eventuality. Instead, we focus on areas we deem much more important than tissue tests: Does the gun work with the test ammo?, Is it accurate enough?, and Can we shoot it comfortably?
We set stringent standards for reliability, consistent velocity, accuracy, and controllable recoil level — four crucial aspects of performance that allow you to put rounds on a target, paper or otherwise. We began with a number of loads representative of a certain type and ran them through our test protocol. If, at any point, a particular load failed one of the tests, it was thrown out. Only loads making the cut moved on to the next test. At the end, any loads meeting all standards were tested for penetration, expansion, and weight retention.
In more detail, here’s how we test.
Test 1: Reliability
Reliability is the non-negotiable baseline for a self-defense gun. Before anything else is considered, the gun must go bang every time you pull the trigger. We settle for nothing less than 100 percent reliability. If, at any time during the test protocol, the gun has an ammunition-related malfunction (for an auto pistol that means a failure to feed, fire, extract, eject; for a revolver it means skipping, binding, extreme stickiness during extraction), it’s eliminated from the test. The only exception is if we determine the malfunctions can be traced to the gun or magazine; i.e., we get similar malfunctions with several loads.
Test 2: Velocity
The ammunition must generate consistent velocities. Wild shot-to-shot velocity variances are not desirable in defensive ammunition. In addition to affecting bullet performance (accuracy, penetration, expansion), velocity swings make it much harder to fire the gun fast and well. A well-trained shooter needs to be very grooved-in on the recoil impulse generated by his or her particular gun/load combination. This is impossible if the recoil impulse changes markedly from shot to shot. You can get meaningful data on bullet velocity with as little as three shots; however, statisticians usually recommend 20 shots for more reliable results. We fired 20 rounds of each load over a CED Millennium chronograph. If any load showed an extreme spread (difference between the fastest and slowest shot) exceeding 100 fps, or a standard deviation of more than 20 fps, it was eliminated from the test.
Of these two measurements, extreme spread and standard deviation, the latter is by far the most important. It’s also a measurement not fully understood by many people, thus not often used by non-professionals. This is unfortunate because it’s the single most meaningful measurement of velocity results. The mathematical equation used to determine standard deviation is extremely complex, and prior to the existence of computers was seldom used by shooters. Fortunately, these days many chronographs, including the CED Millennium, will calculate standard deviation for you at the push of a button.
Standard deviation tells you how close you can expect the velocity of any shot you fire to be to the average velocity. Statistically, it’s been proven that 99.7 percent of rounds fired will fall within three standard deviations above or below the average velocity. 95.4 percent will fall within two standard deviations above or below. And 68 percent will fall within one standard deviation above or below. For example, if our 20 shots generate an average velocity of 1,000 fps with a standard deviation of 10, we can bet the farm that almost every round of that ammo we fire out of that gun (99.7 percent) will fall between 970 and 1,030 fps. Similarly, 95.4 percent will be between 980 and 1,020 fps. And 68 percent will be between 990 and 1,010 fps. Now that’s some meaningful data.
A small standard deviation is always a sign of ammunition that’s been assembled with skill and attention to detail. By the same token, a wide standard deviation can usually be taken as an indictment of quality control. The more consistent the ammunition, the better the odds it will produce the exact same result, both in accuracy and terminally, every time.
Test 3: Accuracy
Four 5-shot groups are fired with the test gun bolted into a Ransom Rest. To ensure consistent temperature and lighting conditions, all groups are fired at 50 feet, the maximum distance possible at the indoor range on which we do our ammunition testing, then group size is multiplied by 1.5 to extrapolate results to the traditional 25-yard standard. The four group sizes are averaged. Our baseline standard for accuracy in this test is 2.0 inches.
We expect variations between how our gun performs and how yours shoots with the same ammo brand. Accuracy results do vary from gun to gun. However, egregious accuracy in a quality gun is never a sign of good ammunition, and 2 inches at 25 yards isn’t that demanding a standard. For testing in your own gun, we recommend starting with our winners and seeing how they shoot in your gun. If they shoot well, then you can keep them as your personal defense loads.
Test 4: Recoil Level
Ammunition for self-defense must not generate such heavy recoil it’s difficult to control for the typical, reasonably well-practiced shooter. The method we use for assessing shooter reaction to recoil is IPSC power factor (pf). Take the bullet weight in grains, multiply it by the velocity in feet per second, then divide by 1000. The resulting number is the power factor. Example: A typical 9mm defense load is a 115-grain jacketed hollowpoint moving at 1,150 fps. 115 x 1,150 = 132,250/1000 = 132.25 pf. Power factor was originally intended to measure the amount of stopping power generated by a particular bullet moving at a particular velocity. Today we know any such correlation is illusory. However power factor is an excellent way to compare how much recoil is generated.
In our subjective experience, most shooters do best with a recoil level not exceeding 150 pf. Ideally, the 125 to 150 pf range seems to give a good balance of power with controllability. For cartridges necessarily generating more recoil, like .45 ACP because of its bullet weight or .357 Magnum because of its high velocity, we could raise the threshold to 200 for the well-trained shooter who’s inured to that level of recoil. Anything above 200 is going to be difficult to control in accurate, self-defense type rapid fire for all but the most hardened pistoleros. Therefore, generating over a 200 pf is an automatic disqualification.
Test 5: Penetration, Expansion And Weight Retention
Any rounds meeting all criteria for reliability, consistent velocity, accuracy, and controllable recoil are then fired into water. According to Ed Sanow in Handgun Stopping Power, the book he wrote with former Detroit Police detective Evan Marshall, water generates the same results, vis-a-vis penetration and expansion, as 10-percent ordnance gelatin. It just slightly overstates expansion and penetration. Take the expansion and penetration results in water, delete 10 percent, and you know what a bullet would do in gelatin. Water also correlates very well to what a bullet can be expected to do in living tissue, especially lungs. Even large police agencies using sophisticated gelatin techniques for ammo testing often doublecheck with water as a failsafe.
Before we go further, we should make it clear our water testing gives only a rough estimate of penetration to the nearest 3 inches. This is not a protocol that will tell you, “Ah, yes, precisely 8.3794 inches of penetration.” It’s more like, “Ah yes, 9 inches.” Or 12, 15, etc.
If you want to check our results at your range, gather a number of empty gallon milk jugs, at least three per load to be tested. Fill with water and cap them. Line them up in a row, 10 feet from the gun muzzle. Ask the permission of range personnel before doing any of this, of course. Some people who water-test bullets actually build “water boxes” to hold the jugs in a row, but it’s not really necessary.
Carefully align the gun so the bullet will travel through the bottom “fat” part of every jug. Fire the gun. Water will fly everywhere. (This portion of the ammo testing is done on an outdoor range, for obvious reasons.) The bullet will wind up inside one of the milk jugs. A gallon milk jug measures 6 inches from front to back. If the bullet is inside jug 2, you’ve got a hole in the front but no penetration in back, count that as 9 inches of penetration (6 inches for the first milk jug, then figure half the second jug equals another 3 inches). If you have a hole in the back of jug 2 where the bullet whacked and penetrated, but the bullet is still inside that jug, count that as 12 inches of penetration. And so on.
Uncap the jug, pour the remaining water and bullet out into your hand. Voila! You have an expanded and recovered bullet. Take a plastic trash bag with you to the range and carry away the shredded milk jugs, to keep the range clean. The only sign you were there will be a bit of water on the ground that will swiftly evaporate. At Gun Tests, our prejudice is that, for a self-defense load, we would prefer the bullet come to rest in jug 2. Stopping in jug 1 would equal 6 inches or less of penetration (about 5 1/2 in gelatin), which could translate into a failure to reach vital organs on an oblique shot, especially if an arm or other barrier was hit first. We could live with having the bullet stop in jug 3, though that would be toward the deep end for our preferences. If the bullet totally penetrates all three jugs, it is dangerously over-penetrative for self-defense and dangerous to innocent bystanders. The only situation in which we’d want to see the bullet travel beyond jug 3 is if we were testing rounds for hunting and deemed more penetration desirable than would be appropriate for an antipersonnel load.
The recovered bullet is measured for expansion with dial calipers, and on an electronic scale for weight retention (i.e. how much of the bullet is left after penetration and expansion).
The Bottom Line
When all is said and done, we will be able to recommend ammunition put together with care and craftsmanship, functionally reliable, generating consistent velocities, that is accurate and controllable as well. Penetration, expansion, and weight retention data on such loads is provided for your information.
