Learn from other people’s mistakes
Whether you are thinking about building a product or planning your product launch, you should keep product testing in mind. Here are some statistics to consider:
- More than 30,000 new consumer products are launched each year and 95% of them fail (Harvard Business School)
- Only 40% of developed products make it to market, and only 60% of that figure will make revenue (the Marketing Research Association)
- 63% of survey respondents said they like it when manufacturers offer new products (NIelsen).
- Only 11% of new product consumers remain engaged after 52 weeks (Catalina).
- 57% of survey participants reported they purchased a new product during their last grocery shopping trip (Nielsen)
- 27% of consumers reported that they wished more products could make their life easier. (Nielsen)
Although there is no precise data to show how many products failed because of design flaws, failure to understand consumer needs, or failure to launch, we believe that launching new products requires more than just designing a product to fit a market need.
Successful product launches are a result of excellent product development strategies that involve, among others, rigorous product testing.
So what is product testing anyway?
Product testing is a method used to evaluate a product concept: the product’s properties or qualities, its features, overall performance, or how potential customers may use or react to a product
It is important for a company to perform product testing throughout the product development process. The company may do it at any step, from the ideation to development and even during manufacturing.
In developing a new product, it is crucial to start testing early with a minimum viable product – to test every prototype for its function, aesthetics, or user experience. It is also critical that testing be done by the right people throughout the development stage.
Product testing can be done by the designer, manufacturer, or independent laboratory. There are many standardized test methods, but the test itself often varies to fit each individual product.
Whether it is screening concepts, validating suitability for the intended product use, ensuring design safety, or solving manufacturing problems, companies use product testing to solve problems proactively and make for a smoother product launch experience.
We’ll be diving deep today into each aspect of product testing. If you’re looking for something in particular, jump there directly. This is what we will cover today:
What does product testing entail?
The testing process usually starts with a problem that needs a solution, or a question about a product that needs to be answered. Will changing the packaging of a food product prolong its shelf life and attract new users? Will upgrading the camera feature of a mobile device encourage current users to upgrade to this new model?
Product testing may occur in various stages throughout the development process. While testing may vary from project to project, there are some common types that may be similar across various industries.
1. Concept Testing
Concept testing involves asking customers questions about your product concepts and ideas prior to development. It explores the feasibility of a product idea or concept and evaluates the customers’ acceptance and response to it.
The primary purpose of concept testing is to see if customers like your product idea and to gauge whether they will actually buy it.
Depending on the type of product, concept testing often includes defining the goal of the testing, choosing the sample population, deciding upon a survey format, communicating the concept, measuring consumer response, and interpreting the results.
Many big brands invest a lot in market research and concept testing because it is necessary for building and maintaining great relationships with customers. And it is also crucial to developing successful and profitable brand.
2. Testing Design Conditions
Similar to mechanical testing, which helps you understand the mechanical properties of a material or a response to a given action, testing design conditions helps you understand the properties of your whole product or a sub-assembly. The aim is to find out whether a product will perform as it was designed to.
Here are some ways of testing design conditions:
Stress Testing
Stress testing is a type of material testing done on a product to see how it will react under design conditions. The goal is to test your product’s strength to ensure it can withstand the forces it will be subjected to. A test scenario is created to simulate an extreme design condition. The test scenario is repeated to confirm that the theoretical analysis behind the design and the integrity of the assembly.
Impact Testing
Impact tests are done on a product to know how it will behave when subjected to high loads for short periods of time. A common scenario is a drop test.
Cyclical Loading
Cyclical tests are done on a product to know how it will behave when subjected to repeated loading over time. This is often done for sports equipment (think snowboard bindings).
Environment Testing
Environmental testing simulates the different environmental conditions, mechanical stress, and material stress that products are exposed to during their lifetime. It shows how a product will survive and continue to operate as designed while being subjected to environmental conditions during storage, transportation, or end use.
Depending on the product, environmental testing can include:
- Climatic testing (temperature, water and humidity, altitude, Icing, etc)
- Mechanical testing (vibration, sand and dust, flammability)
- Materials testing (Salt fog and cyclic corrosion, UV and weathering, Chemical and fungus resistance, Fluid and groundwater immersion)
Exposing the product to the harshest conditions allows you to analyze risk and breaking points. Knowing this information will guide the development team to further.
3. User Testing
User testing, or usability testing, is the stage in the design process that exposes your product to real users and evaluates it to create human-centric products.
The goal of user testing is to detect product usability issues, with the aim of improving the overall user experience. High fidelity user testing comes after the development team has built an aesthetic prototype, at the later stages of development. The product is then exposed to representative users to monitor how they interact with the product in a realistic setting. This way, you can see exactly what works, if the product is easy to understand and use, and what unforeseen challenges your users face.
The data gathered from this test reveals the customers’ experiences with the product and serve as a basis for last-minute design changes (if issues are significant) or possible future product iterations.
A/B Testing
A/B Testing is a type of User Testing where two versions of a product feature are exposed to customers to determine which version they prefer and to show which product will most likely perform better in the market. A/B testing can aid entrepreneurs to develop products that will resonate most with users.
With A/B testing, teams can create apples-to-apples comparisons of a single product variant, to ensure their results reflect how actual users respond specifically to that variant.
A/B testing is often done by presenting two variations of a product to test subjects, and it can be done much earlier in the design cycle by describing the specific features verbally or with imperfect prototypes to gather valuable feedback.
4. Quality Control Testing
Quality control (QC) testing is a process to ensure that your product is of the highest possible quality for your customers. QC employs techniques to prevent issues with your product and to ensure an excellent user experience for your customers.
The aim of QC testing is to make certain that a product works and looks as expected and helps the development team identify problems during production, before shipping and launching.
Depending on the type of product, a lot of testing needs to be included in the production set-up. In the case of mass production, there are usually tooling samples (T0/T1/T2 samples), the first products made coming from the mass production tooling. Testing these may require a simple visual inspection, and in some cases, more thorough testing.
If you’re interested to learn more about mass production, check out our blog on The Price of Mass Producing Plastic Products.
Bonus: Market Testing
Market Testing is a process that businesses employ to evaluate the viability of their new products prior to releasing them in scale.
To assess its viability, market testing involves introducing a product to a test market, a defined set of customers (the ideal client aka client persona or avatar). The test market can be based on a specific segment, like demographics (female, age 25 to 45), or geographic areas. The test is generally done without the customer’s knowledge to acquire non-biased feedback on the product.
The goal of market testing is to evaluate the potential success of the product in the desired market. Market testing is done for a variety of reasons at different stages of a company’s growth. In early stage product design, market testing allows you to gather valuable feedback on the size of the market and even presales.
This is usually done by the marketing or founding team, a separate topic that we will discuss in another blog.
When should product testing be done?
Generally, product testing should be done throughout the entirety of the product development process.
The most successful products are tested regularly to demonstrate from proof of concept all the way to final product. By doing this, a product gets properly defined and refined as it goes through product development. This is to ensure that there will be no surprises once a product is finally launched.
Why is it important?
Here are the top reasons why product testing is important for anyone designing a product:
Catch product problems at the onset
With inadequate testing, some problems with new products surface only after they are sold, resulting in costly penalties or product recall. Product testing help to avoid costly errors by ensuring that only the right product gets to the market.
Better user experience
Product testing can help ensure that consumers understand what your products will do for them, and which products offer them the best value. It gives an insight into what the user likes and does not like about the product. One of the biggest pitfalls is the failure to properly communicate the purpose of the product to the end-users.
Fit for end use.
Product testing helps validate the suitability of your product to the end user. Satisfied end-users become raving fans and customers of your product, encourage repeat purchases and attract new customers and users.
Ensure product reliability and QA
Product testing ensures that the product performs as designed, consistently, and reliably. It is also crucial to in complying with regulatory requirements for products that operate in critical environments or undergo severe stresses.
Validate your product marketing claim
When launching a new product, it is best to check the claims made during marketing campaigns. Product testing, especially those that give out certifications, provide credibility and build trust for your product with your market.
Maximize the product development cost
The cost of developing or redeveloping products can be huge. Sometimes, manufacturers add features that customers do not value or do product tweaks that make little difference to the bottom line.
Product testing at the start of the product development process can ensure that every cost of research and development adds value to the final product, and prevent unnecessary costs. It can help gain confidence and de-risk the product development process.
Decreased Product Development Time
Testing can minimize time to market by catching flaws early on. This saves the firm from going back to an initial phase to look for and correct a design or manufacturing process mistake. The findings of product tests can also be used to inform work on the product in subsequent phases of development.
Reduce production costs
Companies use testing to refine their designs and production processes to cut production costs. Detecting problems early in the product development cycle lowers the cost of correcting such problems. Testing a product for quality before releasing it to the market can save companies money on refund, returns, legal fees, and ongoing support.
Challenges of Product Testing
Depending on the product and its intended use, testing challenges can range from simple to complex. Some of the more common ones we encounter:
Understanding the Product Requirements
Product requirements are the information that drives product testing. Often, the requirements are ambiguous, missing, and often incomplete.
Some questions to help address the challenges in understanding product requirements, provide the direction on how to define each feature, the estimated cost, and how requirements are prioritized.
Deciding on Cost vs. Risk
A manufacturer may choose to test every single feature of a product to balance safety and liability, but this process can be time and cost-prohibitive. Instead, the most rigorous testing is generally focused on where the biggest risk lies with the product application so that the team can fully comprehend and mitigate it.
Planning to Test
Test planning challenges often occur when there is no proper understanding and appreciation of product requirements. Scheduling can also be hard to define as failed tests require extended timelines, redesign, and retesting.
Selecting the Test Methods
The tests performed on a product should verify and validate the product design. When testing your product, do whatever is needed to understand how the product will hold up under possible and extreme conditions in its lifetime.
Managing and Controlling the Test Process and Data
Crucial to successful testing is controlling the test methods. As you test, make sure that you are in control of the methods and the variables involved. The less control you have, the lower the quality of your data.
Capturing test data in an organized manner will help you analyze the data efficiently and make design decisions accordingly.
The Cost of Product Testing
As with any business activity, product testing comes with upfront costs. Depending on the types of products for testing, what is the testing for, and some other factors, testing costs can vary. Here are some cost considerations:
Cost of Prototypes and Product Components
Some product tests require the destruction of the product or the prototype. Prototypes are not cheap during product development. In many cases, the test requires additional prototypes and some spare parts available to enable repairs. The more units you test, the higher your costs will be, but the better your confidence in the outcome.
Custom Fixtures
Product testing often requires customized test fixtures – complex tests mean complex fixtures, which will mean increased costs. It is important to not over-engineer a product if there isn’t a reason to.
Ancillary Test Equipment and Other Costs
Most product testing will require the use of test equipment that you will need to either rent or purchase. Some testing laboratories can provide the ancillary equipment needed for the test. Other costs may include shipping of the prototype or product to the test facility and the cost of traveling to the test facility with your product, and the resource costs for personnel involved in testing, including salaries.
Lab Cost
This comes with a big price tag. The type of test you perform will also impact the duration of testing, together with the cost. For example, highly accelerated life testing (HALT) allows quick discovery of a product’s weaknesses during the design phase or development stage. Using this test gathers faster results within a week, than using some traditional testing methods which can take more time to generate results.
Working with an experienced product development company can help you reduce the amount of testing time as they may be able to do it in-house.
In-House Testing Costs
You could also opt to test your equipment yourself in an in-house facility instead of working with an outside product testing lab. This approach comes with may mean higher up front costs for equipment or a lower fidelity test if your testing environment doesn’t meet the same standards.
The Costs of NOT testing
While there are significant costs that go with product testing, the costs of not testing a product can prove to be much higher:
- Increased Product Development Time
- Increased Product Failures and Defects
- Increased Repairs and Returns
- Potential Lawsuits
- Legal Issues Due to Non-Compliance
- Missed Market Share Due to Reduced Product Quality
Generally, the cost of product testing depends on what type of testing you do, how thorough you would want the test to be and the statistical confidence you want in your results, how complex the test will be, who needs to be involved, and what testing equipment needs to be acquired for testing to take place.
Let’s Recap
We’ve covered a lot in this article and as you can see, product testing is a very broad area that’s dependent on the product, the application, and the use case. Just remember to keep a few things in mind when approaching this for your own product:
- Test throughout: doing product testing earlier in the product development process can save considerable time and costs by uncovering issues early on
- Use different tests: different tests are suited to different products and different parts of the design process – do what suits your product at this point in its development
- Refrain from making assumptions: keep an open mind, confirmation bias is a very real thing – trust the numbers
- Test successful products: a successful product can have flaws too, so don’t idealize
If you want to learn more, check out our blog on product certifications and prototyping or learn about the process Ventrify uses to bring our client’s products to market here.
About the Author
Ventrify is a product design and manufacturing firm that helps entrepreneurs bring product ideas from concept to market. We take in fledgling ideas and bring them through our iterative design process to create products our clients can be proud of. Then, we work with manufacturing facilities worldwide to bring our clients the highest quality products at competitive prices.
If you have questions about using prototypes in your product development journey or need help, reach out to us through our Website, Facebook, or LinkedIn.
