Understanding Our Review Process
We break down our review process into four categories: : Offering, Technical, Real-World Usage, and Performance vs. Claims. We believe that each of these categories represents the core of every products. The categories have various points associated with them for a total of 100 points per product. This allows each product to be easily compared within their own categories. We chose the weighting of the categories to reflect our opinions of the usefulness of the product.
Offering (10 points)
When a company introduces a product to market, they are trying to create value for their customers. As such there are many things that go into the product offering. Some of this is related to the marketing material (how does the product look, what is written on the back), user-friendliness (if the instructions are clear and easy to follow), the amount of product you get for your dollar (including if they offered any bonus-sizes or add-ons), and more. These elements are what we call the tangibles in the offering category. We will score the tangibles out of 5 points.
Intangibles are the things about the product that make you feel good about using it. It could be something to do with the marketing, but it could also be about how the company operates. For example, we will score products that are environmentally-friendly, or those with corporate social responsibility goals, higher in this section. We score these intangibles out of 5 points.
Technical (30 points)
This is where we look out how the products perform in a ‘lab environment’. This is typically referred to as quantitative testing. This is where we will create repeatable experiments, test the product, and measure the results. We can directly compare similar products using the same test in the same conditions, over and over again. This is important, because this allows us to effectively rank products against each other.
Real-World Usage (50 points)
This is where we use the product out in the wild. Our real-world testing will involve using the products on our own vehicles. This means different situations, environments, applications, materials, etc. We will evaluate our experience using the products. Perhaps they are hard to use or apply, or maybe there are some stains or marks left over. This part of our testing will find out what you’ll likely experience when you buy the products. Because this is largely based on our opinion, this is called qualitative testing and it is subjective.
Performance vs Claims (10 points)
Sometimes the lab results do not equal the results you get in person. This can be one of the most frustrating things about buying a new product: you did all the research, you watched a few videos…and it doesn’t work. At the end of the day, lab tests are irrelevant if they don’t work in the real world. This category ties together the marketing from the earlier Offerings category, and the results from the Technical and Real-World Usage to determine if ‘what you see is what you get’. For example, if a product claims to be ‘long-lasting’, we will check to see what that time-frame could be: it might be a few hours when you thought it might be a few weeks.
Performance vs Claims is a critical category because it proves or disproves whether a manufacturer is telling the truth. This category is a mix between quantitative and qualitative testing. In theory, we should be able to measure the manufacturer claims, so long as they are not too vague. However, some of the judgement will be opinion-based.
Product Review Serial Number
After every review, we issue a product rating out of 100 points. We also want to make sure that our ratings cannot be manipulated by others taking our brand and photoshopping the scores. Our solution is that we issue a digital seal of the score. This is a digital seal has a serial number which uses a hash string to maintain data integrity. The serial number is created by using product details and the rating. You will be able to look up the serial number in our database and compare it to the score of the product you’re interested in. This is another way that we aim to build your trust in us.
Help Us Create Better Reviews
Is there something we are doing in our review process that you would like to see different? Are there testing methods that you know of that you think will work better? We are always open to improving our methods. Please feel free to contact us with your suggestions.