2 types of testing - but only 1 that AI can help with
source link: https://www.mrlacey.com/2023/04/2-types-of-testing-but-only-1-that-ai.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
but only 1 that AI can help with ~ Matt Lacey
This has the opportunity to be valuable, but it's important to consider that there are two broad categories of tests when it comes to software. Each answers one of the following questions:
- Does the code do the right/correct thing?
- Does the code not do the wrong thing?
At first glance, you might mistake these for being the same. But there's a subtle difference.
Does the code do the right/correct thing?
Does it do all that it's supposed to?
Does it correctly implement all of the requirements?
By only looking at the code, it's impossible to know for sure. Well-written code should provide many indications about what it should do, and wider knowledge of the application and the domain to which it relates can also provide insight but without knowing the full requirements, you can't know for sure.
If all you have is the source code, you must assume it's correct. Until you find something wrong or that needs changing.
Does the code not do the wrong thing?Sometimes the requirements for an app might include a statement that "the code should not crash unexpectedly" or "the code should be secure." Often this is just implied. Who wants an app that crashes or is insecure?
Fortunately, these are the kind of things you can test for without any wider knowledge of what the code should do. This is how they're different from the first kind of tests.
AI (& other) tooling can be a really good way of creating tests that try and break code. If it finds something, and you fix the issue, you get better (more reliable) code. But be wary of relying on such tests alone.
Yes, having more tests is generally better. But beware of relying on these tests for identifying regressions and the introduction of bugs in the future.
One of the great benefits of having tests is that they not only help you verify the code is correct now but also that it isn't accidentally changed in unexpected ways in the future.
Having tests that verify the code does the correct thing (produces the correct output given specific inputs) is good. But, of greater benefit is that the tests can verify that the code still produces the correct output if you need to modify the internal code. Tests that only make sure that the app doesn't crash, given particular inputs, aren't as valuable in the long term.
It might be great to say you have lots of tests and to even have high test coverage, but if they're not helpful as the code changes, are they really all that useful?
Yes, "it depends"
It's not as black-and-white as I've laid out above. There are times when AI might be able to generate useful tests of whether the code is doing the right thing based on the names and patterns it identifies in the code. Or comments in (and around) the code may provide sufficient context to generate useful tests.
Your mileage may vary when it comes to having a tool (AI-based or otherwise) generate tests for you, but I've always found thinking about these 2 types of tests helpful.
Related Posts:
- installscript configure website to use specific version of ASP.NETAfter managing to register ASP.NET 1.1 the next step was to tell the newly installed website to use version 1.1 of ASP.NET, rather than the default.Th… Read More
What makes a great application?Really thought this was worth noting from the keynote of #uktechdays online conference today:
Q. What makes a great application?
A. A combination of… Read More
- The most important InstallShield settingIt's highlighted above.Change the Project File Format, to XML, from the default of Binary.This will enable to see the differences in the different ver… Read More
- installscript register ASP.NET 1.1 (if not registered)For some reason, one of our ASP.NET 1.1 based products is often installed on machines where ASP.NET 1.1 is installed on the machine, but not registere… Read More
- Gotchas when integrating with the Music and Video Hub #WP7DevIf you are making a Windows Phone 7 app which can, or needs to, integrate with the Msuic & Video hub, Microsoft have provided some instructions on doi… Read More
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK