I'm a Systems Reliability and DevOps engineer for Netdata Inc. When not working, I enjoy studying linguistics and history, playing video games, and cooking all kinds of international cuisine.
The better question though is: Is it worth the effort?
The problem with going for 100% test coverage is that it quite often ends up warping the design of the code you're trying to test, or making the tests confusing in some way.
As a quick example, consider a simple replace-by-rename routine for writing out whole files atomically (in simplified Python 3 here, but language doesn't really matter):
importosdefatomic_write_file(path,data):path=os.path.abspath(path)tmp=os.path.join(os.path.dirname(path),"."+os.path.basename(path)+".tmp")try:withopen(tmp,mode="w")asfile:file.write(data)except(IOError,OSError):print("Failed to write to temporary file!")returnfalsetry:os.rename(tmp,path)exceptOSError:print("Failed to rename temporary file to final destination!")returnfalsereturntrue
This seems simple to test, except there are a lot of possible conditions you need to be testing if you want 100% coverage:
Succeeds if the destination and temporary file don't exist.
Succeeds if the destination exists, but the temporary file doesn't.
Succeeds if the destination doesn't exist, but the temporary file does.
Succeeds if both the destination and temporary file exist.
Fails if it can't write to the temporary file due to an IO error.
Fails if it can't write to the temporary file due to an OS error.
Fails if the temporary file doesn't exist, but the rename fails.
Fails if the temporary file does exist and the rename fails.
All of the success conditions are easy to check, but you have to remember to check them. The failure conditions though, are trickier. You either have to override both the open() and rename() functions in some way for the test so that they throw appropriate errors, or you have to do creative things in the test case that rely on filesystem semantics and the exact behavior of the language the code is written in.
Realistically though, you probably don't need to care about differentiating those last two failure cases if you're properly checking all the success cases (because you'll already be testing write paths for the temporary file in the success cases). Additionally, if your app is going to bail anyway any time this routine fails, you may not even need to test the failure cases at all, and it may just be better to remove the try/except clauses and have the caller handle the exceptions (which then makes it easy to test the caller in the failure cases).
Something else to consider though. A lot of code coverage tools I've seen focus on either making sure every function is tested or making sure every statement is run during tests. Both of those are OK, but they're also not what you should generally be checking. What matters if you're looking at code coverage is what paths through your code are being tested (so, edges on the control flow graph). That will inherently cover every statement, and it will provide coverage info in a much more useful way than looking at functions or statements.
Possible? Yes. Something to focus on? Meh. There's most likely plenty of code in your projects that simply doesn't warrant automated testing. I would instead focus on having well-written tests for the the most crucial and/or delicate parts of your app.
Yes it's possible but getting 100% coverage should not be the only goal. In a big project, it might make more sense to identify the most important user scenarios and add tests for these scenarios, instead of covering all possible cases.
Btw there are different types of coverage (Function, Statement, Edge, etc).
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (3)
Yes, it's technically possible.
The better question though is: Is it worth the effort?
The problem with going for 100% test coverage is that it quite often ends up warping the design of the code you're trying to test, or making the tests confusing in some way.
As a quick example, consider a simple replace-by-rename routine for writing out whole files atomically (in simplified Python 3 here, but language doesn't really matter):
This seems simple to test, except there are a lot of possible conditions you need to be testing if you want 100% coverage:
All of the success conditions are easy to check, but you have to remember to check them. The failure conditions though, are trickier. You either have to override both the
open()
andrename()
functions in some way for the test so that they throw appropriate errors, or you have to do creative things in the test case that rely on filesystem semantics and the exact behavior of the language the code is written in.Realistically though, you probably don't need to care about differentiating those last two failure cases if you're properly checking all the success cases (because you'll already be testing write paths for the temporary file in the success cases). Additionally, if your app is going to bail anyway any time this routine fails, you may not even need to test the failure cases at all, and it may just be better to remove the try/except clauses and have the caller handle the exceptions (which then makes it easy to test the caller in the failure cases).
Something else to consider though. A lot of code coverage tools I've seen focus on either making sure every function is tested or making sure every statement is run during tests. Both of those are OK, but they're also not what you should generally be checking. What matters if you're looking at code coverage is what paths through your code are being tested (so, edges on the control flow graph). That will inherently cover every statement, and it will provide coverage info in a much more useful way than looking at functions or statements.
Possible? Yes. Something to focus on? Meh. There's most likely plenty of code in your projects that simply doesn't warrant automated testing. I would instead focus on having well-written tests for the the most crucial and/or delicate parts of your app.
Yes it's possible but getting 100% coverage should not be the only goal. In a big project, it might make more sense to identify the most important user scenarios and add tests for these scenarios, instead of covering all possible cases.
Btw there are different types of coverage (Function, Statement, Edge, etc).