Xyphien Forums

Hello Guest!. Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, sell your games, upload content, as well as connect with other members through your own private inbox!

Your experiences/thoughts on writing automated tests for plugins?


Lately I've been trying to write some automated tests to reduce the chance of my plugins to have nontrivial bugs unnoticed. This helped me catch quite some issues that I'd have missed for a long time otherwise, especially when developing a advanced complex battle system plugin. Of course, I still have to debug and dry-run my code to reason about my codebase, but automated tests can help me as well.

For instance, if my plugin relies on some invariants and I accidentally break some of them when adding some new features, those automated tests(which catch regressions in this case) might be able to inform me about that immediately, so I can fix the issues more effectively and efficiently.
Similarly, if my plugin has some compatibility issues with some other plugins and there are automated tests that can inform what assumptions from which plugins are violated(even though some of them should be violated in some cases), I can find the root causes more quickly.
Another useful scenario is to check inputted parameters/added notetags from plugin users to see if the contents are valid, even though it's more like assertions than automated tests. With a well-presented error log to plugin users(assuming that they do enable those tests), they might even be able to fix those invalid parameter/notetag values themselves without asking for the others for help.

Unfortunately, it seems to me that it's quite challenging to write testable high quality codes when developing plugins, especially with high test coverage(be it from unit tests, functional tests, integration tests, etc). For instance, I find it uneasy when trying to write many pure functions with well-defined inputs and outputs to increase testability while still conforming to the default RMMV codebase architecture and style, use dependency injections to facilitate swapping between real and test objects without tearing the plugin business logic apart, or writing automated tests that clearly specifies the preconditions and postconditions of functions solely for well-defined side effects without causing the tests to be tautological. Maybe that's just because I'm still inexperienced in writing testable codes in general :)

While some manual testings are inevitable(apart from the obvious parts like compatibility issues and how the plugin user would feel/think when using the plugins, which can only be tested manually), I still want to at least try to have a higher test coverage, like 80%, which is high enough for me in most cases. Maybe it's because I'm still a very bad junior programmer and don't know how to write tests, but right now the best I can get is 20%, which might be enough in some cases but still unsatisfactory for me(I've made a 10K LoC vanilla ES6 pet project via TDD and reached 80% test coverage, even though that's still nothing to brag for).

Strictly speaking, I guess I can get 80% test coverage with the heavy use of heavy mocking, like mocking a whole battler and even the whole battle scene, but I just don't feel that the return of investment of this approach's worth it in most cases. After all, setting up test fixtures and cleaning up mocks with quite some shared mutable states and implicit stateful dependencies can mean tons of tedious work that can be done all wrong way too easily(Not to mention the dedication needed to maintain such tests). Also, the time needed to run such tests might be rather long, but maybe that's just because I don't know how to mock properly yet.

Therefore, I'd like to know if anyone has any success in writing automated tests when developing plugins, and how you managed to get there(be it test after, test first, test driven development, behavior driven development, etc). Alternatively, if you managed to usually develop plugins effectively and efficiently, even including advanced complex ones like complicated and convoluted battle systems with lots of new features, with almost no nontrivial bugs, but nearly without the use of automated tests, I'd like to know how(like writing codes that are so easy, simple and small that no testings are needed? But how so for advanced complex plugins?) :D