Sure, and my issue isn’t that it’s capable of my requirements, but simple to use and meets those requirements. Simplicity is a quality all its own.
JUnit provides sufficient output to meet my requirements, so why overcomplicate it w/ Cucumber? AFAIK, cucumber tests in Java output JUinit as well (at least ours does), so what value exactly is Cucumber providing?
If you don’t want the scenarios that are readable in a business requirement format then it’s not worth the trouble. I haven’t ever seen anyone use it in that way.
These are for a mix of end to end and integration tests.
I mostly do unit tests as a dev, so our tests are simple enough that they don’t benefit from more structure than being grouped by suite. E.g.:
AuthService
valid user creds can login
invalid user creds cannot login
non-existent user gets same error as wrong creds
UserSettingsService
can change language
cannot set empty password
These don’t have long flows, so there’s no benefit to documenting steps (they usually have one step).
My complaint about cucumber/gherkin isn’t with documenting steps, it’s with managing them in separate files. We have a Service.feature file that documents the scenario and the ServiceTest.java that documents the steps. I don’t see the point in having those be separate files, especially since the only people defining inputs and scenarios are the devs (dedicated QA in our case). We occasionally have our BE devs help write a few tests, and every time it’s a struggle for them to figure out where everything is. It just feels over-engineered.
In unit tests, we parameterize our tests just like with cucumber, we just do so in the code. E.g. in Python:
@parameterized.expand([(1, 2), (2, 4)])
def test_duplicate(num, exp):
res = dup(num)
assert res == exp
I would much prefer something like that in our end to end and integration tests.
Maybe that’s why I hate using it, or maybe I just hate testing frameworks that do more than run tests. Here’s what I want from a testing framework:
Our QA uses cucumber and it works for them, so I only whine when I need to deal with it.
Cucumber can do all of that. Except for failures over time but I’ve never used a framework for that. CI tools typically track that.
Sure, and my issue isn’t that it’s capable of my requirements, but simple to use and meets those requirements. Simplicity is a quality all its own.
JUnit provides sufficient output to meet my requirements, so why overcomplicate it w/ Cucumber? AFAIK, cucumber tests in Java output JUinit as well (at least ours does), so what value exactly is Cucumber providing?
If you don’t want the scenarios that are readable in a business requirement format then it’s not worth the trouble. I haven’t ever seen anyone use it in that way.
I think there’s one question you should answer in order to fully describe your “testing framework”: is it being used for “end to end tests” or “integration tests” or “unit tests”? https://k-hartanto.medium.com/testing-pyramid-and-testing-ice-cream-cone-what-is-the-difference-6ddde3876c20
For unit tests, something like https://en.wikipedia.org/wiki/JUnit is useful. For testing a program after it’s deployed, something like https://jbehave.org/reference/stable/story-syntax.html is useful. You get different information about your program from each type of testing, and one type can detect issues even if the other didn’t, so doing both is useful.
These are for a mix of end to end and integration tests.
I mostly do unit tests as a dev, so our tests are simple enough that they don’t benefit from more structure than being grouped by suite. E.g.:
These don’t have long flows, so there’s no benefit to documenting steps (they usually have one step).
My complaint about cucumber/gherkin isn’t with documenting steps, it’s with managing them in separate files. We have a
Service.featurefile that documents the scenario and theServiceTest.javathat documents the steps. I don’t see the point in having those be separate files, especially since the only people defining inputs and scenarios are the devs (dedicated QA in our case). We occasionally have our BE devs help write a few tests, and every time it’s a struggle for them to figure out where everything is. It just feels over-engineered.In unit tests, we parameterize our tests just like with cucumber, we just do so in the code. E.g. in Python:
I would much prefer something like that in our end to end and integration tests.