Anyone fancy a curry?

For no reason other than it’s interesting, here’s a take on Currying in Java 8. The idea of currying is to convert a function that takes n parameters into one which can receive them one by one, or indeed all at once.

By using partial application to provide some of the inputs to a function we provided a pre-loaded function that just needs the input that the recipient most cares about, in order to do its job. For example, if streaming and mapping, you really only care about the object next in the stream in order to transform it, not any other parameters kicking around that will be the same each time in the transformation function.

Here lies my attempt at a currying library:

package uk.org.webcompere;

import java.util.function.BiFunction;
import java.util.function.Function;
import java.util.function.Supplier;

/**
 * Java implementation of currying
 */
public interface Curry {
   /**
    * Tagging interface for the {@link #curry(Curried)} function
    */
   interface Curried {
   }


   /**
    * A nullary function - takes no parameters, returns a value. Also a supplier
    * with a default for {@link Supplier#get} but requires the apply function
    * to make it consistent with partial application.
    * @param <R> type of return
    */
   @FunctionalInterface
   interface NullaryFunction<R> extends Curried, Supplier<R> {
      R apply();

      default R get() {
         return apply();
      }
   }


   /**
    * Unary function - takes one parameter
    * @param <R> return type
    * @param <T> parameter type
    */
   @FunctionalInterface
   interface UnaryFunction<R, T> extends Curried, Function<T, R> {
      /**
       * Can be converted to nullary function by full application of a parameter.
       * @param t input
       * @return a function that evaluates using the given parameter against the UnaryFunction
       */
      default NullaryFunction<R> asNullary(T t) {
         return () -> apply(t);
      }
   }


   /**
    * Binary function - takes two parameters.
    * @param <R> return type
    * @param <U> first parameter type
    * @param <T> second parameter type
    */
   @FunctionalInterface
   interface BinaryFunction<R, U, T> extends Curried, BiFunction<U, T, R> {
      /**
       * Partial application of binary function to yield unary function where the
       * first parameter of the binary function has been supplied already.
       * @param u first input parametr for partial application
       * @return unary function which takes next input parameter for full application
       */
      default UnaryFunction<R, T> apply(U u) {
         return t -> apply(u, t);
      }

      /**
       * Supply all values to return a supplier/nullary function
       * @param u first parameter
       * @param t second parameter
       * @return a nullary function that returns the equivalent of calling the binary function
       * with all its inputs
       */
      default NullaryFunction<R> asNullary(U u, T t) {
         return () -> apply(u, t);
      }
   }


   /**
    * A ternary function, which takes three inputs and returns a value.
    * @param <R> return type
    * @param <V> first input parameter type
    * @param <U> second input parameter type
    * @param <T> third input parameter type
    */
   @FunctionalInterface
   interface TernaryFunction<R, V, U, T> extends Curried {
      /**
       * The function that's being wrapped for partial application
       * @param v input 1
       * @param u input 2
       * @param t input 3
       * @return the result of applying the function
       */
      R apply(V v, U u, T t);

      /**
       * Partially apply the first two parameters to get a Unary function for the third
       * @param v first parameter
       * @param u second parameter
       * @return a function that can be called with one parameter
       */
      default UnaryFunction<R, T> apply(V v, U u) {
         return t -> apply(v, u, t);
      }

      /**
       * Partially apply the first parameter to get a Binary function for the third
       * @param v first parameter
       * @return a function that can be called with the remaining parameters
       */
      default BinaryFunction<R, U, T> apply(V v) {
         return (u, t) -> apply(v, u, t);
      }

      /**
       * Supply all values to return a supplier/nullary function
       * @param v first parameter
       * @param u second parameter
       * @param t third parameter
       * @return a nullary function that returns the equivalent of calling the function
       * with all its inputs
       */
      default NullaryFunction<R> asNullary(V v, U u, T t) {
         return () -> apply(v, u, t);
      }
   }

   /**
    * A bit of syntactic sugar to convert a plain old function into one of the above types
    * @param t a functional interface implementation - probably a method reference
    * @param <T> type of function we're going to return
    * @return a function cast as one of the partially applicable types
    */
   static <T extends Curried> T curry(T t) {
      return t;
   }
}

And here are some unit tests that give you a taste of using it.

package uk.org.webcompere;

import static uk.org.webcompere.Curry.curry;
import static java.util.stream.Collectors.toList;
import static org.assertj.core.api.Assertions.assertThat;

import java.util.stream.Stream;

import org.junit.Test;

public class CurryTest {
	public static String identity(String a) {
		return a;
	}

	public static String concat(String a, String b) {
		return a + b;
	}

	public static String concat(String a, String b, String c) {
		return a + b + c;
	}

	@Test
	public void curryTernary() {
		Curry.TernaryFunction<String, String, String, String> function = curry(CurryTest::concat);

		assertThat(function.apply("a").apply("b").apply("c")).isEqualTo("abc");

		assertThat(function.apply("a", "b").apply("c")).isEqualTo("abc");

		assertThat(function.apply("a", "b", "c")).isEqualTo("abc");

		assertThat(function.asNullary("a","b","c").apply()).isEqualTo("abc");
	}

	@Test
	public void rightHon() {
		Curry.TernaryFunction<String, String, String, String> function = curry(CurryTest::concat);

		Stream<String> names = Stream.of("Bill", "Bob", "Ben");

		assertThat(names.map(function.apply("Right", " hon "))
			.collect(toList())).containsExactly("Right hon Bill", "Right hon Bob", "Right hon Ben");

	}
}

Advertisements
Posted in Uncategorized

Stop fixing it already…

 photo MicrosoftFIX_IT_zpsoqz3jnck.pngOver in Fix it twice? I discussed why a second attempt at fixing an issue, to use hindsight of an actual fix as a way to improve the software, was a great idea. In this piece, I’d like to discuss the aim of bug fixing.

Why do we fix bugs?

Is it to make the error go away?

Well, kind of, but that’s only part of the story. The aim of fixing a bug is:

  • Make the software work
  • Fix the thing which caused the bug (which can be software, or process, or communication etc)
  • Make it harder to repeat the mistake

I don’t want software with fixes in, I want software without issues. I don’t want issue resolution, I want it to be hard to make that issue occur again.

Removing the source of a mistake and being mistake-proof are two sides of the same coin. However, it seems to be against human behaviour to fix the cause, rather than the side-effect of a bug, and it seems to be hard, sometimes, to identify the cause as something that can be mistake-proofed in future.

Here’s an example:

public class RouteSoFar {
    private final List<String> streetsVisited = new ArrayList<>();

    // adds the street to the route
    public void visitStreet(String street) {
        streetsVisted.add(street);
    }

    // get a fresh copy
    public RouteSoFar makeCopy() {
        RouteSoFar copy = new RouteSoFar();
        copy.streetsVisited.addAll(this.streetsVisited);
        return copy;
    }

    // show route
    public String getRoute() { ... }
}

In the above class we have an object that’s tracking a route by accumulating streets. Maybe the idea is to print out some routes to interesting stores radiating out from an initial street. In fact, that’s a nice algorithm to write.


    StreetMap streetMap = new StreetMap("Bordeaux");
    Street firstStreet = streetMap.get("Rue De Winston Churchill");
    RouteSoFar routeSoFar = new RouteSoFar();
    printRoutesToStores(firstStreet, routeSoFar);

    ...

void printRoutesToStores(Street currentStreet, RouteSoFar routeSoFar) {
    // we're now on this street
    routeSoFar.visitStreet(currentStreet.getName());

    // print interesting stores on this street
    currentStreet.getStores().stream()
        .filter(Store::isInteresting)
        .map(store -> store.getName() + " is at " + routeSoFar.getRoute())
        .forEach(System.out::println);

    // and recurse to all neighbouring streets
    currentStreet.getNeighbours().stream()
        .forEach(neighbour -> printRoutesToStores(neighbour, routeSoFar));
}

Ignoring for a moment the fact that the neighbouring streets could easily give us one of the streets we came from (let’s pretend it’s rigged not to) the above code looks like it will work, but won’t. You’d find out if you ran it that the routeSoFar doesn’t contain the route to the store, but instead is polluted with any street it’s visited. This is because it’s a mutable object and is shared between layers in the hierarchy of the program.

Ah. So that’s why there’s that makeCopy function, a kind of clone method, then? Do we just change one line?

    // replace this
    currentStreet.getNeighbours().stream()
        .forEach(neighbour -> printRoutesToStores(neighbour, routeSoFar));

    // with this
    currentStreet.getNeighbours().stream()
        .forEach(neighbour -> printRoutesToStores(neighbour, routeSoFar.makeCopy()));

}

It’s probably clear by context, that the answer I’m aiming for is no. If you’re pedantic enough to debate that it’s yes, then I applaud you. In this instance as written, yes would be fine… but then what?

In a real life project, I got into the habit of fixing bugs like the one above on a class with a similar purpose to the one above, by remembering to do that thing… Yes, if you’ve got this mutable object and it might travel beyond the method that owns it, then it might be better to send a copy, not the real thing.

How many times do you need to apply that fix to a piece of code before you realise you need to make it mistake proof?

The problem the above has is that it’s a mutable object being used naively by multiple parts of the program. If it were immutable, then you would have to knowingly change it, and you couldn’t accidentally have your copy changed. It’s a reversal of the paradigm, and the side effect is that the thing you’re interested in becomes slightly more conscious a thing to do, and the thing you don’t want to happen become impossible to do accidentally.

Here’s the same Route class made as an immutable object. I think you can see how it might be used.

// immutable pojo - no setters, but can be transformed into a new immutable pojo
public class RouteSoFar {
    private final List<String> streetsVisited = new ArrayList<>();

    // adds the street to the route and returns a new object
    // so you store the reference to that if you want to keep it
    public RouteSoFar havingVisitedStreet(String street) {
        // rather than have a clone method, we innately create a copy 
        // of this, modify that and return IT
        RouteSoFar newRoute = new RouteSoFar();
        newRoute.streetsVisited.addAll(this.streetsVisited);
        newRoute.streetsVisted.add(street);
        return newRoute;
    }

    // no need for a copy/clone method

    // show route
    public String getRoute() { ... }
}

Philosophically, this is what bug fixing is – finding a mistake and eradicating it. However, spotting an opportunity to remove the opportunity for future errors of the same sort is a ninja trick. Use it!

Posted in Uncategorized

Fix it twice?

 photo MicrosoftFIX_IT_zpsoqz3jnck.pngOne of the many catchphrases I use with my team is Fix it twice. This refers more to the Red – Green – Refactor cycle. For clarity, that’s usually used during TDD where you go Red – a failing test. Then you do whatever it takes to make the test pass – Green. Finally, you review the code you’re left with and, with the security of the unit test around you, you refactor it to the simplest possible design/structure for its current feature set.

Where I say Fix it twice, I’m usually referring to the rare case of a bug-fix or similar. With TDD you don’t get bugs so much, because your tests kind of prevent them. You do, however, get some surprises at later points. Tests are never perfect, and some issues only rear their heads when you add more features. So at some point, you need to apply a fix to existing code to make it do what it ought to have done all along.

We still use TDD for bug fixes, you can’t be sure you’ve fixed it until you’ve found a test that needs it to be fixed in order to pass. The problem can sometimes be that you don’t know the scope severity of the fix until you’ve had a crack at solving the problem, after which you can be left with code that’s not at your usual standard.

Refactoring alone may not make the root cause of your problem better. But, once you’ve fixed the code once, to get something that now works again, you’re usually in a better position to judge what you should have done all along. This is the Fix it twice of which me and the team speak.

Maybe you revisit a root cause of the bug. Maybe you generalise the code to avoid edge cases that need resolving individually. Maybe you add more tests to prevent a naive change causing a surprising regression. The philosophy of Fix it twice is the solution to the classic case of L’espirit de l’escalier that we encounter in life sometimes, where you wish you’d said something different in a conversation that’s now over. In software you’ve always got the chance for a better do over.

Posted in Uncategorized

So Let’s Test The POJOs

So the code coverage is mediocre and there are loads of getters and setters and equals methods out there sitting untested. Let’s impress our lords and masters by running some tests against them. Assert that when you set the value, the getter gets the new value and so on.

Shall we?

Is POJO coverage important?

Here are some thoughts.

Code Coverage is less important than Code-that’s-used

Writing placebo tests is just busy work. Asking why the methods aren’t already used by tests that matter for your business scenarios will reveal that either:

  • Not all scenarios are covered – so do that
  • Well, this method isn’t really needed – so delete it

And in a few cases, that’ll still leave some stuff behind that’s kind of needed structurally, but isn’t that important right now.

Absence of code coverage creates a fog

While working on Spectrum, we maintained a rule of 100% code coverage (until some new coverage tool came along and lost us 0.01% for some reason). The idea was that you have a rigid feedback loop if your change does anything to a 100% coverage statistic. We reckoned you could test every scenario, so there shouldn’t be a line of code out of place.

When you have lower code coverage, it’s hard to tell from the statistics whether this means that important or unimportant stuff is not getting tested. When it’s higher – 95%+ – you get the impression that every uncovered line represents a testing misfire and a potential bug.

So find an easy way to cover your minor lines. Use an equals/hashcode test helper to wash away doubts about those methods. EqualsVerifier is one such option.

Testing things in context leads to high coverage

We’re not really here to make code coverage. We’re here to make code that’s guaranteed to do what the business process is expected to do. This means you need high scenario coverage. So maybe your getter/setter combination is only used when you serialize the object, so maybe you need to serialize some, perhaps in the context of the flow which need serialization to work.

Once you focus on the usage in real life of your classes, you can use code coverage to feedback that the intended code paths are tested and that all your code is actually needed.

Caveat Tester!

Despite all of this, code coverage is only a surrogate statistic. Alone it doesn’t prove anything. If it’s low it proves SOMETHING, but if it’s high, it only provides a number in response to what should be a genuine heartfelt attempt to try out the behaviour of your software and show it working as intended.

Posted in Java, tdd

How Mocks Ruin Everything

Despite being an ardent advocate of TDD, mocking and Mockito (which is totally awesome), I’m going to argue against mocks.

It had to happen. I’ve found really clear situations where mocks have made my work harder, and it’s got to stop!

Why?

When used incorrectly Mocks do not test your code

Yep. Mocks can be the fool’s gold of TDD.

What do good tests achieve?

Back to basics then. What’s it all about?

  • Really exercise the components under test
  • Specify what the component is expected to do
  • Speed up development and fault resolution
  • Prevent waste – because developing without the right tests is probably slower and less efficient overall

Why do we mock?

  • Create boundaries in tests
  • Speed of execution
  • Simulate the hard-to-simulate, especially those pesky boundary cases and errors

What is a mock?

For the purposes of this discussion, let’s bundle in all non-real test objects:

  • Pretend test data
  • Stubs – which receive calls and do nothing much
  • Mocks – which can employ pre-programmed behaviours and responses to inputs, despite being just test objects
  • Fakes – implementations that simulate the real thing, but are made for testing – e.g. an in-memory repository

How can mocks go wrong?

Your mock starts to work against you if:

  • It makes your test implementation focused, rather than behaviour focused
  • Mocking requires some obscure practices
  • Mocking becomes a boilerplate ritual performed before testing

Implementation focused?

You’ll know if you’re doing this. If you do it before writing the code, it feels like trying to write the code backwards, via tests that predict the lines of code you need. If you do it after the fact, you’ll find yourself trying to contrive situations to exercise individual lines or statements, or you’ll find yourself pasting production code into your tests to make a mock behave the way you want.

Obscure

Mocks have the power to bypass the real code, so we may find ourselves using the mocks to generate an alternative reality where things kind of work because the mocks happen to behave in a way which gives a sort of an answer. This seems to happen when the thing you’re mocking is quite complex.

Ritual

If all tests begin with the same pasted code, then there’s something odd about your test refactoring and mocking.

So What’s The Solution?

  • You ARE allowed to use real objects in tests
    • Mock at heavy interface boundaries
    • Refactor your code so more of your real algorithms and services can be used at test time
  • You SHOULD test with real-life data
    • Your fancy date algorithm may work fine with 4th July 2006, but if that’s not the sort of date your users will use, come up with more real life ones
    • Make the test data describe real-world use cases so people can better understand the intent of the module
  • Add mocks when you must
    • Add spies to real objects to simulate hard-to-reach places as a preference to total mocking
  • Consider using fakes
    • Complex behaviour of simple things tests best if you can produce a fake implementation – this might even allow for changes in how the code under tests uses the services it depends on
  • Test-first means that you delay thinking about the implementation
  • Test behaviour, not lines of code

In Conclusion

Test what you mean to test. Write tests first about important behaviour. Try to forget any knowledge about the implementation in the tests. Within reason, be proud to weave together a handful of collaborating objects to spin up a unit test.

Posted in Java, tdd

Mockito Argument Matching in Java 8

When using mocks, we often want to check the inputs to a function that was called on the mock. It’s probably a subject for another post whether you should rely on doing this, or whether you should make more code that just returns something, rather than calls something, but let’s agree that you will, at some point, want to check how a mock’s function was called.

With Mockito as our mocking framework of choice, here’s the hard way:

// given some test execution has happened

// construct a fully-blown replica of what you think will have
// been passed into your function under test
SomeObject expectedInput = new SomeObject( ... );

verify(myMock).myMethod(eq(expectedInput));

Why is that hard?

Well, you need to predict a perfect replica of the input, which requires you to bind your test to the exact implementation. In some cases, certain fields, which are less interesting to the behaviour your testing, have to be specified so that the equals comparison works. Worse still, some fields that get arbitrary values at runtime have to have specific values pushed into them to make the test replicatable – timestamps, for instance.

In short, the above doesn’t always work well, so we often resource to the use of argument captors.

// given some test execution has happened

// Find out how the method got called
ArgumentCaptor<SomeObject> captor = ArgumentCaptor.for(SomeObject.class);
verify(myMock).myMethod(captor.capture());

// read the thing you're interested in from the captor
assertThat(captor.getValue().getInterestingProperty(), is("expected"));

Put mildly, this sucks! You have three activities to do with the captor, all of which are to enable you to check one fact. You have to construct one, use one to capture the value, and then read the value from the captor after the Mockito verify method is called. The test also reads oddly. It’s become:

  • When the test was executed
  • Given this captor
  • Check the method was called and capture
  • And check the captured value was expected

Since Mockito 2.1, used with Java 8, there has been a neater technique. You can use the argThat comparator with a verify call to inline your check in the verify method:

// given some test execution has happened

// Find out how the method got called
verify(myMock).myMethod(argThat(
        someObject -> someObject.getInterestingProperty().equals("expected")
    ));

This, I think, makes for a more straightforward test. Verify that the method was called with an argument that matches a certain filter.

The reason this is possible is that Mockito moved away from using Hamcrest internally, replacing it with its own strongly-typed ArgumentMatcher interface, which is essentially a functional interface. This means you can replace it with a lambda.

An extra bonus of this technique is that you can also use argThat within the when or given constructs in Mockito. This means you can neatly specify how your mock will behave based on nuances of the input:

when(myMock.myMethod(argThat(
        someObject -> someObject.getInterestingProperty().equals("expected")
    ))).thenReturn(42);

For more information on this please see the Mockito JavaDoc.

Posted in Java, tdd

Automated Testing is a Broad Church

I’ve had the pleasure to be working on an automated testing framework recently. This seems to solve a problem that we’ve been having with Cucumber. I will write a more detailed piece on this in the near future, but here’s the elevator pitch.

JUnit and Mockito tests tend to be too technical so we can’t use them as an acceptance test framework directly.

Cucumber is the go-to technology for BDD/ATDD but implementing it can be cumbersome (cucumbersome perhaps? or an encucumbrance? – who knows!?).

In short, if you want the Given/When/Then and documentation friendly features of Cucumber you have to pay the price of using Cucumber.

What’s Wrong with Cucumber?

Nothing necessarily. Once you’re dealing with lots of similarly documented specs, especially if they’re simple, Cucumber can be a real boon.

However, for Cucumber to work you have to phrase your Gherkin right, implying but not implementing the test script. Then you need to write your glue code just write, and then you need one or two tiers of test execution code. This means you may have to cross 3 or 4 layers of software/script in various languages/styles to get to the code which reaches out to the system under test.

This is usually good, until you need to remember the outcome of one step in order to use it to verify a later one. At this point, you’ve no way of clearly putting that into the software layers. It ends up somewhere in the Orchestration or World code. It’s hinted at by the Gherkin and glue-code. It’s obscure and it’s caused entirely by the Achilles heel of Cucumber.

Cucumber’s Achilles Heel

To connect your spec with test execution code you have two degrees of separation. Plaintext Gherkin, used at runtime, plus whichever glue-code is kicking about. For tricky cases, this often obscures the intent of the spec/test implementation.

What can we do?

How about we write tests in Java but use the BDD syntax to structure them and report on them? With this in mind there are a few frameworks that offer just this:

Oleaster and Ginkgo4j both try to be equivalents of Jasmine and Mocha. Please see my post for more on these and other BDD frameworks in Java.

I have been working on Spectrum with its founder, Greg Haskins. In the current live release, there’s support for Jasmine/Mocha/RSpec like tests. In the next release (soon) there will be decent support for Gherkin syntax, and some rather neat ways of weaving in your favourite JUnit test frameworks (Spring, Mockito etc) via JUnit Rules.

Have a look.

The Right Test for the Right Job

Success comes not from finding the right tool, but from using the right tool for the job at hand. Where Cucumber succeeds, you should use it. It’s very helpful. Once it gets hard, change tactics.

Posted in Java, tdd