How to get NUnit to run F # tests not exported by a module

Question

I wrote a big module in F # that has a trivial interface. The module contains about 1000 lines of code, 50 unit tests and exports only one easily understood function.

Naturally, what to do next is to write a tiny fsi file. This has many advantages, including preventing pollution of the namespace, which provides an obvious place for documentation, making sure that if someone decides to reuse the internal device, they will have an incentive to clearly define them, and, of course, many others. I am sure that I am preaching a choir here, but still I think it’s worth explaining why I find the fsi file useful.

Now the problem. NUnit will no longer run unit tests, explicitly claiming that they are not public. Well, that would be because they are in no way part of the interface. I don’t really want to add them to the interface, despite this, because it will mean updating it every time I add another test, and also that it will inflate the fsi file by an order of magnitude.

I assume that a trivial workaround is to move the code to another location, import it into a tiny .fs file and just forward one function. With some luck, everyone will agree that this is simply disgusting. Is there a better way please?

Edit: many thanks to everyone who answered. I supported both answers. I would like to share the generosity, however, since this is not possible, I (somewhat arbitrarily) agree with Thomas's answer.

Answer 1

If you add an fsi file to indicate the visibility of modules and functions in your source, then you will need to include declarations of all functions that should be publicly available. This means that if NUnit requires the tests to be publicly available, you need to include them in the fsi file.

However, there is another way to specify visibility in F # - instead of using the fsi file, you can simply add the appropriate visibility modifiers to your ads. Thus, you can hide all implementation details and export only the main function and tests:

 namespace MyLibrary open NUnit.Framework // Implementation details can be in this module // (which will not be visible outside of the library) module private Internal = let foo n = n * 2 let bar n = n + 1 // A public module can contain the public API (and use internal implementation) module public MyModule = open Internal let doWork n = foo (bar n) // To make the tests visible to NUnit, these can be placed in a public module // (but they can still access all functions from 'Internal') module public Tests = open MyModule [<Test>] let ``does work for n = 1``() = Assert.Equals(doWork 1, 4) 

Compared to using fsi files, this has the disadvantage that you do not have a separate file that perfectly describes only the important parts of your API. However, you get what you need - hide the implementation details and show only one function and tests.

Answer 2

An approach

You can resort to using reflection to invoke your private test methods: you will have one public NUnit test method that will iterate over all private methods in the assembly that call those that have the Test attribute. The big side of this approach is that you can only see one failed test method at a time (but maybe you could learn something creative, like using parameterized tests to fix this).

Example

Program.fsi

 namespace MyNs module Program = val visibleMethod: int -> int 

Program.fs

 namespace MyNs open NUnit.Framework module Program = let implMethod1 xy = x + y [<Test>] let testImpleMethod1 () = Assert.AreEqual(implMethod1 1 1, 2) let implMethod2 xyz = x + y + z [<Test>] let testImpleMethod2 () = Assert.AreEqual(implMethod2 1 1 1, 3) let implMethod3 xyzr = x + y + z + r [<Test>] let testImpleMethod3 () = Assert.AreEqual(implMethod3 1 1 1 1, -1) let implMethod4 xyzrs = x + y + z + r + s [<Test>] let testImpleMethod4 () = Assert.AreEqual(implMethod4 1 1 1 1 1, 5) let visibleMethod x = implMethod1 xx + implMethod2 xxx + implMethod3 xxxx 

TestProxy.fs (implementation of our "approach")

 module TestProxy open NUnit.Framework [<Test>] let run () = ///we only want static (ie let bound functions of a module), ///non-public methods (exclude any public methods, including this method, ///since those will not be skipped by nunit) let bindingFlags = System.Reflection.BindingFlags.Static ||| System.Reflection.BindingFlags.NonPublic ///returns true if the given obj is of type TestAttribute, the attribute used for marking nunit test methods let isTestAttr (attr:obj) = match attr with | :? NUnit.Framework.TestAttribute -> true | _ -> false let assm = System.Reflection.Assembly.GetExecutingAssembly() let tys = assm.GetTypes() let mutable count = 0 for ty in tys do let methods = ty.GetMethods(bindingFlags) for mi in methods do let attrs = mi.GetCustomAttributes(false) if attrs |> Array.exists isTestAttr then //using stdout w/ flush instead of printf to ensure messages printed to screen in sequence stdout.Write(sprintf "running test `%s`..." mi.Name) stdout.Flush() mi.Invoke(null,null) |> ignore stdout.WriteLine("passed") count <- count + 1 stdout.WriteLine(sprintf "All %i tests passed." count) 

Output Result (Using TestDriven.NET)

Note that we never get testImplMethod4 as it fails in testImpleMethod3:

 running test `testImpleMethod1`...passed running test `testImpleMethod2`...passed running test `testImpleMethod3`...Test 'TestProxy.run' failed: System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation. ----> NUnit.Framework.AssertionException : Expected: 4 But was: -1 at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner) at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeType typeOwner) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) C:\Users\Stephen\Documents\Visual Studio 2010\Projects\FsOverflow\FsOverflow\TestProxy.fs(29,0): at TestProxy.run() --AssertionException C:\Users\Stephen\Documents\Visual Studio 2010\Projects\FsOverflow\FsOverflow\Program.fs(25,0): at MyNs.Program.testImpleMethod3() 0 passed, 1 failed, 4 skipped (see 'Task List'), took 0.41 seconds (NUnit 2.5.10).