How to use the free monad with Future [M [_]]

Question

I used a simple language for the ETL process using the free monad. When using List as input and output data to retrieve and store data, everything works fine. However, I use asynchronous libraries and work with Future[List]

general imports and definitions

 import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global import cats.free.Free import cats.free.Free._ sealed trait Ops[A] type OpsF[A] = Free[Ops, A] 

works with List

 case class Fetch(offset: Int, amount: Int) extends Ops[List[Record]] case class Store(recs: List[Record]) extends Ops[List[Response]] def fetch(offset: Int, amount: Int): OpsF[List[Record]] = liftF[Ops, List[Record]](Fetch(offset, amount)) def store(recs: List[Record]): OpsF[List[Response]] = liftF[Ops, List[Response]](Store(recs)) def simpleEtl(offset: Int, amount: Int): Free[Ops, List[Response]] = fetch(offset, amount).flatMap(r => store(r)) 

does not work with Future[List]

 case class Fetch(offset: Int, amount: Int) extends Ops[Future[List[Record]]] case class Store(recs: List[Record]) extends Ops[Future[List[Response]]] def fetch(offset: Int, amount: Int): OpsF[Future[List[Record]]] = liftF[Ops, Future[List[Record]]](Fetch(offset, amount)) def store(recs: List[Record]): OpsF[Future[List[Response]]] = liftF[Ops, Future[List[Response]]](Store(recs)) // explicit types in case I am misunderstanding more than I think def simpleEtl(offset: Int, amount: Int): Free[Ops, Future[List[Response]]] = fetch(offset, amount).flatMap { rf: Future[List[Record]] => val getResponses: OpsF[Future[List[Response]]] = rf map { r: List[Record] => store(r) } getResponses } 

as expected, the type returned with flatMap / map is incorrect - I don't get OpsF[Future] , but Future[OpsF]

 Error:(34, 60) type mismatch; found : scala.concurrent.Future[OpsF[scala.concurrent.Future[List[Response]]]] (which expands to) scala.concurrent.Future[cats.free.Free[Ops,scala.concurrent.Future[List[String]]]] required: OpsF[scala.concurrent.Future[List[Response]]] (which expands to) cats.free.Free[Ops,scala.concurrent.Future[List[String]]] val getResponses: OpsF[Future[List[Response]]] = rf map { r: List[Record] => 

My current workaround is to store accept Future[List[Record]] and give the interpreter a map over Future , but it seems awkward.

The problem does not apply to List - for example. Option will also be useful.

Am I doing it wrong? Is there some kind of monad transformer for this?

Answer 1

The Ops abstract data type defines an algebra for retrieving and storing multiple Record s. He describes two operations, but this is also the only thing algebra should do. How the operations are actually performed doesn't matter at all for Fetch and the Store , the only useful thing you expect is List[Record] and List[Response] , respectively.

By making the expected result type Fetch and Store a Future[List[Record]]] , you will limit the interpretation of this algebra. You may not want to connect asynchronously to a web service or database in your tests and just want to test using Map[Int, Result] or Vector[Result] , but now you need to return Future , which makes the tests more than they can be .

But saying that you do not need ETL[Future[List[Record]]] does not solve your question: you use asynchronous libraries, and you probably want to return some Future .

Starting from your first implementation:

 import scala.concurrent.Future import scala.concurrent.ExecutionContext.Implicits.global import cats.implicits._ import cats.free.Free type Record = String type Response = String sealed trait EtlOp[T] case class Fetch(offset: Int, amount: Int) extends EtlOp[List[Record]] case class Store(recs: List[Record]) extends EtlOp[List[Response]] type ETL[A] = Free[EtlOp, A] def fetch(offset: Int, amount: Int): ETL[List[Record]] = Free.liftF(Fetch(offset, amount)) def store(recs: List[Record]): ETL[List[Response]] = Free.liftF(Store(recs)) def fetchStore(offset: Int, amount: Int): ETL[List[Response]] = fetch(offset, amount).flatMap(store) 

But now we still don't have Future ? That the work of our translator:

 import cats.~> val interpretFutureDumb: EtlOp ~> Future = new (EtlOp ~> Future) { def apply[A](op: EtlOp[A]): Future[A] = op match { case Store(records) => Future.successful(records.map(rec => s"Resp($rec)")) // store in DB, send to webservice, ... case Fetch(offset, amount) => Future.successful(List.fill(amount)(offset.toString)) // get from DB, from webservice, ... } } 

With this interpreter (where, of course, you replace Future.successful(...) with something more useful), we can get our Future[List[Response]] :

 val responses: Future[List[Response]] = fetchStore(1, 5).foldMap(interpretFutureDumb) val records: Future[List[Record]] = fetch(2, 4).foldMap(interpretFutureDumb) responses.foreach(println) // List(Resp(1), Resp(1), Resp(1), Resp(1), Resp(1)) records.foreach(println) // List(2, 2, 2, 2) 

But we can still create another interpreter that does not return Future :

 import scala.collection.mutable.ListBuffer import cats.Id val interpretSync: EtlOp ~> Id = new (EtlOp ~> Id) { val records: ListBuffer[Record] = ListBuffer() def apply[A](op: EtlOp[A]): Id[A] = op match { case Store(recs) => records ++= recs records.toList case Fetch(offset, amount) => records.drop(offset).take(amount).toList } } val etlResponse: ETL[List[Response]] = for { _ <- store(List("a", "b", "c", "d")) records <- fetch(1, 2) resp <- store(records) } yield resp val responses2: List[Response] = etlResponse.foldMap(interpretSync) // List(a, b, c, d, b, c)