Workflow system

Question

I just started working in a new communications company, and we are considering a workflow / intranet system for managing tasks and processes.

Basically, we receive data files from clients, which are then processed through our systems.

• Receive data file (FTP, email, etc.)
• Process data file (a common script with matching data to a custom file or ETL package). Adds address values
• Create printstream (send the processed data file to postscript / PDF layout engine) or create email output
• Sending output to the production floor (copy to the printer input stream, mail machines)
• Process other streams (for example, send e-mail / faxes, upload to electronic archive).
• Upgrading internal systems (e.g. stock, billing)

We also have many other internal business processes (for example, processing damaged output, processing dead / returned mail).

I am trying to separate all the elements. Some of them will be off the shelf (for example, composition for printing, sending / managing e-mail, CRM). Some of them will be built in the house (for example, a damaged processing result).

But, I am looking for something to tie it all together, and put the business process processes. scheduling work tasks, simultaneously performing data processing tasks and error management. Many of them will have human steps. Also include monitoring and reporting on SLA and business management.

One of the key requirements in the near future is the automatic receipt and processing of files (i.e. viewing and matching the directory with the client / application).

I am fond of being easy to manage and maintain (for example, adding new steps to a workflow or conditional logic or something else).

I understand that this is a lot of work, and at the moment we focus on each individual component and apply manual processes until we get a system to control it. We don’t want to create a giant system to order in order to connect all this, but rather look at the purchase of some kind of workflow or integration system.

Any suggestions? I took a look at Biztalk, but I'm not sure if it overflows or is not suitable for internal systems. Another product that I have been exposed to is Sagent Automation, but it looks a bit quiet.

- EDIT -

Forgetting to mention, our existing skillset is mostly Microsoft. Therefore, anything in Microsoft / .Net technologies would be preferable. But if there is a fantastic product, we don’t mind raising the level of service

Answer 1

Give up Apache Active MQ . It implements the Java Message Service 1.1 specification, layers in the servlet API, and has many features that should meet your requirements. You can also overlay Camel , which adds a rich implementation of many enterprise integration schemes .

Typically, JMS messages are stored in a transaction database, which can be configured to provide you with an extremely high degree of fault tolerance (for example, RAID arrays of master-backup databases, several copies of transaction log files). In addition to the database, several load-balanced application servers running Active MQ can operate to provide scalability and high availability. I think you will find that you can write your components in a very loose way if you use Active MQ as a common message bus.

In JMS, when a message is queued by a consumer, the consumption process must later acknowledge that the message was successfully processed. If the confirmation does not arrive on time, the JMS will enliven the message so that another consumption process may try to process it. This means that you can run multiple copies of your application to increase reliability and fault tolerance.

Take a look at the O'Reilly Java Message Service, 2nd Edition , which just came out this week.

Another way is to look in BPEL (Business Process Execution).

Edit: I'm not very familiar with Microsoft suggestions, but MSMQ seems to be equivalent to JMS,

You should be able to use ActiveMQ in a Microsoft environment. They claim to support " multi-language clients " like "C # and .NET." And even if this should be problematic, since ActiveMQ has a Java servlet-based API for queuing and de-queuing messages, the outside world should be able to send HTTP requests to the ActiveMQ server. This should limit your team’s training. Good luck, that sounds like an amazing project!

Answer 2

SharePoint has a workflow engine that works very well. You can create your workflow using SharePoint Designer or Visual Studio 2008. It uses Windows Workflow, which is similar to BizTalk (if not the same engine), but without other BizTalk applications that might not be needed for your application.