docs: style and formatting cleanup

* Amends documentation to use more neutral, impersonal tone.

* Removes duplicated parts of the documentation.

* Fixes display of external images by using raw insertion.

* Adds CHANGES and LICENSE sections to documentation.

* Renames task example documentation sections.

* Simplifies README and uses it as long package description.

Signed-off-by: Tibor Simko <[email protected]>

Author: tiborsimko

AUTHORS.rst

@@ -6,11 +6,10 @@ developed and maintained by the Invenio collaboration.  You can
 contact us at
 `[email protected] <mailto:[email protected]>`_.
 
-Contributors
-^^^^^^^^^^^^
+Contributors:
 
-- Roman Chyla <[email protected]>
-- Raja Sripada <[email protected]>
-- Jiri Kuncar <[email protected]>
-- Tibor Simko <[email protected]>
-- Brett Anthoine <[email protected]>
+* Roman Chyla <[email protected]>
+* Raja Sripada <[email protected]>
+* Jiri Kuncar <[email protected]>
+* Tibor Simko <[email protected]>
+* Brett Anthoine <[email protected]>

CHANGES.rst

@@ -0,0 +1,10 @@
+Changes
+=======
+
+Version 1.0 (released 2011-07-07):
+
+- Initial public release.
+- Includes the code created by Roman Chyla, the core of the workflow
+  engine together with some basic patterns.
+- Raja Sripada <rsripada at cern ch> contributed improvements to the
+  pickle&restart mechanism.

CHANGES.txt

@@ -6,7 +6,6 @@
 # more details.
 
 include LICENSE
-include CHANGES.txt
 include docs/*.rst docs/*.py docs/Makefile
 include *.rst
 include tests/*.ini

MANIFEST.in

@@ -11,27 +11,13 @@
 .. image:: https://pypip.in/d/workflow/badge.png
    :target: https://pypi.python.org/pypi/workflow/
 
-Introduction
-============
-
-I was looking for a workflow engine some time ago, and there weren't many for
-Python. Google will show up quite a few, but:
-
-* They are Plone or Django or Project-X specific
-* I found them too complicated (non-intuitive)
-* or abandoned
-* or any combination of the above...
-
-So I created my own workflow engine (alas) - but it sort of works... quite well,
-so I haven't looked for a better alternative.
-
-Details
-=======
+About
+=====
 
-Workflow engine is a Finite State Machine with memory.
-It is used to execute set of methods in a specified order.
+Workflow is a Finite State Machine with memory.  It is used to execute
+set of methods in a specified order.
 
-Here is a simple example of a configuration:
+Here is a simple example of a workflow configuration:
 
 .. code-block:: text
 
@@ -51,298 +37,28 @@ Here is a simple example of a configuration:
       translate_token,
     ]
 
-You can probably guess what the processing pipeline does with tokens - the
-whole task is made of four steps and the whole configuration is just stored
-as a Python list. Every task is implemented as a function that takes two objects:
-
-* currently processed object
-* workflow engine instance
-
-Example:
-
-.. code-block:: python
-
-    def next_token(obj, eng):
-        eng.ContinueNextToken()
-
-There are NO explicit states, conditions, transitions - the job of the
-engine is simply to run the tasks one after another. It is the
-responsibility of the task to tell the engine what is going to happen
-next; whether to continue, stop, jump back, jump forward and few other
-options.
-
-This is actually a *feature*, I knew that there will be a lot of possible
-exceptions and transition states to implement for NLP processing and I also
-wanted to make the workflow engine simple and fast -- but it has disadvantages,
-you can make more errors and workflow engine will not warn you.
-
-The workflow module comes with many patterns that can be directly used in the
-definition of the pipeline, such as IF, IF_NOT, PARALLEL_SPLIT and others.
-
-The individual tasks then can influence the whole pipeline, available
-''commands'' are:
-
-.. code-block:: text
-
-    eng.stopProcessing #stops the current workflow
-    eng.haltProcessing: #halts the workflow (can be used for nested wf engines)
-    eng.continueNextToken # can be called many levels deep, jumps up to next token
-    eng.jumpTokenForward # will skip the next object and continue with the next one
-    eng.jumpTokenBack # will return back, start processing again
-    eng.jumpCallForward #in one loop [call, call...] jumps x steps forward
-    eng.jumpCallBack #in one loop [call, call...] jumps x steps forward
-    eng.breakFromThisLoop #break from this loop, but do not stop processing
-
-Consider this example of a task:
-
-.. code-block:: python
-
-    def if_else(call):
-        def inner_call(obj, eng):
-           if call(obj, eng):     #if True, continue processing
-              eng.jumpForward(1)
-           else:                  #else, skip the next step
-              eng.jumpForward(2)
-        return inner_call
-
-We can then write *workflow definition* like:
-
-.. code-block:: text
-
-    if_else(stage_submission),
-    [
-        [if_else(fulltext_available),  #this will be run only when fulltext is uploaded during form submission
-         [extract_metadata, populate_empty_fields],
-         [#do nothing ]],
-        [if_else(check_for_duplicates),
-         [stop_processing],
-         [synchronize_fields, replace_values]],
-        check_mandatory_fields,]
-        ],
-        [
-        check_mandatory_fields,        # this will run only for 'review' stage
-        check_preferred_values,
-        save_record
-    ]
-
-Tasks
------
-
-Tasks are simple python functions, we can enforce rules (not done yet!) in
-a pythonic way using pydoc conventions, consider this:
-
-.. code-block:: python
-
-    def check_duplicate(obj, eng):
-       """
-       This task checks if the uploaded fulltext is a duplicate
-            @type obj: InspireGeneralForm
-            @precondition: obj.paths[]
-                    list, list of paths to uploaded files
-            @postcondition: obj.fulltext[]
-                    list containing txt for the extracted document
-                            obj.duplicateids[]
-                    list of inspire ids records that contain the duplicate of this document
-            @raise: stopProcessing on error
-            @return: True if duplicate found
-
-       """
-       ...
-
-So using the python docs, we can instruct workflow engine what types of
-arguments are acceptable, what is the expected outcome and what happens
-after the task finished.  And let's say, there will be a testing framework
-which will run the workflow pipeline with fake arguments and will test all
-sorts of conditions. So, the configuration is not cluttered with states
-and transitions that are possible, developers can focus on implementation
-of the individual tasks, and site admins should have a good understanding
-what the task is supposed to do -- the description of the task will be
-displayed through the web GUI.
-
-Some examples
--------------
-
-Here are some examples of workflow patterns (images are from
-`http://www.yawlfoundation.org`_) and their implementation in
-Python. This gives you an idea that workflow engine remains very
-simple and by supplying special functions, we can implement different
-patterns.
-
-
-.. image:: http://www.yawlfoundation.org/images/patterns/basic_ps.jpg
-
-This pattern is called Parallel split (as tasks B,C,D are all started in
-parallel after task A). It could be implemented like this:
-
-.. code-block:: python
-
-    def PARALLEL_SPLIT(*args):
-        """
-        Tasks A,B,C,D... are all started in parallel
-        @attention: tasks A,B,C,D... are not addressable, you can't
-            you can't use jumping to them (they are invisible to
-            the workflow engine). Though you can jump inside the
-            branches
-        @attention: tasks B,C,D... will be running on their own
-            once you have started them, and we are not waiting for
-            them to finish. Workflow will continue executing other
-            tasks while B,C,D... might be still running.
-        @attention: a new engine is spawned for each branch or code,
-            all operations works as expected, but mind that the branches
-            know about themselves, they don't see other tasks outside.
-            They are passed the object, but not the old workflow
-            engine object
-        @postcondition: eng object will contain lock (to be used
-            by threads)
-        """
-
-        def _parallel_split(obj, eng, calls):
-            lock=thread.allocate_lock()
-            i = 0
-            eng.setVar('lock', lock)
-            for func in calls:
-                new_eng = duplicate_engine_instance(eng)
-                new_eng.setWorkflow([lambda o,e: e.setVar('lock', lock), func])
-                thread.start_new_thread(new_eng.process, ([obj], ))
-                #new_eng.process([obj])
-        return lambda o, e: _parallel_split(o, e, args)
-
-
-And is used like this:
-
-.. code-block:: python
-
-    from workflow.patterns import PARALLEL_SPLIT
-    from my_module_x import task_a,task_b,task_c,task_d
-
-    [
-     task_a,
-     PARALLEL_SPLIT(task_b,task_c,task_d)
-    ]
-
-Arbitrary cycle(s)
-------------------
-
-.. image:: http://www.yawlfoundation.org/images/patterns/struc_arb.jpg
-
-This is just for your amusement (and to see how complicated it looks in the
-configuration).
-
-
-.. code-block:: text
-
-    #!python
-    [
-      ...        #here some conditional start
-      task_a,
-      task_b,
-      task_c,
-      if_else(some_test),
-        [task_d, [if_else(some_test),
-                    lambda obj, eng: eng.jumpCallBack(-6),  #jump back to task_a
-                    some_other_task,
-                  ]]
-        [some_other_task],
-      ...
-    ]
-
-.. admonition:: TODO
-
-    Jumping back and forward is obviously dangerous and tedious
-    (depending on the actual configuration), we need a better solution.
-
-Synchronization
----------------
-
-.. image:: http://www.yawlfoundation.org/images/patterns/basic_synch.jpg
-
-After the execution of task B, task C, and task D, task E can be executed
-(I will present the threaded version, as the sequential version would be dead simple).
-
-.. code-block:: python
+Documentation
+=============
 
-    def SYNCHRONIZE(*args, **kwargs):
-        """
-        After the execution of task B, task C, and task D, task E can be executed.
-        @var *args: args can be a mix of callables and list of callables
-                    the simplest situation comes when you pass a list of callables
-                    they will be simply executed in parallel.
-                       But if you pass a list of callables (branch of callables)
-                    which is potentionally a new workflow, we will first create a
-                    workflow engine with the workflows, and execute the branch in it
-        @attention: you should never jump out of the synchronized branches
-        """
-        timeout = MAX_TIMEOUT
-        if 'timeout' in kwargs:
-            timeout = kwargs['timeout']
+Documentation is readable at http://workflow.readthedocs.org or can be built using Sphinx: ::
 
-        if len(args) < 2:
-            raise Exception('You must pass at least two callables')
+    pip install Sphinx
+    python setup.py build_sphinx
 
-        def _synchronize(obj, eng):
-            queue = MyTimeoutQueue()
-            #spawn a pool of threads, and pass them queue instance
-            for i in range(len(args)-1):
-                t = MySpecialThread(queue)
-                t.setDaemon(True)
-                t.start()
-
-            for func in args[0:-1]:
-                if isinstance(func, list) or isinstance(func, tuple):
-                    new_eng = duplicate_engine_instance(eng)
-                    new_eng.setWorkflow(func)
-                    queue.put(lambda: new_eng.process([obj]))
-                else:
-                    queue.put(lambda: func(obj, eng))
-
-            #wait on the queue until everything has been processed
-            queue.join_with_timeout(timeout)
-
-            #run the last func
-            args[-1](obj, eng)
-        _synchronize.__name__ = 'SYNCHRONIZE'
-        return _synchronize
-
-
-Configuration (i.e. what would admins write):
-
-.. code-block:: text
+Installation
+============
 
-    from workflow.patterns import SYNCHRONIZE
-    from my_module_x import task_a,task_b,task_c,task_d
+Workflow is on PyPI so all you need is: ::
 
-    [
-     synchronize(task_b,task_c,task_d, task_a)
-    ]
+    pip install workflow
 
 Testing
 =======
 
-Running the test suite is as simple as:
-
-.. code-block:: console
-
-   $ python setup.py test
-
-on Windows, you may want to do:
-
-.. code-block:: console
-
-   $ python setup.py test --pytest-args=tests
-
-or, to also show code coverage:
-
-.. code-block:: console
-
-   $ ./run-tests.sh
-
-TODO
-====
-
-.. admonition:: TODO
+Running the test suite is as simple as: ::
 
-    There already exist a web-based GUI for construction of the workflow, publish it!
+    python setup.py test
 
-    Fix the bin/run-workflow.py script for executing the workflows.
+or, to also show code coverage: ::
 
-    Explain how the workflows can be saved and organized, embedded.
+    ./run-tests.sh