RFC: Custom backends and policies for Dynamic Selection

rfcs/experimental/dynamic_selection/README.md

@@ -18,7 +18,7 @@ a timely manner, for justifying removal of the feature.
 ## Overview of Architecture and Execution Flow
 
 The key components of the Dynamic Selection API are shown below, including the
-[Free Functions](#free_functions_id) (such as `submit`, `select`, `wait`, etc), a
+[Free Functions](#free_functions_id) (such as `submit`, `wait`, etc), a
 [Policy](#policy_req_id) object (such as [fixed_resource_policy](#concrete_policies_id),
 [round_robin_policy](#concrete_policies_id), [dynamic_load_policy](#concrete_policies_id) and
 [auto_tune_policy](#concrete_policies_id)) and a [Backend](#backend_req_id) object (currently only
@@ -106,36 +106,33 @@ The type `T` satisfies *Policy* if given,
 
 - `p` an arbitrary identifier of type `T`
 - `args` an arbitrary parameter pack of types `typename… Args`
-- `s` a selection of a type `selection_t<T>` , which satisfies [Selection](#selection_req_id), and was made by `p`.
-- `f` a function object with signature `wait_t<T> fun(resource_t<T>, Args…);`
+- `f` a function object with signature `ret-type fun(resource_t<T>, Args…);`, where *ret-type* is provided by the documentation of the individual backend.
 
 | *Must* be well-formed | Description |
 | --------------------- | ----------- |
 | `p.get_resources()` | Returns a `std::vector<resource_t<T>>`. |
-| `p.select(args…)` | Returns `selection_t<T>` that satisfies [Selection](#selection_req_id). The selected resource must be within the set of resources returned by `p.get_resources()`. |
-| `p.submit(s, f, args…)` | Returns `submission_t<T>` that satisfies [Submission](#submission_req_id). The function invokes `f` with the selected resource `s` and the arguments `args...`. |
+| `p.try_select_impl(args…)` | Returns `std::shared_ptr<selection_t<T>>` that satisfies [Selection](#selection_req_id). The selected resource must be within the set of resources returned by `p.get_resources()`. |
 
-| *Optional* | Description |
+| *Optional*  | Description |
+| `p.select_impl(args…)` | Returns `selection_t<T>` that satisfies [Selection](#selection_req_id). The selected resource must be within the set of resources returned by `p.get_resources()`. |
+
+| *Optional* (at least one must be well-formed) | Description |
 | --------------------- | ----------- |
-| `p.submit_and_wait(s, f, args…)` | Returns `void`. The function invokes `f` with `s` and `args...` and waits for the `wait_t<T>` it returns to complete. |
+| `p.try_submit(f, args…)` | Returns `std::shared_ptr<submission_t<T>>` that satisfies [Submission](#submission_req_id). The function selects a resource and invokes `f` with the selected resource and `args...`. Returns null shared_ptr if no resource is available for selection |
 | `p.submit(f, args…)` | Returns `submission_t<T>` that satisfies [Submission](#submission_req_id). The function selects a resource and invokes `f` with the selected resource and `args...`. |
-| `p.submit_and_wait(f, args…)` | Returns `void`. The function selects a resource, invokes `f` and waits for the `wait_t<T>` it returns to complete. |
+| `p.submit_and_wait(f, args…)` | Returns `void`. The function selects a resource, invokes `f` and waits for the job to complete. |
 
-| Policy Traits* | Description |
+| Policy Traits | Description |
 | ------- | ----------- |
-| `policy_traits<T>::selection_type`, `selection_t<T>` | The wrapped select type returned by `T`. Must satisfy [Selection](#selection_req_id). |
 | `policy_traits<T>::resource_type`, `resource_t<T>` | The backend defined resource type that is passed to the user function object. Calling `unwrap` an object of type `selection_t<T>` returns an object of type `resource_t<T>`. |
-| `policy_traits<T>::wait_type`, `wait_type_t<T>` | The backend type that is returned by the user function object. Calling `unwrap` on an object that satisfies [Submission](#submission_req_id) returns on object of type `wait_type_t<T>`. |
 
 The default implementation of these traits depends on types defined in the Policy:
 
 ```cpp
   template <typename Policy>
   struct policy_traits
   {
-      using selection_type = typename std::decay_t<Policy>::selection_type;
       using resource_type = typename std::decay_t<Policy>::resource_type;
-      using wait_type = typename std::decay_t<Policy>::wait_type;
   };
 ```
 
@@ -180,6 +177,8 @@ The following concrete policies are provided in the experimental implementation.
 | [`round_robin_policy`](https://www.intel.com/content/www/us/en/docs/onedpl/developer-guide/2022-8/round-robin-policy.html) |
 | [`dynamic_load_policy`](https://www.intel.com/content/www/us/en/docs/onedpl/developer-guide/2022-8/dynamic-load-policy.html) |
 | [`auto_tune_policy`](https://www.intel.com/content/www/us/en/docs/onedpl/developer-guide/2022-8/auto-tune-policy.html) |
+| `token_policy`, documentation pending |
+
 
 <a id="backend_req_id"></a>
 ## Backends
@@ -197,11 +196,11 @@ The type `T` satisfies the *Backend* contract if given,
 - `b` an arbitrary identifier of type `T`
 - `args` an arbitrary parameter pack of types `typename… Args`
 - `s` is of type `S` and satisfies *Selection* and `is_same_v<resource_t<S>, resource_t<T>>` is `true`
-- `f` a function object with signature `wait_t<T> fun(resource_t<T>, Args…);`
+- `f` a function object with signature `ret-type fun(resource_t<T>, Args…)`  where the required *ret-type* is described by documentation of the individual backend.
 
 | *Must* be well-formed | Description |
 | --------------------- | ----------- |
-| `b.submit(s, f, args…)` | Returns an object that satisfies *Submission*. The function invokes `f` but does not wait for the `wait_t<T>` object returned by it. |
+| `b.submit(s, f, args…)` | Returns an object that satisfies *Submission*. The function invokes `f` but does not wait on the type returned by it for the job to complete. |
 | `b.get_submission_group()` | Returns an object that has a member function `void wait()`. Calling this wait function blocks until all previous submissions to this backend are complete. |
 | `b.get_resources()` | Returns a `std::vector<resource_t<T>>`. |
 
@@ -215,11 +214,9 @@ The type `T` satisfies the *Backend* contract if given,
 | Signature | Description |
 | --------- | ----------- |
 | `vector<typename policy_traits<P>::resource_type> get_resources(P&& p);` | Returns the resources associated with the Policy `p`. |
-| `template<typename P, typename... Args> selection_t<P> select(P&& p, Args&&... args);` | Applies the policy `p` and returns a *Selection*. |
-| `template<Selection S, typename F, typename... Args> auto submit(Selection s, F&& f, Args&&... args);` | Invokes `f` with the unwrapped resource from selection `s` and `args`. Implements any instrumentation necessary for the backend to report necessary execution information. May be implemented as `s.get_policy().submit(s, f, args…)`. |
-| `template<Policy P, typename F, typename... Args> auto submit(P&& p, F&& f, Args&&... args);` | Invokes `f` with the unwrapped resource returned by `select(p, f, args…)` and `args`. Implements any instrumentation necessary for the backend to report necessary execution information. May be implemented as `p.submit(p.select(p, f, args…), f, args…)`. |
-| `template<Selection S, typename F, typename... Args> auto submit_and_wait(Selection s, F&& f, Args&&... args);` | Invokes `f` with the unwrapped resource from selection `s` and `args`. And then waits on object returned by the `f`.  May be implemented as `wait(s.get_policy().submit(s, f, args…))`. |
-| `template<Policy P, typename F, typename... Args> auto submit_and_wait(P&& p, F&& f, Args&&... args);` |  Invokes `f` with the unwrapped resource returned by `select(p, f, args…)` and `args`.And then waits on object returned by the `f`. May be implemented as `wait(p.submit(p.select(f, args…),f,args…))`. |
+| `template<Policy P, typename F, typename... Args> auto try_submit(P&& p, F&& f, Args&&... args);` | Attempts to select a resource. If successful, invokes `f` with the unwrapped resource selected by `p.try_select_impl(args…)` and `args`. Implements any instrumentation necessary for the backend to report necessary execution information. Returns a `std::shared_ptr` of the submission type if successful, or a null `std::shared_ptr` if unable to select a resource. |
+| `template<Policy P, typename F, typename... Args> auto submit(P&& p, F&& f, Args&&... args);` | Invokes `f` with the unwrapped resource selected by `p.select_impl(args…)` and `args`. Implements any instrumentation necessary for the backend to report necessary execution information. |
+| `template<Policy P, typename F, typename... Args> auto submit_and_wait(P&& p, F&& f, Args&&... args);` |  Invokes `f` with the unwrapped resource selected by `p.select_impl(args…)` and `args` and then waits on object returned by the `f`. |
 | `template<typename P> auto get_submission_group(P&& p);` | Returns an object that has a member function `void wait()`. Calling this wait function blocks until all previous submissions to this policy are complete. |
 | `template<typename W> void unwrap(W&& w) noexcept;` | Returns `w.unwrap()` if available, otherwise returns `w`. |
 | `template<typename W> void wait(W&& w);` | Calls `w.wait()`. |
@@ -228,7 +225,7 @@ The type `T` satisfies the *Backend* contract if given,
 
 ### Deferred Initialization
 
-A call to `get_resources`, `select`, `submit` or `submit_and_wait` may initialize
+A call to `get_resources`, `submit` or `submit_and_wait` may initialize
 underlying state variables, including dynamic allocation. Initialization may throw `std::bad_alloc`.
 If `p` is a policy constructed with deferred initialization, calling these functions before 
 calling `initialize` will throw `std::logic_error`.
@@ -239,7 +236,7 @@ calling `initialize` will throw `std::logic_error`.
 Policies are informed of key events through the reporting of Execution Info.
 Most commonly, this reporting is done by a backend and is not visible to the
 end user.  However, developers that implement custom backends, or that
-manage work submission without using the `submit` or `submit_and_wait` functions,
+manage work submission without using the `try_submit`, `submit` or `submit_and_wait` functions,
 will need to report Execution Info to allow policies to work properly. There
 are currently three kinds of Execution Info that may be required by a Policy:
 
@@ -254,6 +251,10 @@ are currently three kinds of Execution Info that may be required by a Policy:
 | `report_info<S,Info>::value`, `report_info_v<S,Info>` | 'true' if the *Selection* requires the event type to be reported |
 | `report_value<S,Info,V>::value`, `report_info_v<S,Info,V>` | `true` if the *Selection* requires the event value to be reported |
 
+Backends must accept a (possibly empty) variadic list of execution-info reporting requirements in their constructors. At construction the backend should validate and, where possible, filter the available resources to remove those that cannot satisfy the requested reporting features (for example, missing device aspects or queue profiling properties). If filtering leaves no usable resources, the backend must throw a clear runtime error.
+
+Backends must also provide a template `template <typename... ReportingReqs> struct scratch_space_t` which supplies per-selection scratch storage tailored to the requested reporting requirements.
+
 Backend traits can be used to determine what events are need by the *Policy* that provided a *Selection*.
 For example, below is code a function the receives a *Selection* and uses traits to determine if the
 `task_submission_t` must be reported. If so, it is reported using the `report` free function.

rfcs/experimental/dynamic_selection/customization/README.md

@@ -0,0 +1,132 @@
+# Simplified Customization for Dynamic Selection
+
+Dynamic Selection is a Technology Preview feature 
+[documented in the oneDPL Developer Guide](https://www.intel.com/content/www/us/en/docs/onedpl/developer-guide/2022-8/dynamic-selection-api.html)
+and its current design is described by 
+[an experimental RFC](https://github.com/uxlfoundation/oneDPL/tree/main/rfcs/experimental/dynamic_selection).
+When applying Dynamic Selection to use cases, there is often the desire to add new
+backends or new selection policies. While it is possible to add new policies or backends
+by following the contract described in the current design, the process is non-trivial
+and requires some (unnecessarily) verbose code.
+
+## Proposed Customization Solutions
+
+This RFC presents two complementary proposals to simplify Dynamic Selection customization:
+
+### Backend Customization
+
+When applying Dynamic Selection, there is often the desire to add a new backend for
+different resource types. With sensible defaults, this proposal aims to simplify 
+backend writing to open up Dynamic Selection to more use cases. The approach presents a flexible 
+backend system based on a `backend_base` template class and a `default_backend` template that 
+can be used for most resource types.
+
+For detailed information about backend customization, see [Custom Backends](custom_backends.md).
+
+### Policy Customization
+
+There is also the expectation that new selection policies will be implemented. This proposal
+aims to  simplify policy writing by providing a base class that handles the common functionality.
+The approach presents a flexible policy system based on a `policy_base` template class that can
+be used for most selection strategies.
+
+For detailed information about policy customization, see [Custom Policies](custom_policies.md).
+
+### High Level Class Diagram
+
+The following diagram shows the relationships of the helpers for customization, and the entry points for the user and for the customizer of policies and backends. This diagram shows `round_robin_policy` specifically, but the same relationship exists for other policies.
+
+```mermaid
+
+classDiagram
+    direction BT
+
+    %% Backend Layer
+    class backend_base~ResourceType, Backend~ {
+        #resources_: vector~ResourceType~
+        +backend_base(ReportReqs...)
+        +backend_base(vector~ResourceType~, ReportReqs...)
+        +get_resources()
+        +get_submission_group()
+        +submit(SelectionHandle, f, args...)
+    }
+
+
+    %% Public API Layer
+    class FreeFunctions["Free Functions (dynamic_selection_traits.h)"] {
+        <<namespace>>
+        +try_submit(policy, f, args...)
+        +submit(policy, f, args...)
+        +submit_and_wait(policy, f, args...)
+        +wait(wait_object)
+        +get_resources(policy)
+        +unwrap(value)
+    }
+
+    class InternalFallbacks["internal namespace"] {
+        <<namespace>>
+        +submit_fallback(policy, f, args...)
+        +submit_and_wait_fallback(policy, f, args...)
+    }
+
+    %% Policy Layer
+    class policy_base~Policy, ResourceAdapter, Backend, ReportReqs~ {
+        <<CRTP Base>>
+        #backend_: shared_ptr~Backend~
+        +initialize()
+        +initialize(vector~resource_type~)
+        +initialize(vector~resource_type~, adapter)
+        +try_submit(f, args...) shared_ptr~WaitType~
+        +submit(f, args...)
+        +submit_and_wait(f, args...)
+        +get_resources()
+        +get_submission_group()
+    }
+
+    class round_robin_policy~ResourceType, ResourceAdapter, Backend~ {
+        #selector_: shared_ptr~selector_t~
+        +round_robin_policy()
+        +round_robin_policy(vector~ResourceType~)
+        +initialize_impl()
+        +try_select_impl(args...) shared_ptr~selection_type~
+    }
+
+    class default_backend_impl~BaseResourceType, ResourceType, ResourceAdapter~ {
+        -adapter: ResourceAdapter
+        +default_backend_impl(ReportReqs...)
+        +default_backend_impl(vector~ResourceType~, adapter, ReportReqs...)
+        +submit(s, f, args...) //to override default
+    }
+
+    class default_backend~ResourceType, ResourceAdapter~ {
+        +default_backend(ReportReqs...)
+        +default_backend(vector~ResourceType~, adapter, ReportReqs...)
+    }
+
+    %% Relationships - Inheritance
+    round_robin_policy --|> policy_base : inherits (CRTP)
+    default_backend_impl --|> backend_base : inherits
+    default_backend --|> default_backend_impl : inherits
+
+    %% Relationships - Composition
+    policy_base *-- default_backend : backend_
+
+
+    %% Relationships - Dependencies
+    FreeFunctions ..> policy_base : calls methods
+    FreeFunctions ..> InternalFallbacks : delegates
+    InternalFallbacks ..> FreeFunctions : calls (recursive)
+    policy_base ..> InternalFallbacks : delegates submit/submit_and_wait
+    policy_base ..> default_backend : submit(selection, f, args)
+    round_robin_policy ..> default_backend : default template param
+
+
+
+    %% Notes
+    note for FreeFunctions "Entry points for users in the form of free functions for submission"
+    note for round_robin_policy "Entry points for users in the form of member functions for submission"
+    note for default_backend_impl "Customize by partially specializing for specific BaseResourceType inheriting from backend_base"
+    note for round_robin_policy "Customize at this level with minimal effort inheriting from policy_base"
+
+```
+