From 5caba5ac257b27aa472901ccfd1ae6fcf780b01b Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Thu, 18 Apr 2024 18:07:59 +0200 Subject: [PATCH 1/7] Add pipeline composition RFC --- docs/rfcs/PipelineComposition.md | 73 ++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 docs/rfcs/PipelineComposition.md diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md new file mode 100644 index 000000000..910d3bae2 --- /dev/null +++ b/docs/rfcs/PipelineComposition.md @@ -0,0 +1,73 @@ +# RFC: High Level MLIR Pass Pipeline Composition + +Ivan Butygin, Renato Golin + +## Motivation + +TBD use cases from IREE, TPP + +## Proposal + +We propose a new API to build a dependencies-based graph of pass pipelines. +This API also allows a dynamic control flow between pipelines in graph, controlled by passes inside said pipelines. + +## New APIs + +### PipelineGraph +```C++ +class PipelineGraph { +public: + void registerPipeline( + StringRef name, + ArrayRef predecessors, + ArrayRef successors, + ArrayRef jumpTargets, + std::function populateFunc); + + FailureOr createPipelineSchedule(raw_ostream &errorStream) const; +}; +``` +`PipelineGraph` is main entry point for this new API. +User is adding pipelines into graph via `registerPipeline` function, each pipeline have the following properties: +* `name` - Name of thepipeline +* `predecessors` - List of names of pipelines which must be run before current pipeline. +* `successors` - List of the names of pipelines which must be run after current pipeline. +* `jumpTargets` - List of the names of pipelines to which we can dynamically jump after current pipeline. +* `populateFunc` - Callback to populate pipeline with passes. + +After user populated the graph object they must call `createPipelineSchedule` metdod to compile the resulted graph into runnable schedule. +`createPipelineSchedule` will build a DAG from pipelines dependencies provided by user, and will try to get linear execution order to satify these dependencies. + +If two pipelines doesn't have direct and indirect dependencies, order in which they will be executed is not specified, but stable. + +Implicit cycles in graph are not allowed and will result in `createPipelineSchedule` returning error. But cycles via `jumpTargets` are allowed (see later). + +Empty pipelines (i.e. pipelines without passes, when `populateFunc` does nothing) are allowed and sometimes desirable. + +One usecase is using empty pipelines as anchors for other pipelines. Let's say use wants to split hist entire compiler pipeline into 3 stages: `high`, `middle` and `low`. +They can create a two empty pipelines `high-to-middle` and `midlle-to-low` with appropriate dependencies and the use those as anchors to specify at which compiler stage insert stages which do actual work. + +### PipelineSchedule +```C++ +class PipelineSchedule { +public: + LogicalResult run(Operation *op); +}; +``` +`PipelineSchedule` object encapsulates compiled pipeline graph. Main method is `LogicalResult run(Operation *op);` which follows existing MLIR `PassManager::run`. + +### Dynamic pipeline control flow + +If pipeline has `jumpTargets` populated, it can possibly jump to one of those `jumpTargets` after finishing instead of continuing normally. + +Jump is controlled via special MLIR attribute `pipeline.jump_target`, attached to top-level op (usually `builtin.module`). +``` +builtin.module {pipeline.jump_target="Foo"} { +... +} +``` + +Passes inside pipeline can set this attribute to indicate they want compilatin flow to jump to the specific point. +After current pipeline is finished, runtime will check if module object have attribute set and if it does, jump to the selected pipeline and clear the attribute. + +Setting attribute to the value, which wasnt in `jumpTargets` for the current pipeline will result in error and abort the compilation flow. \ No newline at end of file From 668be9c84e6ee97fc72d74380b5a52fdf9145f07 Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Fri, 3 May 2024 16:09:04 +0200 Subject: [PATCH 2/7] typos --- docs/rfcs/PipelineComposition.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md index 910d3bae2..17f9a9cac 100644 --- a/docs/rfcs/PipelineComposition.md +++ b/docs/rfcs/PipelineComposition.md @@ -35,7 +35,7 @@ User is adding pipelines into graph via `registerPipeline` function, each pipeli * `jumpTargets` - List of the names of pipelines to which we can dynamically jump after current pipeline. * `populateFunc` - Callback to populate pipeline with passes. -After user populated the graph object they must call `createPipelineSchedule` metdod to compile the resulted graph into runnable schedule. +After user populated the graph object they must call `createPipelineSchedule` method to compile the resulted graph into runnable schedule. `createPipelineSchedule` will build a DAG from pipelines dependencies provided by user, and will try to get linear execution order to satify these dependencies. If two pipelines doesn't have direct and indirect dependencies, order in which they will be executed is not specified, but stable. @@ -70,4 +70,4 @@ builtin.module {pipeline.jump_target="Foo"} { Passes inside pipeline can set this attribute to indicate they want compilatin flow to jump to the specific point. After current pipeline is finished, runtime will check if module object have attribute set and if it does, jump to the selected pipeline and clear the attribute. -Setting attribute to the value, which wasnt in `jumpTargets` for the current pipeline will result in error and abort the compilation flow. \ No newline at end of file +Setting attribute to the value, which wasnt in `jumpTargets` for the current pipeline will result in error and abort the compilation flow. From 32104b2d32123a55ec94e66e456c4c99ed29ff7a Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Mon, 6 May 2024 16:55:43 +0200 Subject: [PATCH 3/7] motivation --- docs/rfcs/PipelineComposition.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md index 17f9a9cac..1760e87ff 100644 --- a/docs/rfcs/PipelineComposition.md +++ b/docs/rfcs/PipelineComposition.md @@ -2,13 +2,25 @@ Ivan Butygin, Renato Golin +## Summary + +We propose a new API to build a dependencies-based graph of pass pipelines. + ## Motivation -TBD use cases from IREE, TPP +At this point MLIR pass infrastruture only allows to define a linear sequence of passes. While this approach works +for simple cases, more complex compiler pipelines may require more control over passes execution order. + +Two main usecases we are considering: +* Extensibility. We want to have the abilty for users to extend existing pipeline by inserting theirs custom passes +into various places while reusing most of the exiting pipeline. With current approach the most common way way to achieve this is +for pipeline developer to add a fixed 'extension point' during initial pipeline design. +* Dynamic pipeline control flow. It's often required to have an ability select specific sequence of passes based on some info which +is only becomes available durin compilation process. It can be either branching to some separate sequnces of passes based on selected codegen path +or running sequence of passes in the loop until some runtime condition is reached. ## Proposal -We propose a new API to build a dependencies-based graph of pass pipelines. This API also allows a dynamic control flow between pipelines in graph, controlled by passes inside said pipelines. ## New APIs From 53f094a2c346275ed3c7f66c48a29b1f8532f53a Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Tue, 7 May 2024 15:20:24 +0200 Subject: [PATCH 4/7] add numba-mlir example --- docs/rfcs/PipelineComposition.md | 54 ++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md index 1760e87ff..9fd5b6292 100644 --- a/docs/rfcs/PipelineComposition.md +++ b/docs/rfcs/PipelineComposition.md @@ -83,3 +83,57 @@ Passes inside pipeline can set this attribute to indicate they want compilatin f After current pipeline is finished, runtime will check if module object have attribute set and if it does, jump to the selected pipeline and clear the attribute. Setting attribute to the value, which wasnt in `jumpTargets` for the current pipeline will result in error and abort the compilation flow. + + +### Pipeline examples + +Here is some examples where non-trivial pipeline dependencies are needed. + +#### Numba-mlir + +``` + frontend + | + V + cfg-to-scf + / ^ + / \ + V \ + python-to-std | + \ / + \ / + V / + numpy-to-linalg + | + V + bufferization + | + V + optimization + / \ + / \ + V \ + lower-to-gpu | + \ / + \ / + V V + lower-to-llvm +``` +In this pipeline we are lowering scalar python ops in `python-to-std` stage and +numpy ops in `numpy-to-linalg` and we may need to jump backwards to `cfg-to-scf`/`python-to-std` +multiple times to lower generated `linalg.generic` body, which are represented as +normal python function in our numpy->linalg conversion. + +Pipeline description for this will looks like (pseudocode): +``` +# pipeline format: +# name: [predecessors], [successors], [jumps] +frontend: [], [], [] +cfg-to-scf: [frontend], [optimization], [] +python-to-std: [cfg-to-scf], [optimization], [] +numpy-to-linalg: [python-to-std], [bufferization, optimization], [cfg-to-scf] +bufferization: [], [optimization], [] +optimization: [], [], [] +lower-to-gpu: [optimization], [lower-to-llvm], [] +lower-to-llvm: [optimization], [], [] +``` From 29b08c23543e786013fc58fc5a3f051fe4240db6 Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Tue, 7 May 2024 15:20:56 +0200 Subject: [PATCH 5/7] TPP TBD --- docs/rfcs/PipelineComposition.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md index 9fd5b6292..7b22caf84 100644 --- a/docs/rfcs/PipelineComposition.md +++ b/docs/rfcs/PipelineComposition.md @@ -137,3 +137,7 @@ optimization: [], [], [] lower-to-gpu: [optimization], [lower-to-llvm], [] lower-to-llvm: [optimization], [], [] ``` + +#### TPP + +TBD From a3c8cc69e55db71e56c9215db6fd98848336ceb1 Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Tue, 7 May 2024 16:02:18 +0200 Subject: [PATCH 6/7] add TPP pipeline --- docs/rfcs/PipelineComposition.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md index 7b22caf84..f7c8ac063 100644 --- a/docs/rfcs/PipelineComposition.md +++ b/docs/rfcs/PipelineComposition.md @@ -91,6 +91,8 @@ Here is some examples where non-trivial pipeline dependencies are needed. #### Numba-mlir +https://github.com/numba/numba-mlir + ``` frontend | @@ -140,4 +142,29 @@ lower-to-llvm: [optimization], [], [] #### TPP -TBD +https://github.com/plaidml/tpp-mlir +``` + frontend + / \ + V V + gpu-pipeline default-pipeline + \ / + V V + bufferization + | + V + linalg-lowering + | + V + lower-to-llvm +``` + +Pipeline will looks like: +``` +frontend: [], [], [] +gpu-pipeline: [frontend], [bufferization], [] +default-pipeline: [frontend], [bufferization], [] +bufferization: [], [linalg-lowering], [] +linalg-lowering: [], [lower-to-llvm], [] +lower-to-llvm: [], [], [] +``` From 117f1c154453c3de2b1a947a3d9fe8d855d3b5e2 Mon Sep 17 00:00:00 2001 From: Ivan Butygin Date: Fri, 17 May 2024 17:11:14 +0200 Subject: [PATCH 7/7] subgraphs, some clarifications --- docs/rfcs/PipelineComposition.md | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/docs/rfcs/PipelineComposition.md b/docs/rfcs/PipelineComposition.md index f7c8ac063..2d1d8cf57 100644 --- a/docs/rfcs/PipelineComposition.md +++ b/docs/rfcs/PipelineComposition.md @@ -29,12 +29,21 @@ This API also allows a dynamic control flow between pipelines in graph, controll ```C++ class PipelineGraph { public: - void registerPipeline( + LogicalResult registerPipeline( StringRef name, ArrayRef predecessors, ArrayRef successors, ArrayRef jumpTargets, - std::function populateFunc); + std::function populateFunc, + raw_ostream &errorStream); + + LogicalResult registerPipeline( + StringRef name, + ArrayRef predecessors, + ArrayRef successors, + ArrayRef jumpTargets, + std::function populateFunc, + raw_ostream &errorStream); FailureOr createPipelineSchedule(raw_ostream &errorStream) const; }; @@ -47,17 +56,20 @@ User is adding pipelines into graph via `registerPipeline` function, each pipeli * `jumpTargets` - List of the names of pipelines to which we can dynamically jump after current pipeline. * `populateFunc` - Callback to populate pipeline with passes. +User can either register linear sequence of passes via `OpPassManager` variant or a subgraph via `PipelineGraph`. +Registered subgraphs are isolated from the current graph and always executed as single entity (i.e. control flow can't jump into or from the middle of subgraph). + After user populated the graph object they must call `createPipelineSchedule` method to compile the resulted graph into runnable schedule. `createPipelineSchedule` will build a DAG from pipelines dependencies provided by user, and will try to get linear execution order to satify these dependencies. -If two pipelines doesn't have direct and indirect dependencies, order in which they will be executed is not specified, but stable. +If two pipelines doesn't have direct and indirect dependencies, order in which they will be executed is not specified. Implicit cycles in graph are not allowed and will result in `createPipelineSchedule` returning error. But cycles via `jumpTargets` are allowed (see later). Empty pipelines (i.e. pipelines without passes, when `populateFunc` does nothing) are allowed and sometimes desirable. One usecase is using empty pipelines as anchors for other pipelines. Let's say use wants to split hist entire compiler pipeline into 3 stages: `high`, `middle` and `low`. -They can create a two empty pipelines `high-to-middle` and `midlle-to-low` with appropriate dependencies and the use those as anchors to specify at which compiler stage insert stages which do actual work. +They can create a two empty pipelines `high-to-middle` and `middle-to-low` with appropriate dependencies and the use those as anchors to specify at which compiler stage insert stages which do actual work. ### PipelineSchedule ```C++ @@ -66,7 +78,9 @@ public: LogicalResult run(Operation *op); }; ``` -`PipelineSchedule` object encapsulates compiled pipeline graph. Main method is `LogicalResult run(Operation *op);` which follows existing MLIR `PassManager::run`. +`PipelineSchedule` object encapsulates compiled pipeline graph. +Main method is `LogicalResult run(Operation *op);` which follows existing MLIR `PassManager::run` semantics. +`run` will execute ### Dynamic pipeline control flow